* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
@ 2017-03-02 15:20 ` Laurent Pinchart
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
2 siblings, 0 replies; 28+ messages in thread
From: Laurent Pinchart @ 2017-03-02 15:20 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Jonathan Corbet, linux-doc, Mauro Carvalho Chehab,
DRI Development, Daniel Vetter
Hi Daniel,
Thank you for the patch.
On Thursday 02 Mar 2017 16:16:34 Daniel Vetter wrote:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
>
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
>
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
>
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
However, you might want to explicitly check the Sphinx version, as done in
Documentation/conf.py:
import sphinx
# Get Sphinx version
major, minor, patch = map(int, sphinx.__version__.split("."))
if major == 1 and minor > 3:
...
This will make it clearer what version we depend on, and easier to remove dead
code when we'll bump the minimum version requirements.
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Cc: Markus Heiser <markus.heiser@darmarit.de>
> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/doc-guide/hello.dot | 3 +
> Documentation/doc-guide/sphinx.rst | 90 ++++++-
> Documentation/doc-guide/svg_image.svg | 10 +
> Documentation/process/changes.rst | 7 +-
> Documentation/sphinx/kfigure.py | 444 ++++++++++++++++++++++++++++++
> 6 files changed, 550 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
--
Regards,
Laurent Pinchart
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread* [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-02 15:20 ` Laurent Pinchart
@ 2017-03-02 15:40 ` Daniel Vetter
2017-03-02 18:06 ` Mauro Carvalho Chehab
2017-03-02 21:29 ` Mauro Carvalho Chehab
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
2 siblings, 2 replies; 28+ messages in thread
From: Daniel Vetter @ 2017-03-02 15:40 UTC (permalink / raw)
To: linux-doc
Cc: Markus Heiser, Jonathan Corbet, DRI Development,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
From: Markus Heiser <markus.heiser@darmarit.de>
This patch brings scalable figure, image handling and a concept to
embed *render* markups:
* DOT (http://www.graphviz.org)
* SVG
For image handling use the 'image' replacement::
.. kernel-image:: svg_image.svg
:alt: simple SVG image
For figure handling use the 'figure' replacement::
.. kernel-figure:: svg_image.svg
:alt: simple SVG image
SVG image example
Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::
.. kernel-render:: DOT
:alt: foobar digraph
:caption: Embedded **DOT** (Graphviz) code.
digraph foo {
"bar" -> "baz";
}
The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:
* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
- Remove path detection and setup/check code for that. In
Documentation/media/Makefile we already simply use these tools,
better to have one consolidated check if we want/need one. Also
remove the convertsvg support, we require ImageMagick's convert
already in the doc build, no need for a 2nd fallback.
- Use sphinx for depency tracking, remove hand-rolled version.
- Forward stderr from dot and convert, otherwise debugging issues with
the diagrams is impossible.
v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
Implement Markus suggestion for backwards compatability with earlier
releases. Laurent reported this, running sphinx 1.3. Solution entirely
untested.
v5: Use an explicit version check (suggested by Laurent).
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Cc: Markus Heiser <markus.heiser@darmarit.de>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Markus Heiser <markus.heiser@darmarit.de>
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/conf.py | 2 +-
Documentation/doc-guide/hello.dot | 3 +
Documentation/doc-guide/sphinx.rst | 90 ++++++-
Documentation/doc-guide/svg_image.svg | 10 +
Documentation/process/changes.rst | 7 +-
Documentation/sphinx/kfigure.py | 450 ++++++++++++++++++++++++++++++++++
6 files changed, 556 insertions(+), 6 deletions(-)
create mode 100644 Documentation/doc-guide/hello.dot
create mode 100644 Documentation/doc-guide/svg_image.svg
create mode 100644 Documentation/sphinx/kfigure.py
diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4
if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
new file mode 100644
index 000000000000..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+ Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
To generate documentation, Sphinx (``sphinx-build``) must obviously be
installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
* .. _`last row`:
- column 3
+
+
+Figures & Images
+================
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+ .. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+.. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+The kernel figure (and image) directive support **DOT** formated files, see
+
+* DOT: http://graphviz.org/pdf/dotguide.pdf
+* Graphviz: http://www.graphviz.org/content/dot-language
+
+A simple example::
+
+ .. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+.. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
+``kernel-render`` directives.::
+
+ .. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+How this will be rendered depends on the installed tools. If Graphviz is
+installed, you will see an vector image. If not the raw markup is inserted as
+*literal-block*.
+
+.. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+The *render* directive has all the options known from the *figure* directive,
+plus option ``caption``. If ``caption`` has a value, a *figure* node is
+inserted. If not, a *image* node is inserted.
+
+Embedded **SVG**::
+
+ .. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
+ ...
+ </svg>
+
+.. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+ </svg>
diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
new file mode 100644
index 000000000000..5405f85b8137
--- /dev/null
+++ b/Documentation/doc-guide/svg_image.svg
@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full"
+ width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
+ <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+</svg>
diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 56ce66114665..e4f25038ef65 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
.. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
- version 3.14159265. Depending on the distribution, you may also need
- to install a series of ``texlive`` packages that provide the minimal
- set of functionalities required for ``XeLaTex`` to work.
+ version 3.14159265. Depending on the distribution, you may also need to
+ install a series of ``texlive`` packages that provide the minimal set of
+ functionalities required for ``XeLaTex`` to work. For PDF output you'll also
+ need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools
-----------
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
new file mode 100644
index 000000000000..32eab0f4cfba
--- /dev/null
+++ b/Documentation/sphinx/kfigure.py
@@ -0,0 +1,450 @@
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0103
+u"""
+ scalable figure and image handling
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ Sphinx extension which implements scalable image handling.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The build for image formats depence on image's source format and output's
+ destination format. This extension implement methods to simplify image
+ handling from the author's POV. Directives like ``kernel-figure`` implement
+ methods *to* always get the best output-format even if some tools are not
+ installed.For more details take a look at ``convert_image(...)`` which is
+ the core of all conversions.
+
+ * ``.. kernel-image``: for image handling / ``.. image::`` replacement
+
+ * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
+
+ * ``.. kernel-render``: for render markup / a concept to embed *render*
+ markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
+
+ + ``DOT``: render embedded Graphviz's **DOC**
+ + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
+ + ... *developable*
+
+ Used tools:
+
+ * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
+ available, the DOT language is inserted as literal-block.
+
+ * SVG to PDF: To generate PDF, you need at least one of this tools:
+
+ - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+
+ List of customizations:
+
+ * generate PDF from SVG / used by PDF (LaTeX) builder
+
+ * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
+ DOT: see http://www.graphviz.org/content/dot-language
+
+ """
+
+import os
+from os import path
+import subprocess
+from hashlib import sha1
+import sys
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives
+from docutils.parsers.rst.directives import images
+
+import sphinx
+
+# Get Sphinx version
+major, minor, patch = map(int, sphinx.__version__.split("."))
+if major == 1 and minor > 3:
+ # patches.Figure only landed in Sphinx 1.4
+ from sphinx.directives.patches import Figure
+else:
+ Figure = images.Figure
+
+__version__ = '1.0'
+
+# simple helper
+# -------------
+
+def mkdir(folder, mode=0o775):
+ if not path.isdir(folder):
+ os.makedirs(folder, mode)
+
+# def debug_handle(self, node): # pylint: disable=W0613
+# from linuxdoc.kernel_doc import CONSOLE
+# CONSOLE()
+
+def pass_handle(self, node): # pylint: disable=W0613
+ pass
+
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+def setup(app):
+ # image handling
+ app.add_directive("kernel-image", KernelImage)
+ app.add_node(kernel_image,
+ html = (visit_kernel_image, pass_handle),
+ latex = (visit_kernel_image, pass_handle),
+ texinfo = (visit_kernel_image, pass_handle),
+ text = (visit_kernel_image, pass_handle),
+ man = (visit_kernel_image, pass_handle), )
+
+ # figure handling
+ app.add_directive("kernel-figure", KernelFigure)
+ app.add_node(kernel_figure,
+ html = (visit_kernel_figure, pass_handle),
+ latex = (visit_kernel_figure, pass_handle),
+ texinfo = (visit_kernel_figure, pass_handle),
+ text = (visit_kernel_figure, pass_handle),
+ man = (visit_kernel_figure, pass_handle), )
+
+ # render handling
+ app.add_directive('kernel-render', KernelRender)
+ app.add_node(kernel_render,
+ html = (visit_kernel_render, pass_handle),
+ latex = (visit_kernel_render, pass_handle),
+ texinfo = (visit_kernel_render, pass_handle),
+ text = (visit_kernel_render, pass_handle),
+ man = (visit_kernel_render, pass_handle), )
+
+ return dict(
+ version = __version__,
+ parallel_read_safe = True,
+ parallel_write_safe = True
+ )
+
+# integrate conversion tools
+# --------------------------
+
+RENDER_MARKUP_EXT = {
+ # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
+ # <name> : <.ext>
+ 'DOT' : '.dot'
+ , 'SVG' : '.svg'
+}
+
+def convert_image(img_node, translator): # pylint: disable=R0912
+ """Convert an image node for the builder.
+
+ Different builder prefer different image formats, e.g. *latex* builder
+ prefer PDF while *html* builder prefer SVG format for images.
+
+ This function handles outputs image formats in depence of source the format
+ of the image and the translator's output format. This also means to
+ manipulate/update the *image* dictionary of the builder (``builder.images``)
+
+ """
+ fname, in_ext = path.splitext(path.basename(img_node['uri']))
+ src_fname = path.join(translator.builder.srcdir, img_node['uri'])
+ src_folder = path.dirname(img_node['uri'])
+ out_dir = translator.builder.outdir
+ dst_fname = None
+
+ # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
+ verbose = translator.builder.app.verbose
+ warn = translator.builder.warn
+
+ verbose('assert best format for: ' + img_node['uri'])
+
+ if in_ext == '.dot':
+
+ # ----------
+ # handle DOT
+ # ----------
+
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if translator.builder.format == 'html':
+ dst_fname = path.join(out_dir, src_folder, fname + '.svg')
+ else:
+ # all other builder formats will include DOT as raw
+ with open(src_fname, "r") as dot:
+ data = dot.read()
+ node = nodes.literal_block(data, data)
+ img_node.replace_self(node)
+
+
+ elif in_ext == '.svg':
+
+ # ----------
+ # handle SVG
+ # ----------
+
+ if translator.builder.format == 'latex':
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if dst_fname:
+ name = dst_fname[len(out_dir) + 1:]
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ img_node['uri'] = dst_fname
+ img_node['candidates'] = {'*': dst_fname}
+
+ mkdir(path.dirname(dst_fname))
+
+ if in_ext == '.dot':
+ verbose('convert DOT to: {out}/' + name)
+ dot2format(src_fname, dst_fname)
+
+ elif in_ext == '.svg':
+ verbose('convert SVG to: {out}/' + name)
+ svg2pdf(src_fname, dst_fname)
+
+def dot2format(dot_fname, out_fname):
+ """Converts DOT file to ``out_fname`` using ``dot(1)``.
+
+ * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
+ * ``out_fname`` pathname of the output file, including format extension
+
+ The *format extension* depends on the ``dot`` command (see ``man dot``
+ option ``-Txxx``). Normally you will use one of the following extensions:
+
+ - ``.ps`` for PostScript,
+ - ``.svg`` or ``svgz`` for Structured Vector Graphics,
+ - ``.fig`` for XFIG graphics and
+ - ``.png`` or ``gif`` for common bitmap graphics.
+
+ """
+ out_format = path.splitext(out_fname)[1][1:]
+ cmd = ['dot', '-T%s' % out_format, dot_fname]
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
+
+def svg2pdf(svg_fname, pdf_fname):
+ """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
+
+ Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
+ conversion. Returns ``True`` on success and ``False`` if an error occurred
+ (e.g. none of the conversion tool is available).
+
+ * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
+ * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
+
+ """
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
+
+
+# image handling
+# ---------------------
+
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
+
+class kernel_image(nodes.General, nodes.Element):
+ """Node for ``kernel-image`` directive."""
+ pass
+
+class KernelImage(images.Image):
+ u"""KernelImage directive
+
+ Earns everything from ``.. image::`` directive, except *remote URI* and
+ *glob* pattern. The KernelImage wraps a image node into a
+ kernel_image node. See ``visit_kernel_image``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s": glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = images.Image.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (image_node,) = result
+ # wrap image node into a kernel_image node / see visitors
+ node = kernel_image('', image_node)
+ return [node]
+
+# figure handling
+# ---------------------
+
+def visit_kernel_figure(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_figure`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0][0]
+ convert_image(img_node, self)
+
+class kernel_figure(nodes.General, nodes.Element):
+ """Node for ``kernel-figure`` directive."""
+
+class KernelFigure(Figure):
+ u"""KernelImage directive
+
+ Earns everything from ``.. figure::`` directive, except *remote URI* and
+ *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
+ node. See ``visit_kernel_figure``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s":'
+ ' glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = Figure.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (figure_node,) = result
+ # wrap figure node into a kernel_figure node / see visitors
+ node = kernel_figure('', figure_node)
+ return [node]
+
+
+# render handling
+# ---------------------
+
+def visit_kernel_render(self, node):
+ """Visitor of the ``kernel_render`` Node.
+
+ If rendering tools available, save the markup of the ``literal_block`` child
+ node into a file and replace the ``literal_block`` node with a new created
+ ``image`` node, pointing to the saved markup file. Afterwards, handle the
+ image child-node with the ``convert_image(...)``.
+ """
+
+ verbose = self.builder.app.verbose
+ warn = self.builder.warn
+ srclang = node.get('srclang')
+
+ verbose('visit kernel-render node lang: "%s"' % (srclang))
+
+ tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
+ if tmp_ext is None:
+ warn('kernel-render: "%s" unknow / include raw.' % (srclang))
+ return
+
+ literal_block = node[0]
+ code = literal_block.astext()
+
+ if tmp_ext:
+ hashobj = code.encode('utf-8') # str(node.attributes)
+ fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
+ tmp_fname = path.join(
+ self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
+
+ if not path.isfile(tmp_fname):
+ mkdir(path.dirname(tmp_fname))
+ with open(tmp_fname, "w") as out:
+ out.write(code)
+
+ image_node = nodes.image(node.rawsource, **node.attributes)
+ image_node['uri'] = tmp_fname
+
+ literal_block.replace_self(image_node)
+ convert_image(image_node, self)
+
+
+class kernel_render(nodes.General, nodes.Inline, nodes.Element):
+ """Node for ``kernel-render`` directive."""
+ pass
+
+class KernelRender(Figure):
+ u"""KernelRender directive
+
+ Render content by external tool. Has all the options known from the
+ *figure* directive, plus option ``caption``. If ``caption`` has a
+ value, a figure node with the *caption* is inserted. If not, a image node is
+ inserted.
+
+ The KernelRender directive wraps the text of the directive into a
+ literal_block node and wraps it into a kernel_render node. See
+ ``visit_kernel_render``.
+ """
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ # earn options from 'figure'
+ option_spec = Figure.option_spec.copy()
+ option_spec['caption'] = directives.unchanged
+
+ def run(self):
+ return [self.build_node()]
+
+ def build_node(self):
+
+ srclang = self.arguments[0].strip()
+ if srclang not in RENDER_MARKUP_EXT.keys():
+ return [self.state_machine.reporter.warning(
+ 'Unknow source language "%s", use one of: %s.' % (
+ srclang, ",".join(RENDER_MARKUP_EXT.keys())),
+ line=self.lineno)]
+
+ code = '\n'.join(self.content)
+ if not code.strip():
+ return [self.state_machine.reporter.warning(
+ 'Ignoring "%s" directive without content.' % (
+ self.name),
+ line=self.lineno)]
+
+ node = kernel_render()
+ node['alt'] = self.options.get('alt','')
+ node['srclang'] = srclang
+ literal_node = nodes.literal_block(code, code)
+ node += literal_node
+
+ caption = self.options.get('caption')
+ if caption:
+ # parse cation's content
+ parsed = nodes.Element()
+ self.state.nested_parse(
+ ViewList([caption], source=''), self.content_offset, parsed)
+ caption_node = nodes.caption(
+ parsed[0].rawsource, '', *parsed[0].children)
+ caption_node.source = parsed[0].source
+ caption_node.line = parsed[0].line
+
+ figure_node = nodes.figure('', node)
+ for k,v in self.options.items():
+ figure_node[k] = v
+ figure_node += caption_node
+
+ node = figure_node
+
+ return node
+
--
2.11.0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
@ 2017-03-02 18:06 ` Mauro Carvalho Chehab
2017-03-02 19:06 ` Markus Heiser
2017-03-02 21:29 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 18:06 UTC (permalink / raw)
To: Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 16:40:02 +0100
Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> v5: Use an explicit version check (suggested by Laurent).
Tested here with the enclosed patch. It crashed:
Exception occurred:
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
sys.stderr.write(err)
TypeError: write() argument must be str, not bytes
The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
make[1]: *** [htmldocs] Error 1
Makefile:1450: recipe for target 'htmldocs' failed
make: *** [htmldocs] Error 2
Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
Full trace:
Traceback (most recent call last):
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/util/parallel.py", line 73, in _process
ret = func(arg)
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 380, in write_process
self.write_doc(docname, doctree)
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/builders/html.py", line 448, in write_doc
self.docwriter.write(doctree, destination)
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/writers/__init__.py", line 80, in write
self.translate()
File "/home/mchehab/.local/lib/python3.5/site-packages/sphinx/writers/html.py", line 47, in translate
self.document.walkabout(visitor)
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
if child.walkabout(visitor):
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
if child.walkabout(visitor):
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
if child.walkabout(visitor):
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 166, in walkabout
visitor.dispatch_visit(self)
File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", line 1882, in dispatch_visit
return method(node)
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 302, in visit_kernel_figure
convert_image(img_node, self)
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 193, in convert_image
dot2format(src_fname, dst_fname)
File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
sys.stderr.write(err)
TypeError: write() argument must be str, not bytes
Thanks,
Mauro
--
[PATCH] docs-rst: Don't use explicit Makefile rules to build SVG and
DOT files
Now that we have an extension to handle images, use it.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
index 32663602ff25..5bd52ea1eff0 100644
--- a/Documentation/media/Makefile
+++ b/Documentation/media/Makefile
@@ -1,51 +1,6 @@
-# Rules to convert DOT and SVG to Sphinx images
-
-SRC_DIR=$(srctree)/Documentation/media
-
-DOTS = \
- uapi/v4l/pipeline.dot \
-
-IMAGES = \
- typical_media_device.svg \
- uapi/dvb/dvbstb.svg \
- uapi/v4l/bayer.svg \
- uapi/v4l/constraints.svg \
- uapi/v4l/crop.svg \
- uapi/v4l/fieldseq_bt.svg \
- uapi/v4l/fieldseq_tb.svg \
- uapi/v4l/nv12mt.svg \
- uapi/v4l/nv12mt_example.svg \
- uapi/v4l/pipeline.svg \
- uapi/v4l/selection.svg \
- uapi/v4l/subdev-image-processing-full.svg \
- uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
- uapi/v4l/subdev-image-processing-crop.svg \
- uapi/v4l/vbi_525.svg \
- uapi/v4l/vbi_625.svg \
- uapi/v4l/vbi_hsync.svg \
-
-DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
-IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
-
-IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
-IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
-
-cmd = $(echo-cmd) $(cmd_$(1))
-
-quiet_cmd_genpdf = GENPDF $2
- cmd_genpdf = convert $2 $3
-
-quiet_cmd_gendot = DOT $2
- cmd_gendot = dot -Tsvg $2 > $3
-
-%.pdf: %.svg
- @$(call cmd,genpdf,$<,$@)
-
-%.svg: %.dot
- @$(call cmd,gendot,$<,$@)
-
# Rules to convert a .h file to inline RST documentation
+SRC_DIR=$(srctree)/Documentation/media
PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
UAPI = $(srctree)/include/uapi/linux
KAPI = $(srctree)/include/linux
diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
index 8f7490c9a8ef..5562fba1d51d 100644
--- a/Documentation/media/intro.rst
+++ b/Documentation/media/intro.rst
@@ -13,8 +13,8 @@ A typical media device hardware is shown at :ref:`typical_media_device`.
.. _typical_media_device:
-.. figure:: typical_media_device.*
- :alt: typical_media_device.pdf / typical_media_device.svg
+.. kernel-figure:: typical_media_device.svg
+ :alt: typical_media_device.svg
:align: center
Typical Media Device
diff --git a/Documentation/media/uapi/dvb/intro.rst b/Documentation/media/uapi/dvb/intro.rst
index 2ed5c23102b4..0e76067a5b58 100644
--- a/Documentation/media/uapi/dvb/intro.rst
+++ b/Documentation/media/uapi/dvb/intro.rst
@@ -55,8 +55,8 @@ Overview
.. _stb_components:
-.. figure:: dvbstb.*
- :alt: dvbstb.pdf / dvbstb.svg
+.. kernel-figure:: dvbstb.svg
+ :alt: dvbstb.svg
:align: center
Components of a DVB card/STB
diff --git a/Documentation/media/uapi/v4l/crop.rst b/Documentation/media/uapi/v4l/crop.rst
index be58894c9c89..adea29b14135 100644
--- a/Documentation/media/uapi/v4l/crop.rst
+++ b/Documentation/media/uapi/v4l/crop.rst
@@ -53,8 +53,8 @@ Cropping Structures
.. _crop-scale:
-.. figure:: crop.*
- :alt: crop.pdf / crop.svg
+.. kernel-figure:: crop.svg
+ :alt: crop.svg
:align: center
Image Cropping, Insertion and Scaling
diff --git a/Documentation/media/uapi/v4l/dev-raw-vbi.rst b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
index baf5f2483927..f761d3a93783 100644
--- a/Documentation/media/uapi/v4l/dev-raw-vbi.rst
+++ b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
@@ -221,8 +221,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
.. _vbi-hsync:
-.. figure:: vbi_hsync.*
- :alt: vbi_hsync.pdf / vbi_hsync.svg
+.. kernel-figure:: vbi_hsync.svg
+ :alt: vbi_hsync.svg
:align: center
**Figure 4.1. Line synchronization**
@@ -230,8 +230,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
.. _vbi-525:
-.. figure:: vbi_525.*
- :alt: vbi_525.pdf / vbi_525.svg
+.. kernel-figure:: vbi_525.svg
+ :alt: vbi_525.svg
:align: center
**Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
@@ -240,8 +240,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
.. _vbi-625:
-.. figure:: vbi_625.*
- :alt: vbi_625.pdf / vbi_625.svg
+.. kernel-figure:: vbi_625.svg
+ :alt: vbi_625.svg
:align: center
**Figure 4.3. ITU-R 625 line numbering**
diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
index cd2870180208..0a54e4d95013 100644
--- a/Documentation/media/uapi/v4l/dev-subdev.rst
+++ b/Documentation/media/uapi/v4l/dev-subdev.rst
@@ -99,8 +99,8 @@ the video sensor and the host image processing hardware.
.. _pipeline-scaling:
-.. figure:: pipeline.*
- :alt: pipeline.pdf / pipeline.svg
+.. kernel-figure:: pipeline.dot
+ :alt: pipeline.dot
:align: center
Image Format Negotiation on Pipelines
@@ -404,8 +404,8 @@ selection will refer to the sink pad format dimensions instead.
.. _subdev-image-processing-crop:
-.. figure:: subdev-image-processing-crop.*
- :alt: subdev-image-processing-crop.pdf / subdev-image-processing-crop.svg
+.. kernel-figure:: subdev-image-processing-crop.svg
+ :alt: subdev-image-processing-crop.svg
:align: center
**Figure 4.5. Image processing in subdevs: simple crop example**
@@ -421,8 +421,8 @@ pad.
.. _subdev-image-processing-scaling-multi-source:
-.. figure:: subdev-image-processing-scaling-multi-source.*
- :alt: subdev-image-processing-scaling-multi-source.pdf / subdev-image-processing-scaling-multi-source.svg
+.. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
+ :alt: subdev-image-processing-scaling-multi-source.svg
:align: center
**Figure 4.6. Image processing in subdevs: scaling with multiple sources**
@@ -437,8 +437,8 @@ an area at location specified by the source crop rectangle from it.
.. _subdev-image-processing-full:
-.. figure:: subdev-image-processing-full.*
- :alt: subdev-image-processing-full.pdf / subdev-image-processing-full.svg
+.. kernel-figure:: subdev-image-processing-full.svg
+ :alt: subdev-image-processing-full.svg
:align: center
**Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
diff --git a/Documentation/media/uapi/v4l/field-order.rst b/Documentation/media/uapi/v4l/field-order.rst
index e05fb1041363..7a1a0ac8e37a 100644
--- a/Documentation/media/uapi/v4l/field-order.rst
+++ b/Documentation/media/uapi/v4l/field-order.rst
@@ -141,8 +141,8 @@ enum v4l2_field
Field Order, Top Field First Transmitted
========================================
-.. figure:: fieldseq_tb.*
- :alt: fieldseq_tb.pdf / fieldseq_tb.svg
+.. kernel-figure:: fieldseq_tb.svg
+ :alt: fieldseq_tb.svg
:align: center
@@ -151,7 +151,7 @@ Field Order, Top Field First Transmitted
Field Order, Bottom Field First Transmitted
===========================================
-.. figure:: fieldseq_bt.*
- :alt: fieldseq_bt.pdf / fieldseq_bt.svg
+.. kernel-figure:: fieldseq_bt.svg
+ :alt: fieldseq_bt.svg
:align: center
diff --git a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
index 32d0c8743460..a4d64edc6200 100644
--- a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
@@ -33,8 +33,8 @@ Layout of macroblocks in memory is presented in the following figure.
.. _nv12mt:
-.. figure:: nv12mt.*
- :alt: nv12mt.pdf / nv12mt.svg
+.. kernel-figure:: nv12mt.svg
+ :alt: nv12mt.svg
:align: center
V4L2_PIX_FMT_NV12MT macroblock Z shape memory layout
@@ -50,8 +50,8 @@ interleaved. Height of the buffer is aligned to 32.
.. _nv12mt_ex:
-.. figure:: nv12mt_example.*
- :alt: nv12mt_example.pdf / nv12mt_example.svg
+.. kernel-figure:: nv12mt_example.svg
+ :alt: nv12mt_example.svg
:align: center
Example V4L2_PIX_FMT_NV12MT memory layout of macroblocks
diff --git a/Documentation/media/uapi/v4l/selection-api-003.rst b/Documentation/media/uapi/v4l/selection-api-003.rst
index 21686f93c38f..04e9384e1be8 100644
--- a/Documentation/media/uapi/v4l/selection-api-003.rst
+++ b/Documentation/media/uapi/v4l/selection-api-003.rst
@@ -7,8 +7,8 @@ Selection targets
.. _sel-targets-capture:
-.. figure:: selection.*
- :alt: selection.pdf / selection.svg
+.. kernel-figure:: selection.svg
+ :alt: selection.svg
:align: center
Cropping and composing targets
diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
index d6152c907b8b..4fbf2e29e653 100644
--- a/Documentation/media/uapi/v4l/subdev-formats.rst
+++ b/Documentation/media/uapi/v4l/subdev-formats.rst
@@ -1514,8 +1514,8 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
.. _bayer-patterns:
-.. figure:: bayer.*
- :alt: bayer.pdf / bayer.svg
+.. figure:: bayer.svg
+ :alt: bayer.svg
:align: center
**Figure 4.8 Bayer Patterns**
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 18:06 ` Mauro Carvalho Chehab
@ 2017-03-02 19:06 ` Markus Heiser
2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:52 ` Daniel Vetter
0 siblings, 2 replies; 28+ messages in thread
From: Markus Heiser @ 2017-03-02 19:06 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, DRI Development,
Linux Doc Mailing List, Laurent Pinchart, Daniel Vetter
Hi Mauro,
> Tested here with the enclosed patch.
great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> It crashed:
> Exception occurred:
> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> sys.stderr.write(err)
> TypeError: write() argument must be str, not bytes
> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> Please also report this if it was a user error, so that a better error message can be provided next time.
> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> make[1]: *** [htmldocs] Error 1
> Makefile:1450: recipe for target 'htmldocs' failed
> make: *** [htmldocs] Error 2
>
> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
I guess that the dot command writes something to stderr. This is captured
by the extension and printed to stderr ...
+def dot2format(dot_fname, out_fname):
...
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> sys.stderr.write(err)
> TypeError: write() argument must be str, not bytes
Do we need this stderr output? For a first test, uncomment the
"sys.stderr.write(err)“ in line 222. Or, if we really need the
stderr, try:
- sys.stderr.write(err)
+ sys.stderr.write(str(err))
I this fixes, there is another "sys.stderr.write(err)" in
func svg2pdf(..) which should also fixed ….
+def svg2pdf(svg_fname, pdf_fname):
...
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
- sys.stderr.write(err)
+ sys.stderr.write(str(err))
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
Thanks!
-- Markus --
>
>
> Thanks,
> Mauro
>
> --
>
> [PATCH] docs-rst: Don't use explicit Makefile rules to build SVG and
> DOT files
>
> Now that we have an extension to handle images, use it.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
>
> diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
> index 32663602ff25..5bd52ea1eff0 100644
> --- a/Documentation/media/Makefile
> +++ b/Documentation/media/Makefile
> @@ -1,51 +1,6 @@
> -# Rules to convert DOT and SVG to Sphinx images
> -
> -SRC_DIR=$(srctree)/Documentation/media
> -
> -DOTS = \
> - uapi/v4l/pipeline.dot \
> -
> -IMAGES = \
> - typical_media_device.svg \
> - uapi/dvb/dvbstb.svg \
> - uapi/v4l/bayer.svg \
> - uapi/v4l/constraints.svg \
> - uapi/v4l/crop.svg \
> - uapi/v4l/fieldseq_bt.svg \
> - uapi/v4l/fieldseq_tb.svg \
> - uapi/v4l/nv12mt.svg \
> - uapi/v4l/nv12mt_example.svg \
> - uapi/v4l/pipeline.svg \
> - uapi/v4l/selection.svg \
> - uapi/v4l/subdev-image-processing-full.svg \
> - uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
> - uapi/v4l/subdev-image-processing-crop.svg \
> - uapi/v4l/vbi_525.svg \
> - uapi/v4l/vbi_625.svg \
> - uapi/v4l/vbi_hsync.svg \
> -
> -DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
> -IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
> -
> -IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
> -IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
> -
> -cmd = $(echo-cmd) $(cmd_$(1))
> -
> -quiet_cmd_genpdf = GENPDF $2
> - cmd_genpdf = convert $2 $3
> -
> -quiet_cmd_gendot = DOT $2
> - cmd_gendot = dot -Tsvg $2 > $3
> -
> -%.pdf: %.svg
> - @$(call cmd,genpdf,$<,$@)
> -
> -%.svg: %.dot
> - @$(call cmd,gendot,$<,$@)
> -
> # Rules to convert a .h file to inline RST documentation
>
> +SRC_DIR=$(srctree)/Documentation/media
> PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
> UAPI = $(srctree)/include/uapi/linux
> KAPI = $(srctree)/include/linux
> diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
> index 8f7490c9a8ef..5562fba1d51d 100644
> --- a/Documentation/media/intro.rst
> +++ b/Documentation/media/intro.rst
> @@ -13,8 +13,8 @@ A typical media device hardware is shown at :ref:`typical_media_device`.
>
> .. _typical_media_device:
>
> -.. figure:: typical_media_device.*
> - :alt: typical_media_device.pdf / typical_media_device.svg
> +.. kernel-figure:: typical_media_device.svg
> + :alt: typical_media_device.svg
> :align: center
>
> Typical Media Device
> diff --git a/Documentation/media/uapi/dvb/intro.rst b/Documentation/media/uapi/dvb/intro.rst
> index 2ed5c23102b4..0e76067a5b58 100644
> --- a/Documentation/media/uapi/dvb/intro.rst
> +++ b/Documentation/media/uapi/dvb/intro.rst
> @@ -55,8 +55,8 @@ Overview
>
> .. _stb_components:
>
> -.. figure:: dvbstb.*
> - :alt: dvbstb.pdf / dvbstb.svg
> +.. kernel-figure:: dvbstb.svg
> + :alt: dvbstb.svg
> :align: center
>
> Components of a DVB card/STB
> diff --git a/Documentation/media/uapi/v4l/crop.rst b/Documentation/media/uapi/v4l/crop.rst
> index be58894c9c89..adea29b14135 100644
> --- a/Documentation/media/uapi/v4l/crop.rst
> +++ b/Documentation/media/uapi/v4l/crop.rst
> @@ -53,8 +53,8 @@ Cropping Structures
>
> .. _crop-scale:
>
> -.. figure:: crop.*
> - :alt: crop.pdf / crop.svg
> +.. kernel-figure:: crop.svg
> + :alt: crop.svg
> :align: center
>
> Image Cropping, Insertion and Scaling
> diff --git a/Documentation/media/uapi/v4l/dev-raw-vbi.rst b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
> index baf5f2483927..f761d3a93783 100644
> --- a/Documentation/media/uapi/v4l/dev-raw-vbi.rst
> +++ b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
> @@ -221,8 +221,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>
> .. _vbi-hsync:
>
> -.. figure:: vbi_hsync.*
> - :alt: vbi_hsync.pdf / vbi_hsync.svg
> +.. kernel-figure:: vbi_hsync.svg
> + :alt: vbi_hsync.svg
> :align: center
>
> **Figure 4.1. Line synchronization**
> @@ -230,8 +230,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>
> .. _vbi-525:
>
> -.. figure:: vbi_525.*
> - :alt: vbi_525.pdf / vbi_525.svg
> +.. kernel-figure:: vbi_525.svg
> + :alt: vbi_525.svg
> :align: center
>
> **Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
> @@ -240,8 +240,8 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>
> .. _vbi-625:
>
> -.. figure:: vbi_625.*
> - :alt: vbi_625.pdf / vbi_625.svg
> +.. kernel-figure:: vbi_625.svg
> + :alt: vbi_625.svg
> :align: center
>
> **Figure 4.3. ITU-R 625 line numbering**
> diff --git a/Documentation/media/uapi/v4l/dev-subdev.rst b/Documentation/media/uapi/v4l/dev-subdev.rst
> index cd2870180208..0a54e4d95013 100644
> --- a/Documentation/media/uapi/v4l/dev-subdev.rst
> +++ b/Documentation/media/uapi/v4l/dev-subdev.rst
> @@ -99,8 +99,8 @@ the video sensor and the host image processing hardware.
>
> .. _pipeline-scaling:
>
> -.. figure:: pipeline.*
> - :alt: pipeline.pdf / pipeline.svg
> +.. kernel-figure:: pipeline.dot
> + :alt: pipeline.dot
> :align: center
>
> Image Format Negotiation on Pipelines
> @@ -404,8 +404,8 @@ selection will refer to the sink pad format dimensions instead.
>
> .. _subdev-image-processing-crop:
>
> -.. figure:: subdev-image-processing-crop.*
> - :alt: subdev-image-processing-crop.pdf / subdev-image-processing-crop.svg
> +.. kernel-figure:: subdev-image-processing-crop.svg
> + :alt: subdev-image-processing-crop.svg
> :align: center
>
> **Figure 4.5. Image processing in subdevs: simple crop example**
> @@ -421,8 +421,8 @@ pad.
>
> .. _subdev-image-processing-scaling-multi-source:
>
> -.. figure:: subdev-image-processing-scaling-multi-source.*
> - :alt: subdev-image-processing-scaling-multi-source.pdf / subdev-image-processing-scaling-multi-source.svg
> +.. kernel-figure:: subdev-image-processing-scaling-multi-source.svg
> + :alt: subdev-image-processing-scaling-multi-source.svg
> :align: center
>
> **Figure 4.6. Image processing in subdevs: scaling with multiple sources**
> @@ -437,8 +437,8 @@ an area at location specified by the source crop rectangle from it.
>
> .. _subdev-image-processing-full:
>
> -.. figure:: subdev-image-processing-full.*
> - :alt: subdev-image-processing-full.pdf / subdev-image-processing-full.svg
> +.. kernel-figure:: subdev-image-processing-full.svg
> + :alt: subdev-image-processing-full.svg
> :align: center
>
> **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
> diff --git a/Documentation/media/uapi/v4l/field-order.rst b/Documentation/media/uapi/v4l/field-order.rst
> index e05fb1041363..7a1a0ac8e37a 100644
> --- a/Documentation/media/uapi/v4l/field-order.rst
> +++ b/Documentation/media/uapi/v4l/field-order.rst
> @@ -141,8 +141,8 @@ enum v4l2_field
> Field Order, Top Field First Transmitted
> ========================================
>
> -.. figure:: fieldseq_tb.*
> - :alt: fieldseq_tb.pdf / fieldseq_tb.svg
> +.. kernel-figure:: fieldseq_tb.svg
> + :alt: fieldseq_tb.svg
> :align: center
>
>
> @@ -151,7 +151,7 @@ Field Order, Top Field First Transmitted
> Field Order, Bottom Field First Transmitted
> ===========================================
>
> -.. figure:: fieldseq_bt.*
> - :alt: fieldseq_bt.pdf / fieldseq_bt.svg
> +.. kernel-figure:: fieldseq_bt.svg
> + :alt: fieldseq_bt.svg
> :align: center
>
> diff --git a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
> index 32d0c8743460..a4d64edc6200 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-nv12mt.rst
> @@ -33,8 +33,8 @@ Layout of macroblocks in memory is presented in the following figure.
>
> .. _nv12mt:
>
> -.. figure:: nv12mt.*
> - :alt: nv12mt.pdf / nv12mt.svg
> +.. kernel-figure:: nv12mt.svg
> + :alt: nv12mt.svg
> :align: center
>
> V4L2_PIX_FMT_NV12MT macroblock Z shape memory layout
> @@ -50,8 +50,8 @@ interleaved. Height of the buffer is aligned to 32.
>
> .. _nv12mt_ex:
>
> -.. figure:: nv12mt_example.*
> - :alt: nv12mt_example.pdf / nv12mt_example.svg
> +.. kernel-figure:: nv12mt_example.svg
> + :alt: nv12mt_example.svg
> :align: center
>
> Example V4L2_PIX_FMT_NV12MT memory layout of macroblocks
> diff --git a/Documentation/media/uapi/v4l/selection-api-003.rst b/Documentation/media/uapi/v4l/selection-api-003.rst
> index 21686f93c38f..04e9384e1be8 100644
> --- a/Documentation/media/uapi/v4l/selection-api-003.rst
> +++ b/Documentation/media/uapi/v4l/selection-api-003.rst
> @@ -7,8 +7,8 @@ Selection targets
>
> .. _sel-targets-capture:
>
> -.. figure:: selection.*
> - :alt: selection.pdf / selection.svg
> +.. kernel-figure:: selection.svg
> + :alt: selection.svg
> :align: center
>
> Cropping and composing targets
> diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
> index d6152c907b8b..4fbf2e29e653 100644
> --- a/Documentation/media/uapi/v4l/subdev-formats.rst
> +++ b/Documentation/media/uapi/v4l/subdev-formats.rst
> @@ -1514,8 +1514,8 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
>
> .. _bayer-patterns:
>
> -.. figure:: bayer.*
> - :alt: bayer.pdf / bayer.svg
> +.. figure:: bayer.svg
> + :alt: bayer.svg
> :align: center
>
> **Figure 4.8 Bayer Patterns**
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:06 ` Markus Heiser
@ 2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:56 ` Mauro Carvalho Chehab
2017-03-02 21:16 ` Markus Heiser
2017-03-02 19:52 ` Daniel Vetter
1 sibling, 2 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 19:34 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Linux Doc Mailing List, DRI Development,
Jonathan Corbet, Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 20:06:39 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Hi Mauro,
>
> > Tested here with the enclosed patch.
>
> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
>
> > It crashed:
> > Exception occurred:
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > Please also report this if it was a user error, so that a better error message can be provided next time.
> > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > make[1]: *** [htmldocs] Error 1
> > Makefile:1450: recipe for target 'htmldocs' failed
> > make: *** [htmldocs] Error 2
> >
> > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
>
> I guess that the dot command writes something to stderr. This is captured
> by the extension and printed to stderr ...
>
> +def dot2format(dot_fname, out_fname):
> ...
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
>
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
>
> Do we need this stderr output? For a first test, uncomment the
> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> stderr, try:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
Yes, this fixed. I actually did:
- sys.stderr.write(err)
+ sys.stderr.write(str(err))
+ sys.stderr.write("\n")
It is now printing:
b''
I added the \n print to avoid it to be mixed with the "writing output"
prints.
No idea how to make sense from it - but clearly, the error report
logic require some care ;-)
Thanks,
Mauro
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:34 ` Mauro Carvalho Chehab
@ 2017-03-02 19:56 ` Mauro Carvalho Chehab
2017-03-02 21:16 ` Markus Heiser
1 sibling, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 19:56 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Linux Doc Mailing List, DRI Development,
Jonathan Corbet, Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 16:34:22 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 20:06:39 +0100
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
> > Hi Mauro,
> >
> > > Tested here with the enclosed patch.
> >
> > great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> >
> > > It crashed:
> > > Exception occurred:
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > make[1]: *** [htmldocs] Error 1
> > > Makefile:1450: recipe for target 'htmldocs' failed
> > > make: *** [htmldocs] Error 2
> > >
> > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> >
> > I guess that the dot command writes something to stderr. This is captured
> > by the extension and printed to stderr ...
> >
> > +def dot2format(dot_fname, out_fname):
> > ...
> > + exit_code = 42
> > + with open(out_fname, "w") as out:
> > + p = subprocess.Popen(
> > + cmd, stdout = out, stderr = subprocess.PIPE )
> > + nil, err = p.communicate()
> > +
> > + sys.stderr.write(err)
> > +
> > + exit_code = p.returncode
> > + out.flush()
> > + return bool(exit_code == 0)
> >
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> >
> > Do we need this stderr output? For a first test, uncomment the
> > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > stderr, try:
> >
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
>
> Yes, this fixed. I actually did:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
> + sys.stderr.write("\n")
>
> It is now printing:
> b''
>
> I added the \n print to avoid it to be mixed with the "writing output"
> prints.
>
> No idea how to make sense from it - but clearly, the error report
> logic require some care ;-)
I'm not a Python programmer, but googling about the right syntax for
p.communicate(), I suspect that the fix should be similar to this code
snippet below.
Without the if, this code:
sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
prints:
Error:None
So, I guess the code below is ok.
Markus, please check.
Daniel, Feel free to merge it with the original patch if OK.
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..b154c5f17752 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,12 @@ def dot2format(dot_fname, out_fname):
with open(out_fname, "w") as out:
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
+
out.flush()
return bool(exit_code == 0)
@@ -239,11 +240,12 @@ def svg2pdf(svg_fname, pdf_fname):
cmd = [convert_cmd, svg_fname, pdf_fname]
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
+
return bool(exit_code == 0)
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:34 ` Mauro Carvalho Chehab
2017-03-02 19:56 ` Mauro Carvalho Chehab
@ 2017-03-02 21:16 ` Markus Heiser
2017-03-02 21:22 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 28+ messages in thread
From: Markus Heiser @ 2017-03-02 21:16 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Jonathan Corbet, Daniel Vetter, DRI Development,
Linux Doc Mailing List, Laurent Pinchart, Daniel Vetter
> Am 02.03.2017 um 20:34 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>
> Em Thu, 2 Mar 2017 20:06:39 +0100
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
>> Hi Mauro,
>>
>>> Tested here with the enclosed patch.
>>
>> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
>>
>>> It crashed:
>>> Exception occurred:
>>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
>>> sys.stderr.write(err)
>>> TypeError: write() argument must be str, not bytes
>>> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
>>> Please also report this if it was a user error, so that a better error message can be provided next time.
>>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>>> make[1]: *** [htmldocs] Error 1
>>> Makefile:1450: recipe for target 'htmldocs' failed
>>> make: *** [htmldocs] Error 2
>>>
>>> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
>>
>> I guess that the dot command writes something to stderr. This is captured
>> by the extension and printed to stderr ...
>>
>> +def dot2format(dot_fname, out_fname):
>> ...
>> + exit_code = 42
>> + with open(out_fname, "w") as out:
>> + p = subprocess.Popen(
>> + cmd, stdout = out, stderr = subprocess.PIPE )
>> + nil, err = p.communicate()
>> +
>> + sys.stderr.write(err)
>> +
>> + exit_code = p.returncode
>> + out.flush()
>> + return bool(exit_code == 0)
>>
>>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
>>> sys.stderr.write(err)
>>> TypeError: write() argument must be str, not bytes
>>
>> Do we need this stderr output? For a first test, uncomment the
>> "sys.stderr.write(err)“ in line 222. Or, if we really need the
>> stderr, try:
>>
>> - sys.stderr.write(err)
>> + sys.stderr.write(str(err))
>
> Yes, this fixed. I actually did:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
> + sys.stderr.write("\n")
>
> It is now printing:
> b''
>
> I added the \n print to avoid it to be mixed with the "writing output"
> prints.
> No idea how to make sense from it - but clearly, the error report
> logic require some care ;-)
Aargh, I’am a idiot ... I guess 'sys.stderr.write(err)‘ is a artefact
of my development, simply drop it and the subprocess.PIPE of stderr
also.
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
- cmd, stdout = out, stderr = subprocess.PIPE )
+ cmd, stdout = out)
+ nil, err = p.communicate()
-
- sys.stderr.write(err)
-
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
I can’t test it ATM, but without redirect stderr, the stderr
of the parent process is inherited.
https://docs.python.org/3.6/library/subprocess.html#popen-constructor
The Popen.communicate() always returns a tuple (stdout_data, stderr_data)
with above the tuple is always (None, None).
https://docs.python.org/3.6/library/subprocess.html#subprocess.Popen.communicate
"""to get anything other than None in the result tuple,
you need to give stdout=PIPE and/or stderr=PIPE too."""
Sorry, that I made all this mistakes, but „here“ I have only mail
and web, no dev-env and I miss my emacs ;)
If the suggestion above does not work, I have to investigate
more time next weekend.
-- Markus --
>
> Thanks,
> Mauro
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:16 ` Markus Heiser
@ 2017-03-02 21:22 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:22 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Linux Doc Mailing List, DRI Development,
Jonathan Corbet, Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 22:16:49 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> > Am 02.03.2017 um 20:34 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >
> > Em Thu, 2 Mar 2017 20:06:39 +0100
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >
> >> Hi Mauro,
> >>
> >>> Tested here with the enclosed patch.
> >>
> >> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> >>
> >>> It crashed:
> >>> Exception occurred:
> >>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> >>> sys.stderr.write(err)
> >>> TypeError: write() argument must be str, not bytes
> >>> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> >>> Please also report this if it was a user error, so that a better error message can be provided next time.
> >>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> >>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >>> make[1]: *** [htmldocs] Error 1
> >>> Makefile:1450: recipe for target 'htmldocs' failed
> >>> make: *** [htmldocs] Error 2
> >>>
> >>> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> >>
> >> I guess that the dot command writes something to stderr. This is captured
> >> by the extension and printed to stderr ...
> >>
> >> +def dot2format(dot_fname, out_fname):
> >> ...
> >> + exit_code = 42
> >> + with open(out_fname, "w") as out:
> >> + p = subprocess.Popen(
> >> + cmd, stdout = out, stderr = subprocess.PIPE )
> >> + nil, err = p.communicate()
> >> +
> >> + sys.stderr.write(err)
> >> +
> >> + exit_code = p.returncode
> >> + out.flush()
> >> + return bool(exit_code == 0)
> >>
> >>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> >>> sys.stderr.write(err)
> >>> TypeError: write() argument must be str, not bytes
> >>
> >> Do we need this stderr output? For a first test, uncomment the
> >> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> >> stderr, try:
> >>
> >> - sys.stderr.write(err)
> >> + sys.stderr.write(str(err))
> >
> > Yes, this fixed. I actually did:
> >
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
> > + sys.stderr.write("\n")
> >
> > It is now printing:
> > b''
> >
> > I added the \n print to avoid it to be mixed with the "writing output"
> > prints.
> > No idea how to make sense from it - but clearly, the error report
> > logic require some care ;-)
>
>
> Aargh, I’am a idiot ... I guess 'sys.stderr.write(err)‘ is a artefact
> of my development, simply drop it and the subprocess.PIPE of stderr
> also.
>
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> - cmd, stdout = out, stderr = subprocess.PIPE )
> + cmd, stdout = out)
> + nil, err = p.communicate()
> -
> - sys.stderr.write(err)
> -
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
>
> I can’t test it ATM, but without redirect stderr, the stderr
> of the parent process is inherited.
>
> https://docs.python.org/3.6/library/subprocess.html#popen-constructor
>
> The Popen.communicate() always returns a tuple (stdout_data, stderr_data)
> with above the tuple is always (None, None).
>
> https://docs.python.org/3.6/library/subprocess.html#subprocess.Popen.communicate
>
> """to get anything other than None in the result tuple,
> you need to give stdout=PIPE and/or stderr=PIPE too."""
>
> Sorry, that I made all this mistakes, but „here“ I have only mail
> and web, no dev-env and I miss my emacs ;)
>
> If the suggestion above does not work, I have to investigate
> more time next weekend.
Hmm... I would be more verbose on output the error code, printing from
where the error came, e. g. printing a message like:
Error #0 when calling dot for '/devel/v4l/patchwork/Documentation/media/uapi/v4l/pipeline.dot': None
As on this patch:
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..a366a89f4f98 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
with open(out_fname, "w") as out:
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code != 0:
+ sys.stderr.write("Error #%d when calling dot for '%s': %s\n" % (exit_code, dot_fname, repr(err[0])))
+
out.flush()
return bool(exit_code == 0)
@@ -239,11 +241,13 @@ def svg2pdf(svg_fname, pdf_fname):
cmd = [convert_cmd, svg_fname, pdf_fname]
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code != 0:
+ sys.stderr.write("Error #%d when calling convert for '%s': %s\n" % (exit_code, dot_fname, repr(err[0])))
+
return bool(exit_code == 0)
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:06 ` Markus Heiser
2017-03-02 19:34 ` Mauro Carvalho Chehab
@ 2017-03-02 19:52 ` Daniel Vetter
2017-03-02 20:04 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 28+ messages in thread
From: Daniel Vetter @ 2017-03-02 19:52 UTC (permalink / raw)
To: Markus Heiser
Cc: Jonathan Corbet, Daniel Vetter, Linux Doc Mailing List,
Mauro Carvalho Chehab, DRI Development, Daniel Vetter,
Laurent Pinchart
On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> Hi Mauro,
>
> > Tested here with the enclosed patch.
>
> great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
>
> > It crashed:
> > Exception occurred:
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > Please also report this if it was a user error, so that a better error message can be provided next time.
> > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > make[1]: *** [htmldocs] Error 1
> > Makefile:1450: recipe for target 'htmldocs' failed
> > make: *** [htmldocs] Error 2
> >
> > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
>
> I guess that the dot command writes something to stderr. This is captured
> by the extension and printed to stderr ...
>
> +def dot2format(dot_fname, out_fname):
> ...
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
>
> > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
>
> Do we need this stderr output? For a first test, uncomment the
> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> stderr, try:
>
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
>
> I this fixes, there is another "sys.stderr.write(err)" in
> func svg2pdf(..) which should also fixed ….
>
> +def svg2pdf(svg_fname, pdf_fname):
> ...
> + cmd = [convert_cmd, svg_fname, pdf_fname]
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> - sys.stderr.write(err)
> + sys.stderr.write(str(err))
> +
> + exit_code = p.returncode
> + return bool(exit_code == 0)
Yes, I very much want stderr to be forward. Without that you don't see
error output from dot or convert, and that makes it impossible to debug
anything. If I want a direct forwarding of the bytes, how should I do this
in python? Capturing stderr and then re-dumping it is kinda silly ...
Note that I copied this pattern from the kernel-doc extension, seems to
have worked there.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 19:52 ` Daniel Vetter
@ 2017-03-02 20:04 ` Mauro Carvalho Chehab
2017-03-02 20:54 ` Mauro Carvalho Chehab
0 siblings, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 20:04 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Daniel Vetter, Linux Doc Mailing List,
DRI Development, Jonathan Corbet, Jani Nikula, Laurent Pinchart,
Daniel Vetter
Em Thu, 2 Mar 2017 20:52:59 +0100
Daniel Vetter <daniel@ffwll.ch> escreveu:
> On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> > Hi Mauro,
> >
> > > Tested here with the enclosed patch.
> >
> > great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> >
> > > It crashed:
> > > Exception occurred:
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > make[1]: *** [htmldocs] Error 1
> > > Makefile:1450: recipe for target 'htmldocs' failed
> > > make: *** [htmldocs] Error 2
> > >
> > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> >
> > I guess that the dot command writes something to stderr. This is captured
> > by the extension and printed to stderr ...
> >
> > +def dot2format(dot_fname, out_fname):
> > ...
> > + exit_code = 42
> > + with open(out_fname, "w") as out:
> > + p = subprocess.Popen(
> > + cmd, stdout = out, stderr = subprocess.PIPE )
> > + nil, err = p.communicate()
> > +
> > + sys.stderr.write(err)
> > +
> > + exit_code = p.returncode
> > + out.flush()
> > + return bool(exit_code == 0)
> >
> > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> >
> > Do we need this stderr output? For a first test, uncomment the
> > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > stderr, try:
> >
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
> >
> > I this fixes, there is another "sys.stderr.write(err)" in
> > func svg2pdf(..) which should also fixed ….
> >
> > +def svg2pdf(svg_fname, pdf_fname):
> > ...
> > + cmd = [convert_cmd, svg_fname, pdf_fname]
> > + p = subprocess.Popen(
> > + cmd, stdout = out, stderr = subprocess.PIPE )
> > + nil, err = p.communicate()
> > +
> > - sys.stderr.write(err)
> > + sys.stderr.write(str(err))
> > +
> > + exit_code = p.returncode
> > + return bool(exit_code == 0)
>
> Yes, I very much want stderr to be forward.
Yes, error report is required.
> Without that you don't see
> error output from dot or convert, and that makes it impossible to debug
> anything. If I want a direct forwarding of the bytes, how should I do this
> in python? Capturing stderr and then re-dumping it is kinda silly ...
Markus or some other Python programmer may help us with that.
> Note that I copied this pattern from the kernel-doc extension, seems to
> have worked there.
Maybe it is broken there too then, or this is another python API
that changed over time. Here, I'm testing with:
python3-3.5.2-4.fc25.x86_64
>From here:
https://docs.python.org/2/library/subprocess.html
communicate returns a tuple.
I used repr(p.communicate()[0], on the code snippet I sent,
as I copied from an example that I found at:
https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr
Thanks,
Mauro
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 20:04 ` Mauro Carvalho Chehab
@ 2017-03-02 20:54 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 20:54 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Daniel Vetter, Linux Doc Mailing List,
DRI Development, Jonathan Corbet, Jani Nikula, Laurent Pinchart,
Daniel Vetter
Em Thu, 2 Mar 2017 17:04:01 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 20:52:59 +0100
> Daniel Vetter <daniel@ffwll.ch> escreveu:
>
> > On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> > > Hi Mauro,
> > >
> > > > Tested here with the enclosed patch.
> > >
> > > great, big step forward making /media/Makefile smaller ... thanks a lot!!!!
> > >
> > > > It crashed:
> > > > Exception occurred:
> > > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > > sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want to report the issue to the developers.
> > > > Please also report this if it was a user error, so that a better error message can be provided next time.
> > > > A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> > > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > > make[1]: *** [htmldocs] Error 1
> > > > Makefile:1450: recipe for target 'htmldocs' failed
> > > > make: *** [htmldocs] Error 2
> > > >
> > > > Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg file.
> > >
> > > I guess that the dot command writes something to stderr. This is captured
> > > by the extension and printed to stderr ...
> > >
> > > +def dot2format(dot_fname, out_fname):
> > > ...
> > > + exit_code = 42
> > > + with open(out_fname, "w") as out:
> > > + p = subprocess.Popen(
> > > + cmd, stdout = out, stderr = subprocess.PIPE )
> > > + nil, err = p.communicate()
> > > +
> > > + sys.stderr.write(err)
> > > +
> > > + exit_code = p.returncode
> > > + out.flush()
> > > + return bool(exit_code == 0)
> > >
> > > > File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in dot2format
> > > > sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > >
> > > Do we need this stderr output? For a first test, uncomment the
> > > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > > stderr, try:
> > >
> > > - sys.stderr.write(err)
> > > + sys.stderr.write(str(err))
> > >
> > > I this fixes, there is another "sys.stderr.write(err)" in
> > > func svg2pdf(..) which should also fixed ….
> > >
> > > +def svg2pdf(svg_fname, pdf_fname):
> > > ...
> > > + cmd = [convert_cmd, svg_fname, pdf_fname]
> > > + p = subprocess.Popen(
> > > + cmd, stdout = out, stderr = subprocess.PIPE )
> > > + nil, err = p.communicate()
> > > +
> > > - sys.stderr.write(err)
> > > + sys.stderr.write(str(err))
> > > +
> > > + exit_code = p.returncode
> > > + return bool(exit_code == 0)
> >
> > Yes, I very much want stderr to be forward.
>
> Yes, error report is required.
>
> > Without that you don't see
> > error output from dot or convert, and that makes it impossible to debug
> > anything. If I want a direct forwarding of the bytes, how should I do this
> > in python? Capturing stderr and then re-dumping it is kinda silly ...
>
> Markus or some other Python programmer may help us with that.
>
> > Note that I copied this pattern from the kernel-doc extension, seems to
> > have worked there.
>
> Maybe it is broken there too then, or this is another python API
> that changed over time. Here, I'm testing with:
> python3-3.5.2-4.fc25.x86_64
>
> From here:
> https://docs.python.org/2/library/subprocess.html
>
> communicate returns a tuple.
>
> I used repr(p.communicate()[0], on the code snippet I sent,
> as I copied from an example that I found at:
> https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr
>
> Thanks,
> Mauro
I suspect that the actual fixup would be something like:
commit ddf93ae81af10bb43caa651b9acd355f1d74cebe
Author: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Date: Thu Mar 2 17:11:47 2017 -0300
kfigure.py fixups
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..19389cb34d6f 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
with open(out_fname, "w") as out:
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
out.flush()
return bool(exit_code == 0)
@@ -239,11 +241,13 @@ def svg2pdf(svg_fname, pdf_fname):
cmd = [convert_cmd, svg_fname, pdf_fname]
p = subprocess.Popen(
cmd, stdout = out, stderr = subprocess.PIPE )
- nil, err = p.communicate()
-
- sys.stderr.write(err)
+ err = p.communicate()
exit_code = p.returncode
+
+ if exit_code == 0:
+ sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
return bool(exit_code == 0)
Thanks,
Mauro
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
2017-03-02 18:06 ` Mauro Carvalho Chehab
@ 2017-03-02 21:29 ` Mauro Carvalho Chehab
2017-03-02 21:36 ` Mauro Carvalho Chehab
1 sibling, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:29 UTC (permalink / raw)
To: Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 16:40:02 +0100
Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> v5: Use an explicit version check (suggested by Laurent).
Found another issue on the patch. The HTML output is pointing to the
wrong place: instead of using a relative patch, it is keeping
an absolute one.
This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
<div class="figure align-center" id="id2">
<img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
<div class="legend">
High quality and high speed pipeline configuration</div>
</div>
There, the "src=" is pointing to the full patch, with doesn't work, as
my html server uses a different patch to find the file. It should,
instead, use a patch relative to the place where the html file is
stored, e. g. in this case, either:
./pipeline.svg
or just:
pipeline.svg
Regards,
Mauro
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:29 ` Mauro Carvalho Chehab
@ 2017-03-02 21:36 ` Mauro Carvalho Chehab
2017-03-02 21:47 ` Mauro Carvalho Chehab
2017-03-03 8:54 ` Daniel Vetter
0 siblings, 2 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:36 UTC (permalink / raw)
To: Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 18:29:39 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Em Thu, 2 Mar 2017 16:40:02 +0100
> Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
>
> > From: Markus Heiser <markus.heiser@darmarit.de>
> >
> > This patch brings scalable figure, image handling and a concept to
> > embed *render* markups:
> >
> > * DOT (http://www.graphviz.org)
> > * SVG
> >
> > For image handling use the 'image' replacement::
> >
> > .. kernel-image:: svg_image.svg
> > :alt: simple SVG image
> >
> > For figure handling use the 'figure' replacement::
> >
> > .. kernel-figure:: svg_image.svg
> > :alt: simple SVG image
> >
> > SVG image example
> >
> > Embed *render* markups (or languages) like Graphviz's **DOT** is
> > provided by the *render* directive.::
> >
> > .. kernel-render:: DOT
> > :alt: foobar digraph
> > :caption: Embedded **DOT** (Graphviz) code.
> >
> > digraph foo {
> > "bar" -> "baz";
> > }
> >
> > The *render* directive is a concept to integrate *render* markups and
> > languages, yet supported markups:
> >
> > * DOT: render embedded Graphviz's **DOT**
> > * SVG: render embedded Scalable Vector Graphics (**SVG**)
> >
> > v2: s/DOC/DOT/ in a few places (by Daniel).
> >
> > v3: Simplify stuff a bit (by Daniel):
> >
> > - Remove path detection and setup/check code for that. In
> > Documentation/media/Makefile we already simply use these tools,
> > better to have one consolidated check if we want/need one. Also
> > remove the convertsvg support, we require ImageMagick's convert
> > already in the doc build, no need for a 2nd fallback.
> >
> > - Use sphinx for depency tracking, remove hand-rolled version.
> >
> > - Forward stderr from dot and convert, otherwise debugging issues with
> > the diagrams is impossible.
> >
> > v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> > Implement Markus suggestion for backwards compatability with earlier
> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
> > untested.
> >
> > v5: Use an explicit version check (suggested by Laurent).
>
> Found another issue on the patch. The HTML output is pointing to the
> wrong place: instead of using a relative patch, it is keeping
> an absolute one.
>
> This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
>
> <div class="figure align-center" id="id2">
> <img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
> <div class="legend">
> High quality and high speed pipeline configuration</div>
> </div>
>
> There, the "src=" is pointing to the full patch, with doesn't work, as
> my html server uses a different patch to find the file. It should,
> instead, use a patch relative to the place where the html file is
> stored, e. g. in this case, either:
> ./pipeline.svg
> or just:
> pipeline.svg
Btw, PDF conversion is also not working:
File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
cmd = [convert_cmd, svg_fname, pdf_fname]
NameError: name 'convert_cmd' is not defined
And including SVG files for HTML output also seems to be problematic.
I'll post the RFCv2 patch that I'm using to test it.
Regards,
Mauro
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:36 ` Mauro Carvalho Chehab
@ 2017-03-02 21:47 ` Mauro Carvalho Chehab
2017-03-02 21:54 ` Markus Heiser
2017-03-03 8:54 ` Daniel Vetter
1 sibling, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 21:47 UTC (permalink / raw)
To: Daniel Vetter, Daniel Vetter
Cc: linux-doc, DRI Development, Markus Heiser, Jonathan Corbet,
Jani Nikula, Laurent Pinchart
Em Thu, 2 Mar 2017 18:36:31 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> > Found another issue on the patch. The HTML output is pointing to the
> > wrong place: instead of using a relative patch, it is keeping
> > an absolute one.
> >
> > This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
> >
> > <div class="figure align-center" id="id2">
> > <img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
> > <div class="legend">
> > High quality and high speed pipeline configuration</div>
> > </div>
> >
> > There, the "src=" is pointing to the full patch, with doesn't work, as
> > my html server uses a different patch to find the file. It should,
> > instead, use a patch relative to the place where the html file is
> > stored, e. g. in this case, either:
> > ./pipeline.svg
> > or just:
> > pipeline.svg
>
> Btw, PDF conversion is also not working:
>
>
> File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
> cmd = [convert_cmd, svg_fname, pdf_fname]
>
> NameError: name 'convert_cmd' is not defined
>
> And including SVG files for HTML output also seems to be problematic.
Forgot to mention, but I'm using here Sphinx 1.4.9, installed via
pip3 (So, python3).
>
> I'll post the RFCv2 patch that I'm using to test it.
Patch posted. Hopefully, it will help you to see the problems I'm
facing on my tests.
Thanks,
Mauro
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:47 ` Mauro Carvalho Chehab
@ 2017-03-02 21:54 ` Markus Heiser
0 siblings, 0 replies; 28+ messages in thread
From: Markus Heiser @ 2017-03-02 21:54 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Daniel Vetter
Cc: Jonathan Corbet, DRI Development, Linux Doc Mailing List,
Laurent Pinchart, Daniel Vetter
>> Btw, PDF conversion is also not working:
>>
>>
>> File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
>> cmd = [convert_cmd, svg_fname, pdf_fname]
>>
>> NameError: name 'convert_cmd' is not defined
>>
>> And including SVG files for HTML output also seems to be problematic.
>
> Forgot to mention, but I'm using here Sphinx 1.4.9, installed via
> pip3 (So, python3).
It seems, that Daniel drops some lines from my first RFC, I miss this lines:
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+# Graphviz's dot(1) support
+dot_cmd = which('dot')
+
+# ImageMagick' convert(1) support
+convert_cmd = which('convert')
@Daniel: can you take a look at this / Thanks
-- Markus --
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 21:36 ` Mauro Carvalho Chehab
2017-03-02 21:47 ` Mauro Carvalho Chehab
@ 2017-03-03 8:54 ` Daniel Vetter
1 sibling, 0 replies; 28+ messages in thread
From: Daniel Vetter @ 2017-03-03 8:54 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Markus Heiser, Jonathan Corbet, DRI Development,
Linux Doc Mailing List, Laurent Pinchart, Daniel Vetter
On Thu, Mar 2, 2017 at 10:36 PM, Mauro Carvalho Chehab
<mchehab@s-opensource.com> wrote:
> Em Thu, 2 Mar 2017 18:29:39 -0300
> Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
>
>> Em Thu, 2 Mar 2017 16:40:02 +0100
>> Daniel Vetter <daniel.vetter@ffwll.ch> escreveu:
>>
>> > From: Markus Heiser <markus.heiser@darmarit.de>
>> >
>> > This patch brings scalable figure, image handling and a concept to
>> > embed *render* markups:
>> >
>> > * DOT (http://www.graphviz.org)
>> > * SVG
>> >
>> > For image handling use the 'image' replacement::
>> >
>> > .. kernel-image:: svg_image.svg
>> > :alt: simple SVG image
>> >
>> > For figure handling use the 'figure' replacement::
>> >
>> > .. kernel-figure:: svg_image.svg
>> > :alt: simple SVG image
>> >
>> > SVG image example
>> >
>> > Embed *render* markups (or languages) like Graphviz's **DOT** is
>> > provided by the *render* directive.::
>> >
>> > .. kernel-render:: DOT
>> > :alt: foobar digraph
>> > :caption: Embedded **DOT** (Graphviz) code.
>> >
>> > digraph foo {
>> > "bar" -> "baz";
>> > }
>> >
>> > The *render* directive is a concept to integrate *render* markups and
>> > languages, yet supported markups:
>> >
>> > * DOT: render embedded Graphviz's **DOT**
>> > * SVG: render embedded Scalable Vector Graphics (**SVG**)
>> >
>> > v2: s/DOC/DOT/ in a few places (by Daniel).
>> >
>> > v3: Simplify stuff a bit (by Daniel):
>> >
>> > - Remove path detection and setup/check code for that. In
>> > Documentation/media/Makefile we already simply use these tools,
>> > better to have one consolidated check if we want/need one. Also
>> > remove the convertsvg support, we require ImageMagick's convert
>> > already in the doc build, no need for a 2nd fallback.
>> >
>> > - Use sphinx for depency tracking, remove hand-rolled version.
>> >
>> > - Forward stderr from dot and convert, otherwise debugging issues with
>> > the diagrams is impossible.
>> >
>> > v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
>> > Implement Markus suggestion for backwards compatability with earlier
>> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
>> > untested.
>> >
>> > v5: Use an explicit version check (suggested by Laurent).
>>
>> Found another issue on the patch. The HTML output is pointing to the
>> wrong place: instead of using a relative patch, it is keeping
>> an absolute one.
>>
>> This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
>>
>> <div class="figure align-center" id="id2">
>> <img alt="pipeline.dot" src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /><p class="caption"><span class="caption-text">Image Format Negotiation on Pipelines</span></p>
>> <div class="legend">
>> High quality and high speed pipeline configuration</div>
>> </div>
>>
>> There, the "src=" is pointing to the full patch, with doesn't work, as
>> my html server uses a different patch to find the file. It should,
>> instead, use a patch relative to the place where the html file is
>> stored, e. g. in this case, either:
>> ./pipeline.svg
>> or just:
>> pipeline.svg
>
> Btw, PDF conversion is also not working:
>
>
> File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
> cmd = [convert_cmd, svg_fname, pdf_fname]
>
> NameError: name 'convert_cmd' is not defined
Oops, should be fixed in the next version. But my tex here is broken,
so I can't test :(
> And including SVG files for HTML output also seems to be problematic.
Yeah, it's all using absolute paths. I'm fixing it.
> I'll post the RFCv2 patch that I'm using to test it.
I'll also fix the stderr string conversion thing and then resend the
new version.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 15:16 ` [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images Daniel Vetter
2017-03-02 15:20 ` Laurent Pinchart
2017-03-02 15:40 ` [PATCH] " Daniel Vetter
@ 2017-03-02 22:15 ` Markus Heiser
2017-03-02 23:19 ` Mauro Carvalho Chehab
2 siblings, 1 reply; 28+ messages in thread
From: Markus Heiser @ 2017-03-02 22:15 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, DRI Development, linux-doc,
Mauro Carvalho Chehab, Laurent Pinchart, Daniel Vetter
Hi Daniel, hi Mauro,
I have to say, that I lost control over our thread ;)
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
I have to come back to a more systematic work, this means;
I have to test this v3 and consider what Mauro and you
already posted.
For this, give me some time. I will send a v4 patch this weekend.
Thanks for all for comments and hints.
-- Markus --
> Am 02.03.2017 um 16:16 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>:
>
> From: Markus Heiser <markus.heiser@darmarit.de>
>
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
>
> * DOT (http://www.graphviz.org)
> * SVG
>
> For image handling use the 'image' replacement::
>
> .. kernel-image:: svg_image.svg
> :alt: simple SVG image
>
> For figure handling use the 'figure' replacement::
>
> .. kernel-figure:: svg_image.svg
> :alt: simple SVG image
>
> SVG image example
>
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
>
> .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
>
> digraph foo {
> "bar" -> "baz";
> }
>
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
>
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
>
> v2: s/DOC/DOT/ in a few places (by Daniel).
>
> v3: Simplify stuff a bit (by Daniel):
>
> - Remove path detection and setup/check code for that. In
> Documentation/media/Makefile we already simply use these tools,
> better to have one consolidated check if we want/need one. Also
> remove the convertsvg support, we require ImageMagick's convert
> already in the doc build, no need for a 2nd fallback.
>
> - Use sphinx for depency tracking, remove hand-rolled version.
>
> - Forward stderr from dot and convert, otherwise debugging issues with
> the diagrams is impossible.
>
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> Cc: Markus Heiser <markus.heiser@darmarit.de>
> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1)
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> Documentation/conf.py | 2 +-
> Documentation/doc-guide/hello.dot | 3 +
> Documentation/doc-guide/sphinx.rst | 90 ++++++-
> Documentation/doc-guide/svg_image.svg | 10 +
> Documentation/process/changes.rst | 7 +-
> Documentation/sphinx/kfigure.py | 444 ++++++++++++++++++++++++++++++++++
> 6 files changed, 550 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index f6823cf01275..e3f537ce2935 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -34,7 +34,7 @@ from load_config import loadConfig
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
> +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
>
> # The name of the math extension changed on Sphinx 1.4
> if major == 1 and minor > 3:
> diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
> new file mode 100644
> index 000000000000..504621dfc595
> --- /dev/null
> +++ b/Documentation/doc-guide/hello.dot
> @@ -0,0 +1,3 @@
> +graph G {
> + Hello -- World
> +}
> diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
> index 532d65b70500..b902744ce7dd 100644
> --- a/Documentation/doc-guide/sphinx.rst
> +++ b/Documentation/doc-guide/sphinx.rst
> @@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
>
> To generate documentation, Sphinx (``sphinx-build``) must obviously be
> installed. For prettier HTML output, the Read the Docs Sphinx theme
> -(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
> -needed. All of these are widely available and packaged in distributions.
> +(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
> +``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
> +from ImageMagick (https://www.imagemagick.org). All of these are widely
> +available and packaged in distributions.
>
> To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
> variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
> @@ -232,3 +234,87 @@ Rendered as:
> * .. _`last row`:
>
> - column 3
> +
> +
> +Figures & Images
> +================
> +
> +If you want to add an image, you should use the ``kernel-figure`` and
> +``kernel-image`` directives. E.g. to insert a figure with a scalable
> +image format use SVG::
> +
> + .. kernel-figure:: svg_image.svg
> + :alt: simple SVG image
> +
> + SVG image example
> +
> +.. kernel-figure:: svg_image.svg
> + :alt: simple SVG image
> +
> + SVG image example
> +
> +The kernel figure (and image) directive support **DOT** formated files, see
> +
> +* DOT: http://graphviz.org/pdf/dotguide.pdf
> +* Graphviz: http://www.graphviz.org/content/dot-language
> +
> +A simple example::
> +
> + .. kernel-figure:: hello.dot
> + :alt: hello world
> +
> + DOT's hello world example
> +
> +.. kernel-figure:: hello.dot
> + :alt: hello world
> +
> + DOT's hello world example
> +
> +Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
> +``kernel-render`` directives.::
> +
> + .. kernel-render:: DOT
> + :alt: foobar digraph
> + :caption: Embedded **DOT** (Graphviz) code.
> +
> + digraph foo {
> + "bar" -> "baz";
> + }
> +
> +How this will be rendered depends on the installed tools. If Graphviz is
> +installed, you will see an vector image. If not the raw markup is inserted as
> +*literal-block*.
> +
> +.. kernel-render:: DOT
> + :alt: foobar digraph
> + :caption: Embedded **DOT** (Graphviz) code.
> +
> + digraph foo {
> + "bar" -> "baz";
> + }
> +
> +The *render* directive has all the options known from the *figure* directive,
> +plus option ``caption``. If ``caption`` has a value, a *figure* node is
> +inserted. If not, a *image* node is inserted.
> +
> +Embedded **SVG**::
> +
> + .. kernel-render:: SVG
> + :caption: Embedded **SVG** markup.
> + :alt: so-nw-arrow
> +
> + <?xml version="1.0" encoding="UTF-8"?>
> + <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
> + ...
> + </svg>
> +
> +.. kernel-render:: SVG
> + :caption: Embedded **SVG** markup.
> + :alt: so-nw-arrow
> +
> + <?xml version="1.0" encoding="UTF-8"?>
> + <svg xmlns="http://www.w3.org/2000/svg"
> + version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
> + </svg>
> diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
> new file mode 100644
> index 000000000000..5405f85b8137
> --- /dev/null
> +++ b/Documentation/doc-guide/svg_image.svg
> @@ -0,0 +1,10 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
> +<svg xmlns="http://www.w3.org/2000/svg"
> + version="1.1" baseProfile="full"
> + width="70px" height="40px" viewBox="0 0 700 400">
> + <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
> + <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
> + <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
> + <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
> +</svg>
> diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
> index 56ce66114665..e4f25038ef65 100644
> --- a/Documentation/process/changes.rst
> +++ b/Documentation/process/changes.rst
> @@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
> .. note::
>
> Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
> - version 3.14159265. Depending on the distribution, you may also need
> - to install a series of ``texlive`` packages that provide the minimal
> - set of functionalities required for ``XeLaTex`` to work.
> + version 3.14159265. Depending on the distribution, you may also need to
> + install a series of ``texlive`` packages that provide the minimal set of
> + functionalities required for ``XeLaTex`` to work. For PDF output you'll also
> + need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
>
> Other tools
> -----------
> diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
> new file mode 100644
> index 000000000000..cbb23e089186
> --- /dev/null
> +++ b/Documentation/sphinx/kfigure.py
> @@ -0,0 +1,444 @@
> +# -*- coding: utf-8; mode: python -*-
> +# pylint: disable=C0103
> +u"""
> + scalable figure and image handling
> + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> + Sphinx extension which implements scalable image handling.
> +
> + :copyright: Copyright (C) 2016 Markus Heiser
> + :license: GPL Version 2, June 1991 see Linux/COPYING for details.
> +
> + The build for image formats depence on image's source format and output's
> + destination format. This extension implement methods to simplify image
> + handling from the author's POV. Directives like ``kernel-figure`` implement
> + methods *to* always get the best output-format even if some tools are not
> + installed.For more details take a look at ``convert_image(...)`` which is
> + the core of all conversions.
> +
> + * ``.. kernel-image``: for image handling / ``.. image::`` replacement
> +
> + * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
> +
> + * ``.. kernel-render``: for render markup / a concept to embed *render*
> + markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
> +
> + + ``DOT``: render embedded Graphviz's **DOC**
> + + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
> + + ... *developable*
> +
> + Used tools:
> +
> + * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
> + available, the DOT language is inserted as literal-block.
> +
> + * SVG to PDF: To generate PDF, you need at least one of this tools:
> +
> + - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
> +
> + List of customizations:
> +
> + * generate PDF from SVG / used by PDF (LaTeX) builder
> +
> + * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
> + DOT: see http://www.graphviz.org/content/dot-language
> +
> + """
> +
> +import os
> +from os import path
> +import subprocess
> +from hashlib import sha1
> +import sys
> +
> +from docutils import nodes
> +from docutils.statemachine import ViewList
> +from docutils.parsers.rst import directives
> +from docutils.parsers.rst.directives import images
> +try:
> + from sphinx.directives.patches import Figure
> +except ImportError:
> + Figure = images.Figure
> +
> +__version__ = '1.0'
> +
> +# simple helper
> +# -------------
> +
> +def mkdir(folder, mode=0o775):
> + if not path.isdir(folder):
> + os.makedirs(folder, mode)
> +
> +# def debug_handle(self, node): # pylint: disable=W0613
> +# from linuxdoc.kernel_doc import CONSOLE
> +# CONSOLE()
> +
> +def pass_handle(self, node): # pylint: disable=W0613
> + pass
> +
> +# setup conversion tools and sphinx extension
> +# -------------------------------------------
> +
> +def setup(app):
> + # image handling
> + app.add_directive("kernel-image", KernelImage)
> + app.add_node(kernel_image,
> + html = (visit_kernel_image, pass_handle),
> + latex = (visit_kernel_image, pass_handle),
> + texinfo = (visit_kernel_image, pass_handle),
> + text = (visit_kernel_image, pass_handle),
> + man = (visit_kernel_image, pass_handle), )
> +
> + # figure handling
> + app.add_directive("kernel-figure", KernelFigure)
> + app.add_node(kernel_figure,
> + html = (visit_kernel_figure, pass_handle),
> + latex = (visit_kernel_figure, pass_handle),
> + texinfo = (visit_kernel_figure, pass_handle),
> + text = (visit_kernel_figure, pass_handle),
> + man = (visit_kernel_figure, pass_handle), )
> +
> + # render handling
> + app.add_directive('kernel-render', KernelRender)
> + app.add_node(kernel_render,
> + html = (visit_kernel_render, pass_handle),
> + latex = (visit_kernel_render, pass_handle),
> + texinfo = (visit_kernel_render, pass_handle),
> + text = (visit_kernel_render, pass_handle),
> + man = (visit_kernel_render, pass_handle), )
> +
> + return dict(
> + version = __version__,
> + parallel_read_safe = True,
> + parallel_write_safe = True
> + )
> +
> +# integrate conversion tools
> +# --------------------------
> +
> +RENDER_MARKUP_EXT = {
> + # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
> + # <name> : <.ext>
> + 'DOT' : '.dot'
> + , 'SVG' : '.svg'
> +}
> +
> +def convert_image(img_node, translator): # pylint: disable=R0912
> + """Convert an image node for the builder.
> +
> + Different builder prefer different image formats, e.g. *latex* builder
> + prefer PDF while *html* builder prefer SVG format for images.
> +
> + This function handles outputs image formats in depence of source the format
> + of the image and the translator's output format. This also means to
> + manipulate/update the *image* dictionary of the builder (``builder.images``)
> +
> + """
> + fname, in_ext = path.splitext(path.basename(img_node['uri']))
> + src_fname = path.join(translator.builder.srcdir, img_node['uri'])
> + src_folder = path.dirname(img_node['uri'])
> + out_dir = translator.builder.outdir
> + dst_fname = None
> +
> + # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
> + verbose = translator.builder.app.verbose
> + warn = translator.builder.warn
> +
> + verbose('assert best format for: ' + img_node['uri'])
> +
> + if in_ext == '.dot':
> +
> + # ----------
> + # handle DOT
> + # ----------
> +
> + dst_fname = path.join(out_dir, fname + '.pdf')
> +
> + if translator.builder.format == 'html':
> + dst_fname = path.join(out_dir, src_folder, fname + '.svg')
> + else:
> + # all other builder formats will include DOT as raw
> + with open(src_fname, "r") as dot:
> + data = dot.read()
> + node = nodes.literal_block(data, data)
> + img_node.replace_self(node)
> +
> +
> + elif in_ext == '.svg':
> +
> + # ----------
> + # handle SVG
> + # ----------
> +
> + if translator.builder.format == 'latex':
> + dst_fname = path.join(out_dir, fname + '.pdf')
> +
> + if dst_fname:
> + name = dst_fname[len(out_dir) + 1:]
> + # the builder needs not to copy one more time, so pop it if exists.
> + translator.builder.images.pop(img_node['uri'], None)
> + img_node['uri'] = dst_fname
> + img_node['candidates'] = {'*': dst_fname}
> +
> + mkdir(path.dirname(dst_fname))
> +
> + if in_ext == '.dot':
> + verbose('convert DOT to: {out}/' + name)
> + dot2format(src_fname, dst_fname)
> +
> + elif in_ext == '.svg':
> + verbose('convert SVG to: {out}/' + name)
> + svg2pdf(src_fname, dst_fname)
> +
> +def dot2format(dot_fname, out_fname):
> + """Converts DOT file to ``out_fname`` using ``dot(1)``.
> +
> + * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
> + * ``out_fname`` pathname of the output file, including format extension
> +
> + The *format extension* depends on the ``dot`` command (see ``man dot``
> + option ``-Txxx``). Normally you will use one of the following extensions:
> +
> + - ``.ps`` for PostScript,
> + - ``.svg`` or ``svgz`` for Structured Vector Graphics,
> + - ``.fig`` for XFIG graphics and
> + - ``.png`` or ``gif`` for common bitmap graphics.
> +
> + """
> + out_format = path.splitext(out_fname)[1][1:]
> + cmd = ['dot', '-T%s' % out_format, dot_fname]
> + exit_code = 42
> + with open(out_fname, "w") as out:
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + out.flush()
> + return bool(exit_code == 0)
> +
> +def svg2pdf(svg_fname, pdf_fname):
> + """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
> +
> + Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
> + conversion. Returns ``True`` on success and ``False`` if an error occurred
> + (e.g. none of the conversion tool is available).
> +
> + * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
> + * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
> +
> + """
> + cmd = [convert_cmd, svg_fname, pdf_fname]
> + p = subprocess.Popen(
> + cmd, stdout = out, stderr = subprocess.PIPE )
> + nil, err = p.communicate()
> +
> + sys.stderr.write(err)
> +
> + exit_code = p.returncode
> + return bool(exit_code == 0)
> +
> +
> +# image handling
> +# ---------------------
> +
> +def visit_kernel_image(self, node): # pylint: disable=W0613
> + """Visitor of the ``kernel_image`` Node.
> +
> + Handles the ``image`` child-node with the ``convert_image(...)``.
> + """
> + img_node = node[0]
> + convert_image(img_node, self)
> +
> +class kernel_image(nodes.General, nodes.Element):
> + """Node for ``kernel-image`` directive."""
> + pass
> +
> +class KernelImage(images.Image):
> + u"""KernelImage directive
> +
> + Earns everything from ``.. image::`` directive, except *remote URI* and
> + *glob* pattern. The KernelImage wraps a image node into a
> + kernel_image node. See ``visit_kernel_image``.
> + """
> +
> + def run(self):
> + env = self.state.document.settings.env
> +
> + uri = self.arguments[0]
> + if uri.endswith('.*') or uri.find('://') != -1:
> + raise self.severe(
> + 'Error in "%s: %s": glob pattern and remote images are not allowed'
> + % (self.name, uri))
> +
> + # Tell sphinx of the dependency
> + env.note_dependency(os.path.abspath(uri))
> +
> + result = images.Image.run(self)
> + if len(result) == 2 or isinstance(result[0], nodes.system_message):
> + return result
> + (image_node,) = result
> + # wrap image node into a kernel_image node / see visitors
> + node = kernel_image('', image_node)
> + return [node]
> +
> +# figure handling
> +# ---------------------
> +
> +def visit_kernel_figure(self, node): # pylint: disable=W0613
> + """Visitor of the ``kernel_figure`` Node.
> +
> + Handles the ``image`` child-node with the ``convert_image(...)``.
> + """
> + img_node = node[0][0]
> + convert_image(img_node, self)
> +
> +class kernel_figure(nodes.General, nodes.Element):
> + """Node for ``kernel-figure`` directive."""
> +
> +class KernelFigure(Figure):
> + u"""KernelImage directive
> +
> + Earns everything from ``.. figure::`` directive, except *remote URI* and
> + *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
> + node. See ``visit_kernel_figure``.
> + """
> +
> + def run(self):
> + env = self.state.document.settings.env
> +
> + uri = self.arguments[0]
> + if uri.endswith('.*') or uri.find('://') != -1:
> + raise self.severe(
> + 'Error in "%s: %s":'
> + ' glob pattern and remote images are not allowed'
> + % (self.name, uri))
> +
> + # Tell sphinx of the dependency
> + env.note_dependency(os.path.abspath(uri))
> +
> + result = Figure.run(self)
> + if len(result) == 2 or isinstance(result[0], nodes.system_message):
> + return result
> + (figure_node,) = result
> + # wrap figure node into a kernel_figure node / see visitors
> + node = kernel_figure('', figure_node)
> + return [node]
> +
> +
> +# render handling
> +# ---------------------
> +
> +def visit_kernel_render(self, node):
> + """Visitor of the ``kernel_render`` Node.
> +
> + If rendering tools available, save the markup of the ``literal_block`` child
> + node into a file and replace the ``literal_block`` node with a new created
> + ``image`` node, pointing to the saved markup file. Afterwards, handle the
> + image child-node with the ``convert_image(...)``.
> + """
> +
> + verbose = self.builder.app.verbose
> + warn = self.builder.warn
> + srclang = node.get('srclang')
> +
> + verbose('visit kernel-render node lang: "%s"' % (srclang))
> +
> + tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
> + if tmp_ext is None:
> + warn('kernel-render: "%s" unknow / include raw.' % (srclang))
> + return
> +
> + literal_block = node[0]
> + code = literal_block.astext()
> +
> + if tmp_ext:
> + hashobj = code.encode('utf-8') # str(node.attributes)
> + fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
> + tmp_fname = path.join(
> + self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
> +
> + if not path.isfile(tmp_fname):
> + mkdir(path.dirname(tmp_fname))
> + with open(tmp_fname, "w") as out:
> + out.write(code)
> +
> + image_node = nodes.image(node.rawsource, **node.attributes)
> + image_node['uri'] = tmp_fname
> +
> + literal_block.replace_self(image_node)
> + convert_image(image_node, self)
> +
> +
> +class kernel_render(nodes.General, nodes.Inline, nodes.Element):
> + """Node for ``kernel-render`` directive."""
> + pass
> +
> +class KernelRender(Figure):
> + u"""KernelRender directive
> +
> + Render content by external tool. Has all the options known from the
> + *figure* directive, plus option ``caption``. If ``caption`` has a
> + value, a figure node with the *caption* is inserted. If not, a image node is
> + inserted.
> +
> + The KernelRender directive wraps the text of the directive into a
> + literal_block node and wraps it into a kernel_render node. See
> + ``visit_kernel_render``.
> + """
> + has_content = True
> + required_arguments = 1
> + optional_arguments = 0
> + final_argument_whitespace = False
> +
> + # earn options from 'figure'
> + option_spec = Figure.option_spec.copy()
> + option_spec['caption'] = directives.unchanged
> +
> + def run(self):
> + return [self.build_node()]
> +
> + def build_node(self):
> +
> + srclang = self.arguments[0].strip()
> + if srclang not in RENDER_MARKUP_EXT.keys():
> + return [self.state_machine.reporter.warning(
> + 'Unknow source language "%s", use one of: %s.' % (
> + srclang, ",".join(RENDER_MARKUP_EXT.keys())),
> + line=self.lineno)]
> +
> + code = '\n'.join(self.content)
> + if not code.strip():
> + return [self.state_machine.reporter.warning(
> + 'Ignoring "%s" directive without content.' % (
> + self.name),
> + line=self.lineno)]
> +
> + node = kernel_render()
> + node['alt'] = self.options.get('alt','')
> + node['srclang'] = srclang
> + literal_node = nodes.literal_block(code, code)
> + node += literal_node
> +
> + caption = self.options.get('caption')
> + if caption:
> + # parse cation's content
> + parsed = nodes.Element()
> + self.state.nested_parse(
> + ViewList([caption], source=''), self.content_offset, parsed)
> + caption_node = nodes.caption(
> + parsed[0].rawsource, '', *parsed[0].children)
> + caption_node.source = parsed[0].source
> + caption_node.line = parsed[0].line
> +
> + figure_node = nodes.figure('', node)
> + for k,v in self.options.items():
> + figure_node[k] = v
> + figure_node += caption_node
> +
> + node = figure_node
> +
> + return node
> +
> --
> 2.11.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
2017-03-02 22:15 ` [PATCH 2/6] " Markus Heiser
@ 2017-03-02 23:19 ` Mauro Carvalho Chehab
0 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-03-02 23:19 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, linux-doc, DRI Development, Jonathan Corbet,
Jani Nikula, Laurent Pinchart, Daniel Vetter
Em Thu, 2 Mar 2017 23:15:49 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:
> Hi Daniel, hi Mauro,
>
> I have to say, that I lost control over our thread ;)
>
> > v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> > Implement Markus suggestion for backwards compatability with earlier
> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
> > untested.
>
> I have to come back to a more systematic work, this means;
> I have to test this v3 and consider what Mauro and you
> already posted.
Yes, you're lost :-) The last patch Daniel Sent was v5. From his last patch:
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
>
> v5: Use an explicit version check (suggested by Laurent).
https://marc.info/?l=linux-doc&m=148847115500505&w=2
Thanks,
Mauro
^ permalink raw reply [flat|nested] 28+ messages in thread
