[PATCH v2 0/5] make htmldocs in CI

[PATCH v2 0/5] make htmldocs in CI

[Heinrich Schuchardt]

This patch series provides a htmldocs target for Travis, Gitlab and Azure
CI.

doc/sphinx/kerneldoc.py is updated to avoid a build warning with current
Sphinx.

The last patch turns build warnings of 'make htmldocs' into errors.

While first four patches can be immediately merged the last patch requires
an updated Docker image, cf.

[PATCH 1/1] Dockerfile: add imagemagick
https://lists.denx.de/pipermail/u-boot/2020-February/400983.html

v2:
	update update doc/sphinx/kerneldoc.py
	change sequence of targets in Travis Ci
	change a commit message

Heinrich Schuchardt (5):
  doc: update doc/sphinx/kerneldoc.py
  travis: build HTML docs
  gitlab: build HTML documentation
  azure: build HTML documentation
  doc/Makefile: turn warnings into errors

 .azure-pipelines.yml    | 10 ++++++++
 .gitlab-ci.yml          |  7 ++++++
 .travis.yml             |  5 ++++
 doc/Makefile            |  1 +
 doc/sphinx/kerneldoc.py | 51 +++++++++++++++++++++++++++++++----------
 5 files changed, 62 insertions(+), 12 deletions(-)

--
2.25.0

[PATCH v2 1/5] doc: update doc/sphinx/kerneldoc.py

[Heinrich Schuchardt]

Update doc/sphinx/kerneldoc.py from Linux next-20200219 to avoid warnings
like:

doc/sphinx/kerneldoc.py:125: RemovedInSphinx20Warning:
AutodocReporter is now deprecated. Use
sphinx.util.docutils.switch_source_input() instead.
  self.state.memo.reporter =
  	AutodocReporter(result, self.state.memo.reporter)

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
---
v2:
	new patch
---
 doc/sphinx/kerneldoc.py | 51 +++++++++++++++++++++++++++++++----------
 1 file changed, 39 insertions(+), 12 deletions(-)

diff --git a/doc/sphinx/kerneldoc.py b/doc/sphinx/kerneldoc.py
index e536360de1..4bcbd6ae01 100644
--- a/doc/sphinx/kerneldoc.py
+++ b/doc/sphinx/kerneldoc.py
@@ -37,7 +37,17 @@ import glob
 from docutils import nodes, statemachine
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
-from sphinx.ext.autodoc import AutodocReporter
+
+#
+# AutodocReporter is only good up to Sphinx 1.7
+#
+import sphinx
+
+Use_SSI = sphinx.__version__[:3] >= '1.7'
+if Use_SSI:
+    from sphinx.util.docutils import switch_source_input
+else:
+    from sphinx.ext.autodoc import AutodocReporter

 import kernellog

@@ -49,9 +59,10 @@ class KernelDocDirective(Directive):
     optional_arguments = 4
     option_spec = {
         'doc': directives.unchanged_required,
-        'functions': directives.unchanged_required,
         'export': directives.unchanged,
         'internal': directives.unchanged,
+        'identifiers': directives.unchanged,
+        'functions': directives.unchanged,
     }
     has_content = False

@@ -67,6 +78,10 @@ class KernelDocDirective(Directive):

         tab_width = self.options.get('tab-width', self.state.document.settings.tab_width)

+        # 'function' is an alias of 'identifiers'
+        if 'functions' in self.options:
+            self.options['identifiers'] = self.options.get('functions')
+
         # FIXME: make this nicer and more robust against errors
         if 'export' in self.options:
             cmd += ['-export']
@@ -76,9 +91,13 @@ class KernelDocDirective(Directive):
             export_file_patterns = str(self.options.get('internal')).split()
         elif 'doc' in self.options:
             cmd += ['-function', str(self.options.get('doc'))]
-        elif 'functions' in self.options:
-            for f in str(self.options.get('functions')).split():
-                cmd += ['-function', f]
+        elif 'identifiers' in self.options:
+            identifiers = self.options.get('identifiers').split()
+            if identifiers:
+                for i in identifiers:
+                    cmd += ['-function', i]
+            else:
+                cmd += ['-no-doc-sections']

         for pattern in export_file_patterns:
             for f in glob.glob(env.config.kerneldoc_srctree + '/' + pattern):
@@ -121,13 +140,7 @@ class KernelDocDirective(Directive):
                     lineoffset += 1

             node = nodes.section()
-            buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
-            self.state.memo.title_styles, self.state.memo.section_level = [], 0
-            try:
-                self.state.nested_parse(result, 0, node, match_titles=1)
-            finally:
-                self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+            self.do_parse(result, node)

             return node.children

@@ -136,6 +149,20 @@ class KernelDocDirective(Directive):
                            (" ".join(cmd), str(e)))
             return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]

+    def do_parse(self, result, node):
+        if Use_SSI:
+            with switch_source_input(self.state, result):
+                self.state.nested_parse(result, 0, node, match_titles=1)
+        else:
+            save = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
+            self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
+            self.state.memo.title_styles, self.state.memo.section_level = [], 0
+            try:
+                self.state.nested_parse(result, 0, node, match_titles=1)
+            finally:
+                self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = save
+
+
 def setup(app):
     app.add_config_value('kerneldoc_bin', None, 'env')
     app.add_config_value('kerneldoc_srctree', None, 'env')
--
2.25.0

[PATCH v2 1/5] doc: update doc/sphinx/kerneldoc.py

[Tom Rini]

On Fri, Feb 21, 2020 at 06:23:59PM +0100, Heinrich Schuchardt wrote:

> Update doc/sphinx/kerneldoc.py from Linux next-20200219 to avoid warnings
> like:
> 
> doc/sphinx/kerneldoc.py:125: RemovedInSphinx20Warning:
> AutodocReporter is now deprecated. Use
> sphinx.util.docutils.switch_source_input() instead.
>   self.state.memo.reporter =
>   	AutodocReporter(result, self.state.memo.reporter)
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200225/c334a3c4/attachment.sig>

[PATCH v2 2/5] travis: build HTML docs

[Heinrich Schuchardt]

Several patches delivered incorrect restructured text as documentation.
We should be able to discover this in Travis CI.

Provide a build step for 'make htmldocs'.

Add required package graphviz.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
---
v2:
	change sequence of targets: put htmldocs after cppcheck
---
 .travis.yml | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index e6db9d6a72..911c85e130 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -38,6 +38,7 @@ addons:
     - libisl15
     - clang-7
     - srecord
+    - graphviz

 install:
  # Clone uboot-test-hooks
@@ -352,6 +353,10 @@ matrix:
     - name: "cppcheck"
       script:
         - cppcheck --force --quiet --inline-suppr .
+    # build HTML documentation
+    - name: "htmldocs"
+      script:
+        - make htmldocs
     # search for TODO within source tree
     - name: "grep TODO"
       script:
--
2.25.0

[PATCH v2 2/5] travis: build HTML docs

[Bin Meng]

On Sat, Feb 22, 2020 at 1:24 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Several patches delivered incorrect restructured text as documentation.
> We should be able to discover this in Travis CI.
>
> Provide a build step for 'make htmldocs'.
>
> Add required package graphviz.
>
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> ---
> v2:
>         change sequence of targets: put htmldocs after cppcheck
> ---
>  .travis.yml | 5 +++++
>  1 file changed, 5 insertions(+)
>

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

[PATCH v2 2/5] travis: build HTML docs

[Tom Rini]

On Fri, Feb 21, 2020 at 06:24:00PM +0100, Heinrich Schuchardt wrote:

> Several patches delivered incorrect restructured text as documentation.
> We should be able to discover this in Travis CI.
> 
> Provide a build step for 'make htmldocs'.
> 
> Add required package graphviz.
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200225/eb7902df/attachment.sig>

[PATCH v2 3/5] gitlab: build HTML documentation

[Heinrich Schuchardt]

Several patches delivered incorrect restructured text as documentation. We
should be able to discover this in Gitlab CI.

Provide a build step for 'make htmldocs'.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
---
v2:
	no change
---
 .gitlab-ci.yml | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d486e72042..652ed3b501 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -122,6 +122,13 @@ grep TODO/FIXME/HACK:
     # search for HACK within source tree and ignore HACKKIT board
     - grep -r HACK . | grep -v HACKKIT

+# build HTML documentation
+htmldocs:
+  tags: [ 'all' ]
+  stage: testsuites
+  script:
+    - make htmldocs
+
 # some statistics about the code base
 sloccount:
   tags: [ 'all' ]
--
2.25.0

[PATCH v2 3/5] gitlab: build HTML documentation

[Tom Rini]

On Fri, Feb 21, 2020 at 06:24:01PM +0100, Heinrich Schuchardt wrote:

> Several patches delivered incorrect restructured text as documentation. We
> should be able to discover this in Gitlab CI.
> 
> Provide a build step for 'make htmldocs'.
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200225/ea661448/attachment.sig>

[PATCH v2 4/5] azure: build HTML documentation

[Heinrich Schuchardt]

Several patches delivered incorrect restructured text as documentation. We
should be able to discover this in Azure CI.

Provide a build step for 'make htmldocs'.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
---
v2:
	no change
---
 .azure-pipelines.yml | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml
index c22095830c..129035a1a5 100644
--- a/.azure-pipelines.yml
+++ b/.azure-pipelines.yml
@@ -54,6 +54,16 @@ jobs:
     steps:
       - script: cppcheck --force --quiet --inline-suppr .

+  - job: htmldocs
+    displayName: 'Build HTML documentation'
+    pool:
+      vmImage: $(ubuntu_vm)
+    container:
+      image: $(ci_runner_image)
+      options: $(container_option)
+    steps:
+      - script: make htmldocs
+
   - job: todo
     displayName: 'Search for TODO within source tree'
     pool:
--
2.25.0

[PATCH v2 4/5] azure: build HTML documentation

[Tom Rini]

On Fri, Feb 21, 2020 at 06:24:02PM +0100, Heinrich Schuchardt wrote:

> Several patches delivered incorrect restructured text as documentation. We
> should be able to discover this in Azure CI.
> 
> Provide a build step for 'make htmldocs'.
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200225/a4e0bad6/attachment.sig>

[PATCH v2 5/5] doc/Makefile: turn warnings into errors

[Heinrich Schuchardt]

Several patches delivered incorrect restructured text as documentation. We
should be able to discover this in Travis CI, Gitlab CI, or Azure CI.

So let us turn all build warnings into errors.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
---
v2:
	change commit message
---
 doc/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/Makefile b/doc/Makefile
index 5135a96e88..0e0da5666f 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -56,6 +56,7 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
 	PYTHONDONTWRITEBYTECODE=1 \
 	BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
 	$(SPHINXBUILD) \
+	-W \
 	-b $2 \
 	-c $(abspath $(srctree)/$(src)) \
 	-d $(abspath $(BUILDDIR)/.doctrees/$3) \
--
2.25.0

[PATCH v2 5/5] doc/Makefile: turn warnings into errors

[Bin Meng]

On Sat, Feb 22, 2020 at 1:25 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Several patches delivered incorrect restructured text as documentation. We
> should be able to discover this in Travis CI, Gitlab CI, or Azure CI.
>
> So let us turn all build warnings into errors.
>
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> ---
> v2:
>         change commit message
> ---
>  doc/Makefile | 1 +
>  1 file changed, 1 insertion(+)
>

Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

[PATCH v2 5/5] doc/Makefile: turn warnings into errors

[Tom Rini]

On Fri, Feb 21, 2020 at 06:24:03PM +0100, Heinrich Schuchardt wrote:

> Several patches delivered incorrect restructured text as documentation. We
> should be able to discover this in Travis CI, Gitlab CI, or Azure CI.
> 
> So let us turn all build warnings into errors.
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200225/c09fa54d/attachment.sig>