[RFC PATCH 0/5] doc-rst: improvements Sphinx's C-domain

[RFC PATCH 0/5] doc-rst: improvements Sphinx's C-domain

[Markus Heiser]

From: Markus Heiser <markus.heiser@darmarIT.de>

Hi,

this is my approach to eliminate some distortions we have with the c/cpp Sphinx
domains. The C domain is simple: it assumes that all functions, enums, etc
are global, e. g. there should be just one function called "ioctl", or "open".
With the 'name' option e.g.:

    .. c:function:: int ioctl( int fd, int request )
       :name: VIDIOC_LOG_STATUS

we can rename those functions. Another nice feature around this *global*
namespace topic is, that the *duplicate C object description* warnings for
function declarations are moved to the nitpicky mode.

Thanks for your comments

  -- Markus --

Markus Heiser (5):
  doc-rst: add boilerplate to customize c-domain
  doc-rst:c-domain: ref-name of a function declaration
  doc-rst: moved *duplicate* warnings to nitpicky mode
  doc-rst: Revert "kernel-doc: fix handling of address_space tags"
  doc-rst: migrate ioctl CEC_DQEVENT to c-domain

 Documentation/conf.py                            |   2 +-
 Documentation/kernel-documentation.rst           |  29 +++++++
 Documentation/media/uapi/cec/cec-func-open.rst   |   2 +-
 Documentation/media/uapi/cec/cec-ioc-dqevent.rst |   5 +-
 Documentation/sphinx/cdomain.py                  | 102 +++++++++++++++++++++++
 scripts/kernel-doc                               |   3 -
 6 files changed, 136 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/sphinx/cdomain.py

-- 
2.7.4

[PATCH 1/5] doc-rst: add boilerplate to customize c-domain

[Markus Heiser]

From: Markus Heiser <markus.heiser@darmarIT.de>

Add a sphinx-extension to customize the sphinx c-domain.  No functional
changes right yet, just the boilerplate code.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
 Documentation/conf.py           |  2 +-
 Documentation/sphinx/cdomain.py | 44 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/sphinx/cdomain.py

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 5c06b01..dc46d23 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -29,7 +29,7 @@ from load_config import loadConfig
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kernel-doc', 'rstFlatTable', 'kernel_include']
+extensions = ['kernel-doc', 'rstFlatTable', 'kernel_include', 'cdomain']
 
 # Gracefully handle missing rst2pdf.
 try:
diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
new file mode 100644
index 0000000..c32387a
--- /dev/null
+++ b/Documentation/sphinx/cdomain.py
@@ -0,0 +1,44 @@
+# -*- coding: utf-8; mode: python -*-
+u"""
+    cdomain
+    ~~~~~~~
+
+    Replacement for the sphinx c-domain.
+
+    :copyright:  Copyright (C) 2016  Markus Heiser
+    :license:    GPL Version 2, June 1991 see Linux/COPYING for details.
+"""
+
+from sphinx.domains.c import CObject as Base_CObject
+from sphinx.domains.c import CDomain as Base_CDomain
+
+__version__  = '1.0'
+
+def setup(app):
+
+    app.override_domain(CDomain)
+
+    return dict(
+        version = __version__
+        , parallel_read_safe = True
+        , parallel_write_safe = True
+    )
+
+class CObject(Base_CObject):
+
+    """
+    Description of a C language object.
+    """
+
+class CDomain(Base_CDomain):
+
+    """C language domain."""
+    name = 'c'
+    label = 'C'
+    directives = {
+        'function': CObject,
+        'member':   CObject,
+        'macro':    CObject,
+        'type':     CObject,
+        'var':      CObject,
+    }
-- 
2.7.4

[PATCH 2/5] doc-rst:c-domain: ref-name of a function declaration

[Markus Heiser]

From: Markus Heiser <markus.heiser@darmarIT.de>

Add option 'name' to the "c:function:" directive.  With option 'name'
the ref-name of a function can be modified. E.g.::

    .. c:function:: int ioctl( int fd, int request )
       :name: VIDIOC_LOG_STATUS

The func-name (e.g. ioctl) remains in the output but the ref-name
changed from ``ioctl`` to ``VIDIOC_LOG_STATUS``. The index entry for
this function is also changed to ``VIDIOC_LOG_STATUS`` and the function
can now referenced by::

    :c:func:`VIDIOC_LOG_STATUS`

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
 Documentation/kernel-documentation.rst | 29 +++++++++++++++++++++++++++++
 Documentation/sphinx/cdomain.py        | 31 +++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+)

diff --git a/Documentation/kernel-documentation.rst b/Documentation/kernel-documentation.rst
index 391decc..a0dcae1 100644
--- a/Documentation/kernel-documentation.rst
+++ b/Documentation/kernel-documentation.rst
@@ -107,6 +107,35 @@ Here are some specific guidelines for the kernel documentation:
   the order as encountered."), having the higher levels the same overall makes
   it easier to follow the documents.
 
+
+the C domain
+------------
+
+The `Sphinx C Domain`_ (name c) is suited for documentation of C API. E.g. a
+function prototype:
+
+.. code-block:: rst
+
+    .. c:function:: int ioctl( int fd, int request )
+
+The C domain of the kernel-doc has some additional features. E.g. you can
+*rename* the reference name of a function with a common name like ``open`` or
+``ioctl``:
+
+.. code-block:: rst
+
+     .. c:function:: int ioctl( int fd, int request )
+        :name: VIDIOC_LOG_STATUS
+
+The func-name (e.g. ioctl) remains in the output but the ref-name changed from
+``ioctl`` to ``VIDIOC_LOG_STATUS``. The index entry for this function is also
+changed to ``VIDIOC_LOG_STATUS`` and the function can now referenced by:
+
+.. code-block:: rst
+
+     :c:func:`VIDIOC_LOG_STATUS`
+
+
 list tables
 -----------
 
diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
index c32387a..99cd035 100644
--- a/Documentation/sphinx/cdomain.py
+++ b/Documentation/sphinx/cdomain.py
@@ -7,8 +7,24 @@ u"""
 
     :copyright:  Copyright (C) 2016  Markus Heiser
     :license:    GPL Version 2, June 1991 see Linux/COPYING for details.
+
+    List of customizations:
+
+    * Add option 'name' to the "c:function:" directive.  With option 'name' the
+      ref-name of a function can be modified. E.g.::
+
+          .. c:function:: int ioctl( int fd, int request )
+             :name: VIDIOC_LOG_STATUS
+
+      The func-name (e.g. ioctl) remains in the output but the ref-name changed
+      from 'ioctl' to 'VIDIOC_LOG_STATUS'. The function is referenced by::
+
+          * :c:func:`VIDIOC_LOG_STATUS` or
+          * :any:`VIDIOC_LOG_STATUS` (``:any:`` needs sphinx 1.3)
 """
 
+from docutils.parsers.rst import directives
+
 from sphinx.domains.c import CObject as Base_CObject
 from sphinx.domains.c import CDomain as Base_CDomain
 
@@ -29,6 +45,21 @@ class CObject(Base_CObject):
     """
     Description of a C language object.
     """
+    option_spec = {
+        "name" : directives.unchanged
+    }
+
+    def handle_signature(self, sig, signode):
+        """Transform a C signature into RST nodes."""
+        fullname = super(CObject, self).handle_signature(sig, signode)
+        if "name" in self.options:
+            if self.objtype == 'function':
+                fullname = self.options["name"]
+            else:
+                # FIXME: handle :name: value of other declaration types?
+                pass
+        return fullname
+
 
 class CDomain(Base_CDomain):
 
-- 
2.7.4

[PATCH 3/5] doc-rst: moved *duplicate* warnings to nitpicky mode

[Markus Heiser]

From: Markus Heiser <markus.heiser@darmarIT.de>

Moved the *duplicate C object description* warnings for function
declarations in the nitpicky mode. In nitpick mode, you can suppress
those warnings (e.g. ioctl) with::

  nitpicky = True
  nitpick_ignore = [
      ("c:func", "ioctl"),
  ]

See Sphinx documentation for the config values for ``nitpick`` and
``nitpick_ignore`` [1].

With this change all the ".. cpp:function:: int ioctl(..)" descriptions
(found in the media book) can be migrated to ".. c:function:: int
ioctl(..)", without getting any warnings. E.g.::

  .. cpp:function:: int ioctl( int fd, int request, struct cec_event *argp )

  .. c:function:: int ioctl( int fd, int request, struct cec_event *argp )

The main effect, is that we get those *CPP-types* back into Sphinx's C-
namespace and we need no longer to distinguish between c/cpp references,
when we refer a function like the ioctl.

[1] http://www.sphinx-doc.org/en/stable/config.html?highlight=nitpick#confval-nitpicky

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
 Documentation/sphinx/cdomain.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
index 99cd035..cb4e8c9 100644
--- a/Documentation/sphinx/cdomain.py
+++ b/Documentation/sphinx/cdomain.py
@@ -10,6 +10,10 @@ u"""
 
     List of customizations:
 
+    * Moved the *duplicate C object description* warnings for function
+      declarations in the nitpicky mode. See Sphinx documentation for
+      the config values for ``nitpick`` and ``nitpick_ignore``.
+
     * Add option 'name' to the "c:function:" directive.  With option 'name' the
       ref-name of a function can be modified. E.g.::
 
@@ -60,6 +64,29 @@ class CObject(Base_CObject):
                 pass
         return fullname
 
+    def add_target_and_index(self, name, sig, signode):
+        # for C API items we add a prefix since names are usually not qualified
+        # by a module name and so easily clash with e.g. section titles
+        targetname = 'c.' + name
+        if targetname not in self.state.document.ids:
+            signode['names'].append(targetname)
+            signode['ids'].append(targetname)
+            signode['first'] = (not self.names)
+            self.state.document.note_explicit_target(signode)
+            inv = self.env.domaindata['c']['objects']
+            if (name in inv and self.env.config.nitpicky):
+                if self.objtype == 'function':
+                    if ('c:func', name) not in self.env.config.nitpick_ignore:
+                        self.state_machine.reporter.warning(
+                            'duplicate C object description of %s, ' % name +
+                            'other instance in ' + self.env.doc2path(inv[name][0]),
+                            line=self.lineno)
+            inv[name] = (self.env.docname, self.objtype)
+
+        indextext = self.get_index_text(name)
+        if indextext:
+            self.indexnode['entries'].append(('single', indextext,
+                                              targetname, '', None))
 
 class CDomain(Base_CDomain):
 
-- 
2.7.4

[PATCH 4/5] doc-rst: Revert "kernel-doc: fix handling of address_space tags"

[Markus Heiser]

From: Markus Heiser <markus.heiser@darmarIT.de>

This reverts commit a88b1672d4ddf9895eb53e6980926d5e960dea8e.

>From the origin comit log::

  The RST cpp:function handler is very pedantic: it doesn't allow any
  macros like __user on it

Since the kernel-doc parser does NOT make use of the cpp:domain, there
is no need to change the kernel-doc parser eleminating the address_space
tags.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
 scripts/kernel-doc | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 4f2e904..ba081c7 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1849,9 +1849,6 @@ sub output_function_rst(%) {
 	$count++;
 	$type = $args{'parametertypes'}{$parameter};
 
-	# RST doesn't like address_space tags at function prototypes
-	$type =~ s/__(user|kernel|iomem|percpu|pmem|rcu)\s*//;
-
 	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
 	    # pointer-to-function
 	    print $1 . $parameter . ") (" . $2;
-- 
2.7.4

[PATCH 5/5] doc-rst: migrate ioctl CEC_DQEVENT to c-domain

[Markus Heiser]

From: Markus Heiser <markus.heiser@darmarIT.de>

This is only one example, demonstrating the benefits of the patch
series.  The CEC_DQEVENT ioctl is migrated to the sphinx c-domain and
referred by ":name: CEC_DQEVENT".

With this change the indirection using ":ref:`CEC_DQEVENT` is no longer
needed, we can refer the ioctl directly with ":c:func:`CEC_DQEVENT`". As
addition in the index, there is a entry "CEC_DQEVENT (C function)".

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
 Documentation/media/uapi/cec/cec-func-open.rst   | 2 +-
 Documentation/media/uapi/cec/cec-ioc-dqevent.rst | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/media/uapi/cec/cec-func-open.rst b/Documentation/media/uapi/cec/cec-func-open.rst
index 38fd7e0..7c0f981 100644
--- a/Documentation/media/uapi/cec/cec-func-open.rst
+++ b/Documentation/media/uapi/cec/cec-func-open.rst
@@ -32,7 +32,7 @@ Arguments
     Open flags. Access mode must be ``O_RDWR``.
 
     When the ``O_NONBLOCK`` flag is given, the
-    :ref:`CEC_RECEIVE <CEC_RECEIVE>` and :ref:`CEC_DQEVENT <CEC_DQEVENT>` ioctls
+    :ref:`CEC_RECEIVE <CEC_RECEIVE>` and :c:func:`CEC_DQEVENT` ioctls
     will return the ``EAGAIN`` error code when no message or event is available, and
     ioctls :ref:`CEC_TRANSMIT <CEC_TRANSMIT>`,
     :ref:`CEC_ADAP_S_PHYS_ADDR <CEC_ADAP_S_PHYS_ADDR>` and
diff --git a/Documentation/media/uapi/cec/cec-ioc-dqevent.rst b/Documentation/media/uapi/cec/cec-ioc-dqevent.rst
index 7a6d6d0..4e12e6c 100644
--- a/Documentation/media/uapi/cec/cec-ioc-dqevent.rst
+++ b/Documentation/media/uapi/cec/cec-ioc-dqevent.rst
@@ -15,7 +15,8 @@ CEC_DQEVENT - Dequeue a CEC event
 Synopsis
 ========
 
-.. cpp:function:: int ioctl( int fd, int request, struct cec_event *argp )
+.. c:function:: int ioctl( int fd, int request, struct cec_event *argp )
+   :name: CEC_DQEVENT
 
 Arguments
 =========
@@ -36,7 +37,7 @@ Description
    and is currently only available as a staging kernel module.
 
 CEC devices can send asynchronous events. These can be retrieved by
-calling :ref:`ioctl CEC_DQEVENT <CEC_DQEVENT>`. If the file descriptor is in
+calling :c:func:`CEC_DQEVENT`. If the file descriptor is in
 non-blocking mode and no event is pending, then it will return -1 and
 set errno to the ``EAGAIN`` error code.
 
-- 
2.7.4

Re: [RFC PATCH 0/5] doc-rst: improvements Sphinx's C-domain

[Jonathan Corbet]

On Mon, 15 Aug 2016 16:08:23 +0200
Markus Heiser <markus.heiser@darmarit.de> wrote:

> this is my approach to eliminate some distortions we have with the c/cpp Sphinx
> domains. The C domain is simple: it assumes that all functions, enums, etc
> are global, e. g. there should be just one function called "ioctl", or "open".
> With the 'name' option e.g.:
> 
>     .. c:function:: int ioctl( int fd, int request )
>        :name: VIDIOC_LOG_STATUS
> 
> we can rename those functions. Another nice feature around this *global*
> namespace topic is, that the *duplicate C object description* warnings for
> function declarations are moved to the nitpicky mode.

So these all look reasonable to me.  A lot of it seems aimed at making the
media docs go better; Mauro, have you tried it out and does it, indeed,
have this effect?

Thanks,

jon

Re: [RFC PATCH 0/5] doc-rst: improvements Sphinx's C-domain

[Jonathan Corbet]

On Mon, 15 Aug 2016 16:08:23 +0200
Markus Heiser <markus.heiser@darmarit.de> wrote:

> this is my approach to eliminate some distortions we have with the c/cpp Sphinx
> domains. The C domain is simple: it assumes that all functions, enums, etc
> are global, e. g. there should be just one function called "ioctl", or "open".
> With the 'name' option e.g.:
> 
>     .. c:function:: int ioctl( int fd, int request )
>        :name: VIDIOC_LOG_STATUS
> 
> we can rename those functions. Another nice feature around this *global*
> namespace topic is, that the *duplicate C object description* warnings for
> function declarations are moved to the nitpicky mode.

I've applied these to the docs tree, thanks.

jon