[PATCH 0/3] docs: Add automatic cross-reference for documentation pages

[PATCH 0/3] docs: Add automatic cross-reference for documentation pages

[Nícolas F. R. A. Prado]

Cross-referencing from a documentation page to another can be done using the
:doc:`doc-file` directive from Sphinx.
This however introduces markup that could be avoided to increase readability in
plain text.
This patch series adds automatic markup for cross-referencing between
documentation pages.

This patch series depends on [1].

The first patch refactors the automarkup script to split regex matching and
iteration into one function and text markup substitution into another.
This enables each regex to have its own function that substitutes its matches'
text for the appropriate markup and is necessary for the automatic markup of
documentation pages since it requires slightly different logic from marking up
C references.

The second patch adds automatic markup for cross-referencing documentation
pages by adding an appropriate regex and function to do the markup.
This enables a text like "See Documentation/doc-guide/sphinx.rst." to be
substituted by a cross-reference to that document without any additional markup.
Since this automarkup doesn't work with relative paths (they always need to
start at Documentation/), it will almost always require more typing than the
equivalent :doc:`sphinx`, but it's purpose is to avoid the markup, making it
more readable in plain text.
The .rst extension was left as optional in the regex matching, even though it
only makes sense for .rst documents, because I thought not specifying the
extension, like "See Documentation/doc-guide/sphinx." also made sense as a
cross-reference.

The third patch adds a section about cross-referencing in the "Writing
Documentation" chapter since there was none (other than the one regarding
kernel-doc).
The section describes how to use the new automatic cross-referencing for
documentation pages, and also how to use the already in place Sphinx doc
directive.
It also points to the kernel-doc document for information on cross-referencing
kernel-doc functions and types (using the new automarkup, so it also serves as a
testcase for it).

Thanks,
Nícolas

[1]: https://lore.kernel.org/linux-doc/20200903005747.3900333-1-nfraprado@protonmail.com/

Nícolas F. R. A. Prado (3):
  docs: Allow multiple automarkup functions
  docs: Add automatic cross-reference for documentation pages
  docs: Document cross-referencing between documentation pages

 Documentation/doc-guide/sphinx.rst |  17 ++++
 Documentation/sphinx/automarkup.py | 134 ++++++++++++++++++++---------
 2 files changed, 109 insertions(+), 42 deletions(-)

-- 
2.28.0

[PATCH 1/3] docs: Allow multiple automarkup functions

[Nícolas F. R. A. Prado]

The automarkup script previously matched expressions and substituted
them with markup to enable automatic cross-reference all in the same
function.

Split the expression matching iteration and the markup substitution into
different functions to make it easier to add new regular expressions and
functions to treat each of them.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
---
 Documentation/sphinx/automarkup.py | 97 +++++++++++++++++-------------
 1 file changed, 55 insertions(+), 42 deletions(-)

diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index 272eb2bdfab1..6c8e8475eddb 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -36,65 +36,78 @@ Skipfuncs = [ 'open', 'close', 'read', 'write', 'fcntl', 'mmap',
               'select', 'poll', 'fork', 'execve', 'clone', 'ioctl',
               'socket' ]
 
-#
-# Find all occurrences of C references (function() and struct/union/enum/typedef
-# type_name) and try to replace them with appropriate cross references.
-#
-def markup_c_refs(docname, app, node):
-    class_str = {RE_function: 'c-func', RE_type: 'c-type'}
-    reftype_str = {RE_function: 'function', RE_type: 'type'}
-
-    cdom = app.env.domains['c']
+def markup_refs(docname, app, node):
     t = node.astext()
     done = 0
     repl = [ ]
     #
-    # Sort all C references by the starting position in text
+    # Associate each regex with the function that will markup its matches
+    #
+    markup_func = {RE_type: markup_c_ref,
+                   RE_function: markup_c_ref}
+    match_iterators = [regex.finditer(t) for regex in markup_func]
     #
-    sorted_matches = sorted(chain(RE_type.finditer(t), RE_function.finditer(t)),
-                            key=lambda m: m.start())
+    # Sort all references by the starting position in text
+    #
+    sorted_matches = sorted(chain(*match_iterators), key=lambda m: m.start())
     for m in sorted_matches:
         #
         # Include any text prior to match as a normal text node.
         #
         if m.start() > done:
             repl.append(nodes.Text(t[done:m.start()]))
+
         #
-        # Go through the dance of getting an xref out of the C domain
-        #
-        target = m.group(2)
-        target_text = nodes.Text(m.group(0))
-        xref = None
-        if not (m.re == RE_function and target in Skipfuncs):
-            lit_text = nodes.literal(classes=['xref', 'c', class_str[m.re]])
-            lit_text += target_text
-            pxref = addnodes.pending_xref('', refdomain = 'c',
-                                          reftype = reftype_str[m.re],
-                                          reftarget = target, modname = None,
-                                          classname = None)
-            #
-            # XXX The Latex builder will throw NoUri exceptions here,
-            # work around that by ignoring them.
-            #
-            try:
-                xref = cdom.resolve_xref(app.env, docname, app.builder,
-                                         reftype_str[m.re], target, pxref,
-                                         lit_text)
-            except NoUri:
-                xref = None
-        #
-        # Toss the xref into the list if we got it; otherwise just put
-        # the function text.
+        # Call the function associated with the regex that matched this text and
+        # append its return to the text
         #
-        if xref:
-            repl.append(xref)
-        else:
-            repl.append(target_text)
+        repl.append(markup_func[m.re](docname, app, m))
+
         done = m.end()
     if done < len(t):
         repl.append(nodes.Text(t[done:]))
     return repl
 
+#
+# Try to replace a C reference (function() or struct/union/enum/typedef
+# type_name) with an appropriate cross reference.
+#
+def markup_c_ref(docname, app, match):
+    class_str = {RE_function: 'c-func', RE_type: 'c-type'}
+    reftype_str = {RE_function: 'function', RE_type: 'type'}
+
+    cdom = app.env.domains['c']
+    #
+    # Go through the dance of getting an xref out of the C domain
+    #
+    target = match.group(2)
+    target_text = nodes.Text(match.group(0))
+    xref = None
+    if not (match.re == RE_function and target in Skipfuncs):
+        lit_text = nodes.literal(classes=['xref', 'c', class_str[match.re]])
+        lit_text += target_text
+        pxref = addnodes.pending_xref('', refdomain = 'c',
+                                      reftype = reftype_str[match.re],
+                                      reftarget = target, modname = None,
+                                      classname = None)
+        #
+        # XXX The Latex builder will throw NoUri exceptions here,
+        # work around that by ignoring them.
+        #
+        try:
+            xref = cdom.resolve_xref(app.env, docname, app.builder,
+                                     reftype_str[match.re], target, pxref,
+                                     lit_text)
+        except NoUri:
+            xref = None
+    #
+    # Return the xref if we got it; otherwise just return the plain text.
+    #
+    if xref:
+        return xref
+    else:
+        return target_text
+
 def auto_markup(app, doctree, name):
     #
     # This loop could eventually be improved on.  Someday maybe we
@@ -108,7 +121,7 @@ def auto_markup(app, doctree, name):
     for para in doctree.traverse(nodes.paragraph):
         for node in para.traverse(nodes.Text):
             if not isinstance(node.parent, nodes.literal):
-                node.parent.replace(node, markup_c_refs(name, app, node))
+                node.parent.replace(node, markup_refs(name, app, node))
 
 def setup(app):
     app.connect('doctree-resolved', auto_markup)
-- 
2.28.0

[PATCH 2/3] docs: Add automatic cross-reference for documentation pages

[Nícolas F. R. A. Prado]

Cross-referencing to other documentation pages is possible using the
:doc:`doc-file` directive from Sphinx.

Add automatic markup for references to other documentation pages in the
format Documentation/subfolder/doc-file.rst (the extension being
optional).
This requires that the path be passed all the way from the Documentation
folder, which can be longer than passing a relative path through the
:doc: directive, but avoids the markup, making the text cleaner when
read in plain text.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
---
 Documentation/sphinx/automarkup.py | 39 +++++++++++++++++++++++++++++-
 1 file changed, 38 insertions(+), 1 deletion(-)

diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index 6c8e8475eddb..a1b0f554cd82 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -24,6 +24,11 @@ from itertools import chain
 #
 RE_function = re.compile(r'(([\w_][\w\d_]+)\(\))')
 RE_type = re.compile(r'(struct|union|enum|typedef)\s+([\w_][\w\d_]+)')
+#
+# Detects a reference to a documentation page of the form Documentation/... with
+# an optional extension
+#
+RE_doc = re.compile(r'Documentation(/[\w\-_/]+)(\.\w+)*')
 
 #
 # Many places in the docs refer to common system calls.  It is
@@ -44,7 +49,8 @@ def markup_refs(docname, app, node):
     # Associate each regex with the function that will markup its matches
     #
     markup_func = {RE_type: markup_c_ref,
-                   RE_function: markup_c_ref}
+                   RE_function: markup_c_ref,
+                   RE_doc: markup_doc_ref}
     match_iterators = [regex.finditer(t) for regex in markup_func]
     #
     # Sort all references by the starting position in text
@@ -108,6 +114,37 @@ def markup_c_ref(docname, app, match):
     else:
         return target_text
 
+#
+# Try to replace a documentation reference of the form Documentation/... with a
+# cross reference to that page
+#
+def markup_doc_ref(docname, app, match):
+    stddom = app.env.domains['std']
+    #
+    # Go through the dance of getting an xref out of the std domain
+    #
+    target = match.group(1)
+    xref = None
+    pxref = addnodes.pending_xref('', refdomain = 'std', reftype = 'doc',
+                                  reftarget = target, modname = None,
+                                  classname = None, refexplicit = False)
+    #
+    # XXX The Latex builder will throw NoUri exceptions here,
+    # work around that by ignoring them.
+    #
+    try:
+        xref = stddom.resolve_xref(app.env, docname, app.builder, 'doc',
+                                   target, pxref, None)
+    except NoUri:
+        xref = None
+    #
+    # Return the xref if we got it; otherwise just return the plain text.
+    #
+    if xref:
+        return xref
+    else:
+        return nodes.Text(match.group(0))
+
 def auto_markup(app, doctree, name):
     #
     # This loop could eventually be improved on.  Someday maybe we
-- 
2.28.0

[PATCH 3/3] docs: Document cross-referencing between documentation pages

[Nícolas F. R. A. Prado]

The syntax to cross-reference between documentation pages wasn't
documented anywhere.

Document the cross-referencing using the new automarkup for
Documentation/... and also Sphinx's doc directive for using relative
paths.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
---
 Documentation/doc-guide/sphinx.rst | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index f71ddd592aaa..896478baf570 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -337,6 +337,23 @@ Rendered as:
 
         - column 3
 
+Cross-referencing
+-----------------
+
+Cross-referencing from one documentation page to another can be done by passing
+the path to the file starting from the Documentation folder.
+For example, to cross-reference to this page (the .rst extension is optional)::
+
+    See Documentation/doc-guide/sphinx.rst.
+
+If you want to use a relative path, you need to use Sphinx's ``doc`` directive.
+For example, referencing this page from the same directory would be done as::
+
+    See :doc:`sphinx`.
+
+For information on cross-referencing to kernel-doc functions or types, see
+Documentation/doc-guide/kernel-doc.rst.
+
 .. _sphinx_kfigure:
 
 Figures & Images
-- 
2.28.0

Re: [PATCH 0/3] docs: Add automatic cross-reference for documentation pages

[Jonathan Corbet]

On Fri, 11 Sep 2020 13:34:27 +0000
Nícolas F. R. A. Prado <nfraprado@protonmail.com> wrote:

> Cross-referencing from a documentation page to another can be done using the
> :doc:`doc-file` directive from Sphinx.
> This however introduces markup that could be avoided to increase readability in
> plain text.
> This patch series adds automatic markup for cross-referencing between
> documentation pages.

Once again, this looks great.  I've applied it, many thanks!

jon