[PATCH v2 0/8] Some Sphinx improvements

[PATCH v2 0/8] Some Sphinx improvements

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Hi,

I have collected a few Sphinx-related improvements to improve depfile generation
and add some keyboard navigation. Hope you'll like it.

v2:
 - fix test 'output:' regression
 - fix javascript indentation (Paolo)
 - split "meson: remove explicit extensions dependency file list" (Paolo)

Marc-André Lureau (8):
  docs/sphinx: add loaded modules to generated depfile
  docs/sphinx: add static files to generated depfile
  docs/sphinx: add templates files to generated depfile
  tests/qapi-schema/meson: add depfile to sphinx doc
  meson: drop sphinx_extn_depends
  meson: drop sphinx_template_files
  docs/sphinx: set navigation_with_keys=True
  docs/sphinx: add 's' keyboard binding to focus search

 docs/conf.py                  |  7 ++++++-
 docs/meson.build              | 10 ----------
 docs/sphinx-static/custom.js  |  9 +++++++++
 docs/sphinx/depfile.py        | 19 +++++++++++++++++--
 tests/qapi-schema/meson.build |  6 ++++--
 5 files changed, 36 insertions(+), 15 deletions(-)
 create mode 100644 docs/sphinx-static/custom.js

-- 
2.33.0.721.g106298f7f9

[PATCH v2 1/8] docs/sphinx: add loaded modules to generated depfile

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/sphinx/depfile.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/docs/sphinx/depfile.py b/docs/sphinx/depfile.py
index 277fdf0f56..b6fb926df1 100644
--- a/docs/sphinx/depfile.py
+++ b/docs/sphinx/depfile.py
@@ -12,6 +12,7 @@
 
 import os
 import sphinx
+import sys
 
 __version__ = '1.0'
 
@@ -20,8 +21,17 @@ def get_infiles(env):
         yield env.doc2path(x)
         yield from ((os.path.join(env.srcdir, dep)
                     for dep in env.dependencies[x]))
+    for mod in sys.modules.values():
+        if hasattr(mod, '__file__'):
+            if mod.__file__:
+                yield mod.__file__
 
-def write_depfile(app, env):
+
+def write_depfile(app, exception):
+    if exception:
+        return
+
+    env = app.env
     if not env.config.depfile:
         return
 
@@ -42,7 +52,7 @@ def write_depfile(app, env):
 def setup(app):
     app.add_config_value('depfile', None, 'env')
     app.add_config_value('depfile_stamp', None, 'env')
-    app.connect('env-updated', write_depfile)
+    app.connect('build-finished', write_depfile)
 
     return dict(
         version = __version__,
-- 
2.33.0.721.g106298f7f9

[PATCH v2 2/8] docs/sphinx: add static files to generated depfile

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/sphinx/depfile.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/docs/sphinx/depfile.py b/docs/sphinx/depfile.py
index b6fb926df1..99539adb48 100644
--- a/docs/sphinx/depfile.py
+++ b/docs/sphinx/depfile.py
@@ -13,6 +13,7 @@
 import os
 import sphinx
 import sys
+from pathlib import Path
 
 __version__ = '1.0'
 
@@ -25,6 +26,10 @@ def get_infiles(env):
         if hasattr(mod, '__file__'):
             if mod.__file__:
                 yield mod.__file__
+    # this is perhaps going to include unused files:
+    for static_path in env.config.html_static_path:
+        for path in Path(static_path).rglob('*'):
+            yield str(path)
 
 
 def write_depfile(app, exception):
-- 
2.33.0.721.g106298f7f9

[PATCH v2 3/8] docs/sphinx: add templates files to generated depfile

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/conf.py           | 2 +-
 docs/sphinx/depfile.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/conf.py b/docs/conf.py
index ff6e92c6e2..edc2bf8fcb 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -74,7 +74,7 @@
 extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile', 'qapidoc']
 
 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
+templates_path = [os.path.join(qemu_docdir, '_templates')]
 
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
diff --git a/docs/sphinx/depfile.py b/docs/sphinx/depfile.py
index 99539adb48..afdcbcec6e 100644
--- a/docs/sphinx/depfile.py
+++ b/docs/sphinx/depfile.py
@@ -27,7 +27,7 @@ def get_infiles(env):
             if mod.__file__:
                 yield mod.__file__
     # this is perhaps going to include unused files:
-    for static_path in env.config.html_static_path:
+    for static_path in env.config.html_static_path + env.config.templates_path:
         for path in Path(static_path).rglob('*'):
             yield str(path)
 
-- 
2.33.0.721.g106298f7f9

[PATCH v2 4/8] tests/qapi-schema/meson: add depfile to sphinx doc

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/qapi-schema/meson.build | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index df5acfd08b..3676729fbb 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -238,10 +238,11 @@ if build_docs
   # fuzzy comparison if future Sphinx versions produce different text,
   # but for now the simple comparison suffices.
   qapi_doc_out = custom_target('QAPI rST doc',
-                               output: ['doc-good.txt'],
+                               output: ['doc-good.txt', 'doc-good.stamp'],
                                input: files('doc-good.json', 'doc-good.rst'),
                                build_by_default: true,
                                depend_files: sphinx_extn_depends,
+                               depfile: 'docs.d',
                                # We use -E to suppress Sphinx's caching, because
                                # we want it to always really run the QAPI doc
                                # generation code. It also means we don't
@@ -250,6 +251,8 @@ if build_docs
                                          '-b', 'text', '-E',
                                          '-c', meson.project_source_root() / 'docs',
                                          '-D', 'master_doc=doc-good',
+                                         '-Ddepfile=@DEPFILE@',
+                                         '-Ddepfile_stamp=@OUTPUT1@',
                                          meson.current_source_dir(),
                                          meson.current_build_dir()])
 
-- 
2.33.0.721.g106298f7f9

[PATCH v2 5/8] meson: drop sphinx_extn_depends

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Module dependencies is now handled by depfile.py.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/meson.build              | 9 +--------
 tests/qapi-schema/meson.build | 1 -
 2 files changed, 1 insertion(+), 9 deletions(-)

diff --git a/docs/meson.build b/docs/meson.build
index 19cce670a2..34fda6853d 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -37,13 +37,6 @@ endif
 if build_docs
   SPHINX_ARGS += ['-Dversion=' + meson.project_version(), '-Drelease=' + config_host['PKGVERSION']]
 
-  sphinx_extn_depends = [ meson.current_source_dir() / 'sphinx/depfile.py',
-                          meson.current_source_dir() / 'sphinx/hxtool.py',
-                          meson.current_source_dir() / 'sphinx/kerneldoc.py',
-                          meson.current_source_dir() / 'sphinx/kernellog.py',
-                          meson.current_source_dir() / 'sphinx/qapidoc.py',
-                          meson.current_source_dir() / 'sphinx/qmp_lexer.py',
-                          qapi_gen_depends ]
   sphinx_template_files = [ meson.project_source_root() / 'docs/_templates/footer.html' ]
 
   have_ga = have_tools and config_host.has_key('CONFIG_GUEST_AGENT')
@@ -77,7 +70,7 @@ if build_docs
                 output: 'docs.stamp',
                 input: files('conf.py'),
                 depfile: 'docs.d',
-                depend_files: [ sphinx_extn_depends, sphinx_template_files ],
+                depend_files: [ sphinx_template_files ],
                 command: [SPHINX_ARGS, '-Ddepfile=@DEPFILE@',
                           '-Ddepfile_stamp=@OUTPUT0@',
                           '-b', 'html', '-d', private_dir,
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index 3676729fbb..e505bf709a 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -241,7 +241,6 @@ if build_docs
                                output: ['doc-good.txt', 'doc-good.stamp'],
                                input: files('doc-good.json', 'doc-good.rst'),
                                build_by_default: true,
-                               depend_files: sphinx_extn_depends,
                                depfile: 'docs.d',
                                # We use -E to suppress Sphinx's caching, because
                                # we want it to always really run the QAPI doc
-- 
2.33.0.721.g106298f7f9

[PATCH v2 6/8] meson: drop sphinx_template_files

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Static files dependencies is now handled by depfile.py.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/meson.build | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/docs/meson.build b/docs/meson.build
index 34fda6853d..27c6e156ff 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -37,8 +37,6 @@ endif
 if build_docs
   SPHINX_ARGS += ['-Dversion=' + meson.project_version(), '-Drelease=' + config_host['PKGVERSION']]
 
-  sphinx_template_files = [ meson.project_source_root() / 'docs/_templates/footer.html' ]
-
   have_ga = have_tools and config_host.has_key('CONFIG_GUEST_AGENT')
 
   man_pages = {
@@ -70,7 +68,6 @@ if build_docs
                 output: 'docs.stamp',
                 input: files('conf.py'),
                 depfile: 'docs.d',
-                depend_files: [ sphinx_template_files ],
                 command: [SPHINX_ARGS, '-Ddepfile=@DEPFILE@',
                           '-Ddepfile_stamp=@OUTPUT0@',
                           '-b', 'html', '-d', private_dir,
-- 
2.33.0.721.g106298f7f9

[PATCH v2 7/8] docs/sphinx: set navigation_with_keys=True

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Allow navigating to the previous/next page using the keyboard's left and
right arrows. I wish this would be the default, and that the themes
would provide more key navigation, but that doesn't seem on the roadmap.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/conf.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/conf.py b/docs/conf.py
index edc2bf8fcb..f536483bc3 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -166,6 +166,7 @@
 if LooseVersion(sphinx_rtd_theme.__version__) >= LooseVersion("0.4.3"):
     html_theme_options = {
         "style_nav_header_background": "#802400",
+        "navigation_with_keys": True,
     }
 
 html_logo = os.path.join(qemu_docdir, "../ui/icons/qemu_128x128.png")
-- 
2.33.0.721.g106298f7f9

[PATCH v2 8/8] docs/sphinx: add 's' keyboard binding to focus search

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

This is pretty ubiquitous. ('/' is already taken by some browsers for
quick search)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/conf.py                 | 4 ++++
 docs/sphinx-static/custom.js | 9 +++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 docs/sphinx-static/custom.js

diff --git a/docs/conf.py b/docs/conf.py
index f536483bc3..3161b8b127 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -182,6 +182,10 @@
     'theme_overrides.css',
 ]
 
+html_js_files = [
+    'custom.js',
+]
+
 html_context = {
     "display_gitlab": True,
     "gitlab_user": "qemu-project",
diff --git a/docs/sphinx-static/custom.js b/docs/sphinx-static/custom.js
new file mode 100644
index 0000000000..71a8605305
--- /dev/null
+++ b/docs/sphinx-static/custom.js
@@ -0,0 +1,9 @@
+document.addEventListener('keydown', (event) => {
+    // find a better way to look it up?
+    let search_input = document.getElementsByName('q')[0];
+
+    if (event.code === 'KeyS' && document.activeElement !== search_input) {
+        event.preventDefault();
+        search_input.focus();
+    }
+});
-- 
2.33.0.721.g106298f7f9

Re: [PATCH v2 0/8] Some Sphinx improvements

[Paolo Bonzini]

On 15/10/21 12:53, marcandre.lureau@redhat.com wrote:
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> Hi,
> 
> I have collected a few Sphinx-related improvements to improve depfile generation
> and add some keyboard navigation. Hope you'll like it.
> 
> v2:
>   - fix test 'output:' regression
>   - fix javascript indentation (Paolo)
>   - split "meson: remove explicit extensions dependency file list" (Paolo)
> 
> Marc-André Lureau (8):
>    docs/sphinx: add loaded modules to generated depfile
>    docs/sphinx: add static files to generated depfile
>    docs/sphinx: add templates files to generated depfile
>    tests/qapi-schema/meson: add depfile to sphinx doc
>    meson: drop sphinx_extn_depends
>    meson: drop sphinx_template_files
>    docs/sphinx: set navigation_with_keys=True
>    docs/sphinx: add 's' keyboard binding to focus search
> 
>   docs/conf.py                  |  7 ++++++-
>   docs/meson.build              | 10 ----------
>   docs/sphinx-static/custom.js  |  9 +++++++++
>   docs/sphinx/depfile.py        | 19 +++++++++++++++++--
>   tests/qapi-schema/meson.build |  6 ++++--
>   5 files changed, 36 insertions(+), 15 deletions(-)
>   create mode 100644 docs/sphinx-static/custom.js
> 

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

I think the patches could be reorg'ed as 4, 1+5 squashed, 2+6 squashed, 
3, 7, 8, but that is not a requirement.

Paolo

Re: [PATCH v2 1/8] docs/sphinx: add loaded modules to generated depfile

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]

On Fri, Oct 15, 2021 at 6:57 AM <marcandre.lureau@redhat.com> wrote:

> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>
>
You also fix *when* the deps are generated. Small commit message blurb
would be nice.


> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>

Reviewed-by: John Snow <jsnow@redhat.com>


> ---
>  docs/sphinx/depfile.py | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/docs/sphinx/depfile.py b/docs/sphinx/depfile.py
> index 277fdf0f56..b6fb926df1 100644
> --- a/docs/sphinx/depfile.py
> +++ b/docs/sphinx/depfile.py
> @@ -12,6 +12,7 @@
>
>  import os
>  import sphinx
> +import sys
>
>  __version__ = '1.0'
>
> @@ -20,8 +21,17 @@ def get_infiles(env):
>          yield env.doc2path(x)
>          yield from ((os.path.join(env.srcdir, dep)
>                      for dep in env.dependencies[x]))
> +    for mod in sys.modules.values():
> +        if hasattr(mod, '__file__'):
> +            if mod.__file__:
> +                yield mod.__file__
>
>
(How annoying that built-in modules aren't a different class and that this
attribute really is just entirely missing sometimes ...)


> -def write_depfile(app, env):
> +
> +def write_depfile(app, exception):
> +    if exception:
> +        return
> +
> +    env = app.env
>      if not env.config.depfile:
>          return
>
> @@ -42,7 +52,7 @@ def write_depfile(app, env):
>  def setup(app):
>      app.add_config_value('depfile', None, 'env')
>      app.add_config_value('depfile_stamp', None, 'env')
> -    app.connect('env-updated', write_depfile)
> +    app.connect('build-finished', write_depfile)
>
>      return dict(
>          version = __version__,
> --
> 2.33.0.721.g106298f7f9
>
>
>

[-- Attachment #2: Type: text/html, Size: 3057 bytes --]

Re: [PATCH v2 0/8] Some Sphinx improvements

[John Snow]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Fri, Oct 15, 2021 at 6:57 AM <marcandre.lureau@redhat.com> wrote:

> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> Hi,
>
> I have collected a few Sphinx-related improvements to improve depfile
> generation
> and add some keyboard navigation. Hope you'll like it.
>
> v2:
>  - fix test 'output:' regression
>  - fix javascript indentation (Paolo)
>  - split "meson: remove explicit extensions dependency file list" (Paolo)
>
> Marc-André Lureau (8):
>   docs/sphinx: add loaded modules to generated depfile
>   docs/sphinx: add static files to generated depfile
>   docs/sphinx: add templates files to generated depfile
>   tests/qapi-schema/meson: add depfile to sphinx doc
>   meson: drop sphinx_extn_depends
>   meson: drop sphinx_template_files
>   docs/sphinx: set navigation_with_keys=True
>   docs/sphinx: add 's' keyboard binding to focus search
>
>  docs/conf.py                  |  7 ++++++-
>  docs/meson.build              | 10 ----------
>  docs/sphinx-static/custom.js  |  9 +++++++++
>  docs/sphinx/depfile.py        | 19 +++++++++++++++++--
>  tests/qapi-schema/meson.build |  6 ++++--
>  5 files changed, 36 insertions(+), 15 deletions(-)
>  create mode 100644 docs/sphinx-static/custom.js
>
> --
> 2.33.0.721.g106298f7f9
>
>
The javascript blurb selecting the correct DOM node being any better
probably requires the search input box itself being given an 'id'
parameter, but I don't know if that's something the theme we're using
allows us to customize in any way. I'm not a webdev at all, so ... I'll
assume it's fine?

Reviewed-by: John Snow <jsnow@redhat.com>

[-- Attachment #2: Type: text/html, Size: 2439 bytes --]