Xen-Devel Archive on lore.kernel.org
 help / color / Atom feed
* [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx
@ 2019-10-03 20:56 Andrew Cooper
  2019-10-03 20:56 ` [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0 Andrew Cooper
                   ` (3 more replies)
  0 siblings, 4 replies; 10+ messages in thread
From: Andrew Cooper @ 2019-10-03 20:56 UTC (permalink / raw)
  To: Xen-devel
  Cc: Juergen Gross, Lars Kurth, Stefano Stabellini, Julien Grall,
	Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper,
	Tim Deegan, Rich Persaud, Jan Beulich, Ian Jackson,
	Roger Pau Monné

Various pieces of Sphinx documentation improvements intended for inclusion
into Xen 4.13.  Rendered results can be viewed at

  https://andrewcoop-xen.readthedocs.io/en/docs-devel/index.html

with

  https://andrewcoop-xen.readthedocs.io/en/docs-devel/admin-guide/introduction.html
  https://andrewcoop-xen.readthedocs.io/en/docs-devel/glossary.html
  https://andrewcoop-xen.readthedocs.io/en/docs-devel/misc/tech-debt.html

being the notable additions from this series.

Andrew Cooper (4):
  docs/sphinx: License content with CC-BY-4.0
  docs/sphinx: Indent cleanup
  docs/sphinx: Introduction
  docs/sphinx: Technical Debt

 COPYING                                  |   3 +
 docs/README.source                       |  32 ++++++++
 docs/admin-guide/index.rst               |   5 +-
 docs/admin-guide/introduction.rst        |  40 ++++++++++
 docs/admin-guide/microcode-loading.rst   |   2 +
 docs/admin-guide/xen-overview.drawio.svg |  97 +++++++++++++++++++++++
 docs/conf.py                             |  12 ++-
 docs/glossary.rst                        |  52 +++++++++++++
 docs/guest-guide/index.rst               |   6 +-
 docs/guest-guide/x86/hypercall-abi.rst   |  52 +++++++------
 docs/guest-guide/x86/index.rst           |   6 +-
 docs/hypervisor-guide/code-coverage.rst  |   6 +-
 docs/hypervisor-guide/index.rst          |   6 +-
 docs/index.rst                           |  38 +++++++--
 docs/misc/tech-debt.rst                  | 130 +++++++++++++++++++++++++++++++
 15 files changed, 444 insertions(+), 43 deletions(-)
 create mode 100644 docs/README.source
 create mode 100644 docs/admin-guide/introduction.rst
 create mode 100644 docs/admin-guide/xen-overview.drawio.svg
 create mode 100644 docs/glossary.rst
 create mode 100644 docs/misc/tech-debt.rst

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0
  2019-10-03 20:56 [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Andrew Cooper
@ 2019-10-03 20:56 ` Andrew Cooper
  2019-10-07 12:01   ` Lars Kurth
  2019-10-08 13:09   ` Lars Kurth
  2019-10-03 20:56 ` [Xen-devel] [PATCH 2/4] docs/sphinx: Indent cleanup Andrew Cooper
                   ` (2 subsequent siblings)
  3 siblings, 2 replies; 10+ messages in thread
From: Andrew Cooper @ 2019-10-03 20:56 UTC (permalink / raw)
  To: Xen-devel
  Cc: Juergen Gross, Lars Kurth, Stefano Stabellini, Julien Grall,
	Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper,
	Tim Deegan, Rich Persaud, Jan Beulich, Ian Jackson

Creative Commons is a more common license than GPL for documentation purposes.
Switch to using CC-BY-4.0 to explicitly permit re-purposing and remixing of
the content.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Lars Kurth <lars.kurth@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Rich Persaud <persaur@gmail.com>
CC: Juergen Gross <jgross@suse.com>
---
 COPYING                                 |  3 +++
 docs/README.source                      | 32 ++++++++++++++++++++++++++++++++
 docs/admin-guide/index.rst              |  2 ++
 docs/admin-guide/microcode-loading.rst  |  2 ++
 docs/conf.py                            |  1 +
 docs/guest-guide/index.rst              |  2 ++
 docs/guest-guide/x86/hypercall-abi.rst  |  2 ++
 docs/guest-guide/x86/index.rst          |  2 ++
 docs/hypervisor-guide/code-coverage.rst |  2 ++
 docs/hypervisor-guide/index.rst         |  2 ++
 docs/index.rst                          |  2 ++
 11 files changed, 52 insertions(+)
 create mode 100644 docs/README.source

diff --git a/COPYING b/COPYING
index 310fd52c27..80fac091d3 100644
--- a/COPYING
+++ b/COPYING
@@ -47,6 +47,9 @@ various drivers, support functions and header files within Xen-aware
 Linux source trees. In all such cases, license terms are stated at the
 top of the file or in a COPYING file in the same directory.
 
+Sphinx documentation is licensed under CC-BY 4.0.  See
+docs/README.source for more specific information.
+
 In some cases, compatible 3rd party code has been imported into the
 Xen tree, retaining the original license, such as
   - AES-128 3.0
diff --git a/docs/README.source b/docs/README.source
new file mode 100644
index 0000000000..f20fa92c28
--- /dev/null
+++ b/docs/README.source
@@ -0,0 +1,32 @@
+Sphinx documentation:
+
+All source rendered by Sphinx is licensed under CC-BY-4.0.
+
+You are free to:
+  Share:
+    Copy and redistribute the material in any medium or format.
+  Adapt:
+    Remix, transform, and build upon the material for any purpose, even
+    commercially.
+
+Under the following terms:
+  Attribution:
+    You must give appropriate credit, provide a link to the license, and
+    indicate if changes were made. You may do so in any reasonable manner, but
+    not in any way that suggests the licensor endorses you or your use.
+  No additional restrictions:
+    You may not apply legal terms or technological measures that legally
+    restrict others from doing anything the license permits.
+
+See https://creativecommons.org/licenses/by/4.0/ for full details.
+
+This includes:
+  * All ReStructured Text files:          docs/*/*.rst
+  * The Sphinx configuration file:        docs/conf.py
+  * Content in Sphinx-exclusive subdirs:  docs/*-guide/*
+
+
+Other documentation:
+
+There are a variety of text documents in various formats.  These, given no
+explicit license guidance, fall under Xen's default GPL-2.0 license.
diff --git a/docs/admin-guide/index.rst b/docs/admin-guide/index.rst
index f725d75ebe..ad1f508a79 100644
--- a/docs/admin-guide/index.rst
+++ b/docs/admin-guide/index.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 Admin Guide
 ===========
 
diff --git a/docs/admin-guide/microcode-loading.rst b/docs/admin-guide/microcode-loading.rst
index 1858ed4627..8265b917a9 100644
--- a/docs/admin-guide/microcode-loading.rst
+++ b/docs/admin-guide/microcode-loading.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 Microcode Loading
 =================
 
diff --git a/docs/conf.py b/docs/conf.py
index 73b7b9bfa2..50e41501db 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -1,4 +1,5 @@
 # -*- coding: utf-8 -*-
+# SPDX-License-Identifier: CC-BY-4.0
 #
 # Configuration file for the Sphinx documentation builder.
 #
diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
index 108e0b8d77..03c5b37bd1 100644
--- a/docs/guest-guide/index.rst
+++ b/docs/guest-guide/index.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 Guest documentation
 ===================
 
diff --git a/docs/guest-guide/x86/hypercall-abi.rst b/docs/guest-guide/x86/hypercall-abi.rst
index dee25853d4..edb10b1b2e 100644
--- a/docs/guest-guide/x86/hypercall-abi.rst
+++ b/docs/guest-guide/x86/hypercall-abi.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 Hypercall ABI
 =============
 
diff --git a/docs/guest-guide/x86/index.rst b/docs/guest-guide/x86/index.rst
index a368392087..121cddca62 100644
--- a/docs/guest-guide/x86/index.rst
+++ b/docs/guest-guide/x86/index.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 x86
 ===
 
diff --git a/docs/hypervisor-guide/code-coverage.rst b/docs/hypervisor-guide/code-coverage.rst
index 6c7552d691..641aac25fc 100644
--- a/docs/hypervisor-guide/code-coverage.rst
+++ b/docs/hypervisor-guide/code-coverage.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 Code Coverage
 =============
 
diff --git a/docs/hypervisor-guide/index.rst b/docs/hypervisor-guide/index.rst
index cbcae398a2..7ba37b6e54 100644
--- a/docs/hypervisor-guide/index.rst
+++ b/docs/hypervisor-guide/index.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 Hypervisor documentation
 ========================
 
diff --git a/docs/index.rst b/docs/index.rst
index 31bb8927f2..7bd9955a97 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,3 +1,5 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
 The Xen Hypervisor documentation
 ================================
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Xen-devel] [PATCH 2/4] docs/sphinx: Indent cleanup
  2019-10-03 20:56 [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Andrew Cooper
  2019-10-03 20:56 ` [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0 Andrew Cooper
@ 2019-10-03 20:56 ` Andrew Cooper
  2019-10-08 12:09   ` Lars Kurth
  2019-10-03 20:56 ` [Xen-devel] [PATCH 4/4] docs/sphinx: Technical Debt Andrew Cooper
  2019-10-04  4:53 ` [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Jürgen Groß
  3 siblings, 1 reply; 10+ messages in thread
From: Andrew Cooper @ 2019-10-03 20:56 UTC (permalink / raw)
  To: Xen-devel
  Cc: Juergen Gross, Lars Kurth, Stefano Stabellini, Julien Grall,
	Wei Liu, Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper,
	Tim Deegan, Jan Beulich, Ian Jackson

Sphinx, its linters, and RST modes in common editors, expect 3 spaces of
indentation.  Some bits already conform to this expectation.  Update the
rest to match.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Lars Kurth <lars.kurth@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Juergen Gross <jgross@suse.com>
---
 docs/admin-guide/index.rst              |  2 +-
 docs/guest-guide/index.rst              |  4 +--
 docs/guest-guide/x86/hypercall-abi.rst  | 50 ++++++++++++++++-----------------
 docs/guest-guide/x86/index.rst          |  4 +--
 docs/hypervisor-guide/code-coverage.rst |  4 +--
 docs/hypervisor-guide/index.rst         |  4 +--
 docs/index.rst                          | 16 +++++------
 7 files changed, 42 insertions(+), 42 deletions(-)

diff --git a/docs/admin-guide/index.rst b/docs/admin-guide/index.rst
index ad1f508a79..1da7c8bf4d 100644
--- a/docs/admin-guide/index.rst
+++ b/docs/admin-guide/index.rst
@@ -4,4 +4,4 @@ Admin Guide
 ===========
 
 .. toctree::
-  microcode-loading
+   microcode-loading
diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
index 03c5b37bd1..5455c67479 100644
--- a/docs/guest-guide/index.rst
+++ b/docs/guest-guide/index.rst
@@ -4,6 +4,6 @@ Guest documentation
 ===================
 
 .. toctree::
-  :maxdepth: 2
+   :maxdepth: 2
 
-  x86/index
+   x86/index
diff --git a/docs/guest-guide/x86/hypercall-abi.rst b/docs/guest-guide/x86/hypercall-abi.rst
index edb10b1b2e..14c48929d7 100644
--- a/docs/guest-guide/x86/hypercall-abi.rst
+++ b/docs/guest-guide/x86/hypercall-abi.rst
@@ -14,22 +14,22 @@ Registers
 The registers used for hypercalls depends on the operating mode of the guest.
 
 .. list-table::
-  :header-rows: 1
+   :header-rows: 1
 
-  * - ABI
-    - Hypercall Index
-    - Parameters (1 - 6)
-    - Result
+   * - ABI
+     - Hypercall Index
+     - Parameters (1 - 6)
+     - Result
 
-  * - 64bit
-    - RAX
-    - RDI RSI RDX R10 R8 R9
-    - RAX
+   * - 64bit
+     - RAX
+     - RDI RSI RDX R10 R8 R9
+     - RAX
 
-  * - 32bit
-    - EAX
-    - EBX ECX EDX ESI EDI EBP
-    - EAX
+   * - 32bit
+     - EAX
+     - EBX ECX EDX ESI EDI EBP
+     - EAX
 
 32 and 64bit PV guests have an ABI fixed by their guest type.  The ABI for an
 HVM guest depends on whether the vCPU is operating in a 64bit segment or not
@@ -53,22 +53,22 @@ The exact sequence of instructions required to issue a hypercall differs
 between virtualisation mode and hardware vendor.
 
 .. list-table::
-  :header-rows: 1
+   :header-rows: 1
 
-  * - Guest
-    - Transfer instruction
+   * - Guest
+     - Transfer instruction
 
-  * - 32bit PV
-    - INT 0x82
+   * - 32bit PV
+     - INT 0x82
 
-  * - 64bit PV
-    - SYSCALL
+   * - 64bit PV
+     - SYSCALL
 
-  * - Intel HVM
-    - VMCALL
+   * - Intel HVM
+     - VMCALL
 
-  * - AMD HVM
-    - VMMCALL
+   * - AMD HVM
+     - VMMCALL
 
 To abstract away the details, Xen implements an interface known as the
 Hypercall Page.  This allows a guest to make a hypercall without needing to
@@ -91,7 +91,7 @@ To invoke a specific hypercall, ``call`` the relevant stub [3]_:
 
 .. code-block:: none
 
-  call hypercall_page + index * 32
+   call hypercall_page + index * 32
 
 There result is an ABI which is invariant of the exact operating mode or
 hardware vendor.  This is intended to simplify guest kernel interfaces by
diff --git a/docs/guest-guide/x86/index.rst b/docs/guest-guide/x86/index.rst
index 121cddca62..502968490d 100644
--- a/docs/guest-guide/x86/index.rst
+++ b/docs/guest-guide/x86/index.rst
@@ -4,6 +4,6 @@ x86
 ===
 
 .. toctree::
-  :maxdepth: 2
+   :maxdepth: 2
 
-  hypercall-abi
+   hypercall-abi
diff --git a/docs/hypervisor-guide/code-coverage.rst b/docs/hypervisor-guide/code-coverage.rst
index 641aac25fc..49c4a8ad3b 100644
--- a/docs/hypervisor-guide/code-coverage.rst
+++ b/docs/hypervisor-guide/code-coverage.rst
@@ -10,8 +10,8 @@ so some extra steps are required to collect and process the data.
 
 .. warning::
 
-  ARM doesn't currently boot when the final binary exceeds 2MB in size,
-  and the coverage build tends to exceed this limit.
+   ARM doesn't currently boot when the final binary exceeds 2MB in size,
+   and the coverage build tends to exceed this limit.
 
 
 Compiling Xen
diff --git a/docs/hypervisor-guide/index.rst b/docs/hypervisor-guide/index.rst
index 7ba37b6e54..8ea8fcb145 100644
--- a/docs/hypervisor-guide/index.rst
+++ b/docs/hypervisor-guide/index.rst
@@ -4,6 +4,6 @@ Hypervisor documentation
 ========================
 
 .. toctree::
-  :maxdepth: 2
+   :maxdepth: 2
 
-  code-coverage
+   code-coverage
diff --git a/docs/index.rst b/docs/index.rst
index 7bd9955a97..7b441c4180 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -5,8 +5,8 @@ The Xen Hypervisor documentation
 
 .. note::
 
-  Xen's Sphinx/RST documentation is a work in progress.  The existing
-  documentation can be found at https://xenbits.xen.org/docs/
+   Xen's Sphinx/RST documentation is a work in progress.  The existing
+   documentation can be found at https://xenbits.xen.org/docs/
 
 
 User documentation
@@ -18,9 +18,9 @@ preferred distribution, and is attempting to run virtual machines and
 configure the system.
 
 .. toctree::
-  :maxdepth: 2
+   :maxdepth: 2
 
-  admin-guide/index
+   admin-guide/index
 
 
 Guest documentation
@@ -31,9 +31,9 @@ intended for OS developers trying to use a Xen feature, and for Xen developers
 to avoid breaking things.
 
 .. toctree::
-  :maxdepth: 3
+   :maxdepth: 3
 
-  guest-guide/index
+   guest-guide/index
 
 
 Hypervisor developer documentation
@@ -44,6 +44,6 @@ who is building Xen from source, and is running the new hypervisor in some
 kind of development environment.
 
 .. toctree::
-  :maxdepth: 2
+   :maxdepth: 2
 
-  hypervisor-guide/index
+   hypervisor-guide/index
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Xen-devel] [PATCH 4/4] docs/sphinx: Technical Debt
  2019-10-03 20:56 [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Andrew Cooper
  2019-10-03 20:56 ` [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0 Andrew Cooper
  2019-10-03 20:56 ` [Xen-devel] [PATCH 2/4] docs/sphinx: Indent cleanup Andrew Cooper
@ 2019-10-03 20:56 ` Andrew Cooper
  2019-10-04  4:53 ` [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Jürgen Groß
  3 siblings, 0 replies; 10+ messages in thread
From: Andrew Cooper @ 2019-10-03 20:56 UTC (permalink / raw)
  To: Xen-devel
  Cc: Lars Kurth, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Andrew Cooper, Tim Deegan,
	Jan Beulich, Ian Jackson, Roger Pau Monné

This identifies various of areas technical debt, which either need to be, or
are being worked on, along with enough clarifying details for people to
follow.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Lars Kurth <lars.kurth@citrix.com>
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Roger Pau Monné <roger.pau@citrix.com>
---
 docs/conf.py            |  11 +++-
 docs/index.rst          |   8 +++
 docs/misc/tech-debt.rst | 130 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 148 insertions(+), 1 deletion(-)
 create mode 100644 docs/misc/tech-debt.rst

diff --git a/docs/conf.py b/docs/conf.py
index 50e41501db..0d2227f52e 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -53,7 +53,7 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = []
+extensions = ['sphinx.ext.extlinks']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -192,3 +192,12 @@
 
 # A list of files that should not be packed into the epub file.
 epub_exclude_files = ['search.html']
+
+
+# -- Configuration for external links ----------------------------------------
+
+extlinks = {
+    'xen-cs':
+        ('https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=%s',
+         'Xen c/s '),
+}
diff --git a/docs/index.rst b/docs/index.rst
index b8ab13178c..0a2af2db9d 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -59,3 +59,11 @@ Miscellanea
 .. toctree::
 
    glossary
+
+Unsorted
+--------
+
+.. toctree::
+   :maxdepth: 2
+
+   misc/tech-debt
diff --git a/docs/misc/tech-debt.rst b/docs/misc/tech-debt.rst
new file mode 100644
index 0000000000..172ba3bd51
--- /dev/null
+++ b/docs/misc/tech-debt.rst
@@ -0,0 +1,130 @@
+.. SPDX-License-Identifier: CC-BY-4.0
+
+Technical Debt
+==============
+
+Hypervisor
+----------
+
+CONFIG_PDX
+~~~~~~~~~~
+
+Xen uses the term MFN for Machine Frame Number, which is synonymous with
+Linux's PFN, and maps linearly to system/host/machine physical addresses.
+
+For every page of RAM, a ``struct page_info`` is needed for tracking purposes.
+In the simple case, the frametable is an array of ``struct page_info[]``
+indexed by MFN.
+
+However, this is inefficient when a system has banks of RAM at spread out in
+address space, as a large amount of space is wasted on frametable entries for
+non-existent frames.  This wastes both virtual address space and RAM.
+
+As a consequence, Xen has a compression scheme known as PDX which removes
+unused bits out of the middle of MFNs, to make a more tightly packed Page
+inDeX, which in turn reduces the size of the frametable for system.
+
+At the moment, PDX compression is unconditionally used.
+
+However, PDX compression does come with a cost in terms of the complexity to
+convert between PFNs and pages, which is a common operation in Xen.
+
+Typically, ARM32 systems do have RAM banks in discrete locations, and want to
+use PDX compression, while typically ARM64 and x86 systems have RAM packed
+from 0 with no holes.
+
+The goal of this work is to have ``CONFIG_PDX`` selected by ARM32 only.  This
+requires slightly untangling the memory management code in ARM and x86 to give
+it a clean compile boundary where PDX conversions are used.
+
+
+Waitqueue infrastructure
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Livepatching safety in Xen depends on all CPUs rendezvousing on the return to
+guest path, with no stack frame.  The vCPU waitqueue infrastructure undermines
+this safety by copying a stack frame sideways, and ``longjmp()``\-ing away.
+
+Waitqueues are only used by the introspection/mem_event/paging infrastructure,
+where the design of the rings causes some problems.  There is a single 4k page
+used for the ring, which serves both synchronous requests, and lossless async
+requests.  In practice, introspecting an 11-vcpu guest is sufficient to cause
+the waitqueue infrastructure to start to be used.
+
+A better design of ring would be to have a slot per vcpu for synchronous
+requests (simplifies producing and consuming of requests), and a multipage
+ring buffer (of negotiable size) with lossy semantics for async requests.
+
+A design such as this would guarantee that Xen never has to block waiting for
+userspace to create enough space on the ring for a vcpu to write state out.
+
+.. note::
+
+   There are other aspects of the existing ring infrastructure which are
+   driving a redesign, but these don't relate directly to the waitqueue
+   infrastructure and livepatching safety.
+
+   The most serious problem is that the ring infrastructure is GFN based,
+   which leaves the guest either able to mess with the ring, or a shattered
+   host superpage where the ring used to be, and the guest balloon driver able
+   to prevent the introspection agent from connecting/reconnecting the ring.
+
+As there are multiple compelling reasons to redesign the ring infrastructure,
+the plan is to introduce the new ring ABI, deprecate and remove the old ABI,
+and simply delete the waitqueue infrastructure at that point, rather than try
+to redesign livepatching from scratch in an attempt to cope with unwinding old
+stack frames.
+
+
+Dom0
+----
+
+Remove xenstored's dependencies on unstable interfaces
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Various xenstored implementations use libxc for two purposes.  It would be a
+substantial advantage to move xenstored onto entirely stable interfaces, which
+disconnects it from the internal of the libxc.
+
+1. Foreign mapping of the store ring
+
+   This is obsolete since :xen-cs:`6a2de353a9` (2012) which allocated grant
+   entries instead, to allow xenstored to function as a stub-domain without dom0
+   permissions.  :xen-cs:`38eeb3864d` dropped foreign mapping for cxenstored.
+   However, there are no OCaml bindings for libxengnttab.
+
+   Work Items:
+
+   * Minimal ``tools/ocaml/libs/xg/`` binding for ``tools/libs/gnttab/``.
+   * Replicate :xen-cs:`38eeb3864d` for oxenstored as well.
+
+2. Figuring out which domain(s) have gone away
+
+   Currently, the handling of domains is asymmetric.
+
+   * When a domain is created, the toolstack explicitly sends an
+     ``XS_INTRODUCE(domid, store mfn, store evtchn)`` message to xenstored, to
+     cause xenstored to connect to the guest ring, and fire the
+     ``@introduceDomain`` watch.
+
+   * When a domain is destroyed, Xen fires ``VIRQ_DOM_EXC`` which is bound by
+     xenstored, rather than the toolstack.  xenstored updates its idea of the
+     status of domains, and fires the ``@releaseDomain`` watch.
+
+     Xenstored uses ``xc_domain_getinfo()``, to work out which domain(s) have gone
+     away, and only cares about the shutdown status.
+
+     Furthermore, ``@releaseDomain`` (like ``VIRQ_DOM_EXC``) is a single-bit
+     message, which requires all listeners to evaluate whether the message applies
+     to them or not.  This results in a flurry of ``xc_domain_getinfo()`` calls
+     from multiple entities in the system, which all serialise on the domctl lock
+     in Xen.
+
+     Work Items:
+
+     * Figure out how shutdown status can be expressed in a stable way from Xen.
+     * Figure out if ``VIRQ_DOM_EXC`` and ``@releaseDomain`` can be extended
+       or superseded to carry at least a domid, to make domain shutdown scale
+       better.
+     * Figure out if ``VIRQ_DOM_EXC`` would better be bound by the toolstack,
+       rather than xenstored.
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx
  2019-10-03 20:56 [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Andrew Cooper
                   ` (2 preceding siblings ...)
  2019-10-03 20:56 ` [Xen-devel] [PATCH 4/4] docs/sphinx: Technical Debt Andrew Cooper
@ 2019-10-04  4:53 ` Jürgen Groß
  3 siblings, 0 replies; 10+ messages in thread
From: Jürgen Groß @ 2019-10-04  4:53 UTC (permalink / raw)
  To: Andrew Cooper, Xen-devel
  Cc: Lars Kurth, StefanoStabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, George Dunlap, Tim Deegan, Rich Persaud,
	Jan Beulich, Ian Jackson, Roger Pau Monné

On 03.10.19 22:56, Andrew Cooper wrote:
> Various pieces of Sphinx documentation improvements intended for inclusion
> into Xen 4.13.  Rendered results can be viewed at
> 
>    https://andrewcoop-xen.readthedocs.io/en/docs-devel/index.html
> 
> with
> 
>    https://andrewcoop-xen.readthedocs.io/en/docs-devel/admin-guide/introduction.html
>    https://andrewcoop-xen.readthedocs.io/en/docs-devel/glossary.html
>    https://andrewcoop-xen.readthedocs.io/en/docs-devel/misc/tech-debt.html
> 
> being the notable additions from this series.
> 
> Andrew Cooper (4):
>    docs/sphinx: License content with CC-BY-4.0
>    docs/sphinx: Indent cleanup
>    docs/sphinx: Introduction
>    docs/sphinx: Technical Debt
> 
>   COPYING                                  |   3 +
>   docs/README.source                       |  32 ++++++++
>   docs/admin-guide/index.rst               |   5 +-
>   docs/admin-guide/introduction.rst        |  40 ++++++++++
>   docs/admin-guide/microcode-loading.rst   |   2 +
>   docs/admin-guide/xen-overview.drawio.svg |  97 +++++++++++++++++++++++
>   docs/conf.py                             |  12 ++-
>   docs/glossary.rst                        |  52 +++++++++++++
>   docs/guest-guide/index.rst               |   6 +-
>   docs/guest-guide/x86/hypercall-abi.rst   |  52 +++++++------
>   docs/guest-guide/x86/index.rst           |   6 +-
>   docs/hypervisor-guide/code-coverage.rst  |   6 +-
>   docs/hypervisor-guide/index.rst          |   6 +-
>   docs/index.rst                           |  38 +++++++--
>   docs/misc/tech-debt.rst                  | 130 +++++++++++++++++++++++++++++++
>   15 files changed, 444 insertions(+), 43 deletions(-)
>   create mode 100644 docs/README.source
>   create mode 100644 docs/admin-guide/introduction.rst
>   create mode 100644 docs/admin-guide/xen-overview.drawio.svg
>   create mode 100644 docs/glossary.rst
>   create mode 100644 docs/misc/tech-debt.rst
> 

For the series:

Release-acked-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0
  2019-10-03 20:56 ` [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0 Andrew Cooper
@ 2019-10-07 12:01   ` Lars Kurth
  2019-10-07 12:29     ` Andrew Cooper
  2019-10-08 13:09   ` Lars Kurth
  1 sibling, 1 reply; 10+ messages in thread
From: Lars Kurth @ 2019-10-07 12:01 UTC (permalink / raw)
  To: Andrew Cooper, Xen-devel, Rich Persaud, Ian Jackson
  Cc: Juergen Gross, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, Tim \(Xen.org\),
	George Dunlap, Jan Beulich



On 03/10/2019, 21:56, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:

    Creative Commons is a more common license than GPL for documentation purposes.
    Switch to using CC-BY-4.0 to explicitly permit re-purposing and remixing of
    the content.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    ---
    CC: Lars Kurth <lars.kurth@citrix.com>
    CC: George Dunlap <George.Dunlap@eu.citrix.com>
    CC: Ian Jackson <ian.jackson@citrix.com>
    CC: Jan Beulich <JBeulich@suse.com>
    CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
    CC: Stefano Stabellini <sstabellini@kernel.org>
    CC: Tim Deegan <tim@xen.org>
    CC: Wei Liu <wl@xen.org>
    CC: Julien Grall <julien@xen.org>
    CC: Rich Persaud <persaur@gmail.com>
    CC: Juergen Gross <jgross@suse.com>
    ---
     COPYING                                 |  3 +++
     docs/README.source                      | 32 ++++++++++++++++++++++++++++++++
     docs/admin-guide/index.rst              |  2 ++
     docs/admin-guide/microcode-loading.rst  |  2 ++
     docs/conf.py                            |  1 +
     docs/guest-guide/index.rst              |  2 ++
     docs/guest-guide/x86/hypercall-abi.rst  |  2 ++
     docs/guest-guide/x86/index.rst          |  2 ++
     docs/hypervisor-guide/code-coverage.rst |  2 ++
     docs/hypervisor-guide/index.rst         |  2 ++
     docs/index.rst                          |  2 ++
     11 files changed, 52 insertions(+)
     create mode 100644 docs/README.source
    
    diff --git a/COPYING b/COPYING
    index 310fd52c27..80fac091d3 100644
    --- a/COPYING
    +++ b/COPYING
    @@ -47,6 +47,9 @@ various drivers, support functions and header files within Xen-aware
     Linux source trees. In all such cases, license terms are stated at the
     top of the file or in a COPYING file in the same directory.
     
    +Sphinx documentation is licensed under CC-BY 4.0.  See
    +docs/README.source for more specific information.
    +
     In some cases, compatible 3rd party code has been imported into the
     Xen tree, retaining the original license, such as
       - AES-128 3.0
    diff --git a/docs/README.source b/docs/README.source
    new file mode 100644
    index 0000000000..f20fa92c28
    --- /dev/null
    +++ b/docs/README.source
    @@ -0,0 +1,32 @@
    +Sphinx documentation:
    +
    +All source rendered by Sphinx is licensed under CC-BY-4.0.

Sorry for opening this can of worms. 

Although I had seen the discussion between Rich and you about this, I had 
not actually done any groundwork on the licensing. 

So, we have to look at two things:

* Compatibility:
   See https://creativecommons.org/2015/10/08/cc-by-sa-4-0-now-one-way-compatible-with-gplv3/ 
   This makes CC-BY-4.0 inbound compatible with GPLv3
   It's not clear to me whether GPLv2 is compatible with CC-BY 4.0: lack of publicly
   available information implies this is not the case 

* Output License
   But even if it is, the produced sphinx output would be GPLv2, not CC-BY 4.0
   This would even be true if none of the older GPLv2 docs portions were included, as
   the API docs generated from source are GPLv2

As such the statement "All source rendered by Sphinx is licensed under
CC-BY-4.0" is wrong. 

Although it is probably correct to say "All CC-BY 4.0 source rendered by
Sphinx is licensed under CC-BY-4.0", because Sphinx retains the source file
to html mapping and linkage in docs generation works differently
to linkage in code. 

I am wondering whether anyone else has come across this. This question in
particular goes back to Rich who made a very strong case for CC-BY-4.0 based
documentation. I don't think we would have an issue if the entire sphinx doc-set
is GPLv2 if most content is licensed under CC-BY-4.0, except that such an
approach would make re-using the entire sphinx generated docset messy.

We probably also want to maintain the capability to copy text from some
documentation freely into the source tree and vice versa, if needed. This is
particularly true for content in Technical Debt, user content (may end up in
man pages), etc.

Maybe the right approach would be to dually license the documentation
files using both GPLv2 and CC-BY 4.0 and quantifying this in the COPYING
file of the docs directory (starting from a specific date). We could eventually
re-license all the other stuff over time, which should be relatively straightforward
and/or exclude specific problematic directories.

Things like standardising say man pages to rst, would potentially also
create complexities with this patch, because of 
    +This includes:
    +  * All ReStructured Text files:          docs/*/*.rst

I don't want this to become a long-winded conversation during the 4.13 freeze.
Please keep this in mind when responding.

It may mean though, that we can't resolve this before 4.13 is released

Regards
Lars

 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0
  2019-10-07 12:01   ` Lars Kurth
@ 2019-10-07 12:29     ` Andrew Cooper
  2019-10-07 13:09       ` Lars Kurth
  0 siblings, 1 reply; 10+ messages in thread
From: Andrew Cooper @ 2019-10-07 12:29 UTC (permalink / raw)
  To: Lars Kurth, Xen-devel, Rich Persaud, Ian Jackson
  Cc: Juergen Gross, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, Tim \(Xen.org\),
	George Dunlap, Jan Beulich

On 07/10/2019 13:01, Lars Kurth wrote:
>
> On 03/10/2019, 21:56, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:
>
>     Creative Commons is a more common license than GPL for documentation purposes.
>     Switch to using CC-BY-4.0 to explicitly permit re-purposing and remixing of
>     the content.
>     
>     Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>     ---
>     CC: Lars Kurth <lars.kurth@citrix.com>
>     CC: George Dunlap <George.Dunlap@eu.citrix.com>
>     CC: Ian Jackson <ian.jackson@citrix.com>
>     CC: Jan Beulich <JBeulich@suse.com>
>     CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>     CC: Stefano Stabellini <sstabellini@kernel.org>
>     CC: Tim Deegan <tim@xen.org>
>     CC: Wei Liu <wl@xen.org>
>     CC: Julien Grall <julien@xen.org>
>     CC: Rich Persaud <persaur@gmail.com>
>     CC: Juergen Gross <jgross@suse.com>
>     ---
>      COPYING                                 |  3 +++
>      docs/README.source                      | 32 ++++++++++++++++++++++++++++++++
>      docs/admin-guide/index.rst              |  2 ++
>      docs/admin-guide/microcode-loading.rst  |  2 ++
>      docs/conf.py                            |  1 +
>      docs/guest-guide/index.rst              |  2 ++
>      docs/guest-guide/x86/hypercall-abi.rst  |  2 ++
>      docs/guest-guide/x86/index.rst          |  2 ++
>      docs/hypervisor-guide/code-coverage.rst |  2 ++
>      docs/hypervisor-guide/index.rst         |  2 ++
>      docs/index.rst                          |  2 ++
>      11 files changed, 52 insertions(+)
>      create mode 100644 docs/README.source
>     
>     diff --git a/COPYING b/COPYING
>     index 310fd52c27..80fac091d3 100644
>     --- a/COPYING
>     +++ b/COPYING
>     @@ -47,6 +47,9 @@ various drivers, support functions and header files within Xen-aware
>      Linux source trees. In all such cases, license terms are stated at the
>      top of the file or in a COPYING file in the same directory.
>      
>     +Sphinx documentation is licensed under CC-BY 4.0.  See
>     +docs/README.source for more specific information.
>     +
>      In some cases, compatible 3rd party code has been imported into the
>      Xen tree, retaining the original license, such as
>        - AES-128 3.0
>     diff --git a/docs/README.source b/docs/README.source
>     new file mode 100644
>     index 0000000000..f20fa92c28
>     --- /dev/null
>     +++ b/docs/README.source
>     @@ -0,0 +1,32 @@
>     +Sphinx documentation:
>     +
>     +All source rendered by Sphinx is licensed under CC-BY-4.0.
>
> Sorry for opening this can of worms. 
>
> Although I had seen the discussion between Rich and you about this, I had 
> not actually done any groundwork on the licensing. 
>
> So, we have to look at two things:
>
> * Compatibility:
>    See https://creativecommons.org/2015/10/08/cc-by-sa-4-0-now-one-way-compatible-with-gplv3/ 
>    This makes CC-BY-4.0 inbound compatible with GPLv3
>    It's not clear to me whether GPLv2 is compatible with CC-BY 4.0: lack of publicly
>    available information implies this is not the case 
>
> * Output License
>    But even if it is, the produced sphinx output would be GPLv2, not CC-BY 4.0
>    This would even be true if none of the older GPLv2 docs portions were included, as
>    the API docs generated from source are GPLv2
>
> As such the statement "All source rendered by Sphinx is licensed under
> CC-BY-4.0" is wrong.

At the moment, I (and therefore Citrix in practice) holds the copyright
on all rst content which exists in Xen.

The point of this patch is to get it licensed sensibly (and in
particular, not falling back to the GPL default) before 4.13 goes out of
the door.

The result therefore is uniformly CC-BY-4.0, with no GPL in sight.

> Although it is probably correct to say "All CC-BY 4.0 source rendered by
> Sphinx is licensed under CC-BY-4.0", because Sphinx retains the source file
> to html mapping and linkage in docs generation works differently
> to linkage in code. 
>
> I am wondering whether anyone else has come across this. This question in
> particular goes back to Rich who made a very strong case for CC-BY-4.0 based
> documentation. I don't think we would have an issue if the entire sphinx doc-set
> is GPLv2 if most content is licensed under CC-BY-4.0, except that such an
> approach would make re-using the entire sphinx generated docset messy.
>
> We probably also want to maintain the capability to copy text from some
> documentation freely into the source tree and vice versa, if needed. This is
> particularly true for content in Technical Debt, user content (may end up in
> man pages), etc.

I disagree, insofar as blindly copying notes out of source code and into
the sphinx documentation is liable to get a -1 from me.

The types of style/language which are appropriate for these two cases
are a disjoint set.

>
> Maybe the right approach would be to dually license the documentation
> files using both GPLv2 and CC-BY 4.0 and quantifying this in the COPYING
> file of the docs directory (starting from a specific date). We could eventually
> re-license all the other stuff over time, which should be relatively straightforward
> and/or exclude specific problematic directories.

I don't see how that helps for blindly copying things in.  They'd still
be GPL2 strictly, and would need explicit relicensing to become dual
licensed, at which point, why not go straight for consistency?

>
> Things like standardising say man pages to rst, would potentially also
> create complexities with this patch, because of 
>     +This includes:
>     +  * All ReStructured Text files:          docs/*/*.rst
>
> I don't want this to become a long-winded conversation during the 4.13 freeze.
> Please keep this in mind when responding.
>
> It may mean though, that we can't resolve this before 4.13 is released

Getting this fixed is a blocker for 4.13.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0
  2019-10-07 12:29     ` Andrew Cooper
@ 2019-10-07 13:09       ` Lars Kurth
  0 siblings, 0 replies; 10+ messages in thread
From: Lars Kurth @ 2019-10-07 13:09 UTC (permalink / raw)
  To: Andrew Cooper, Xen-devel, Rich Persaud, Ian Jackson
  Cc: Juergen Gross, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, Tim \(Xen.org\),
	George Dunlap, Jan Beulich



On 07/10/2019, 13:29, "Andrew Cooper" <Andrew.Cooper3@citrix.com> wrote:

    On 07/10/2019 13:01, Lars Kurth wrote:
    >
    > On 03/10/2019, 21:56, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:
    >
    >     Creative Commons is a more common license than GPL for documentation purposes.
    >     Switch to using CC-BY-4.0 to explicitly permit re-purposing and remixing of
    >     the content.
    >     
    >     Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
    >     ---
    >     CC: Lars Kurth <lars.kurth@citrix.com>
    >     CC: George Dunlap <George.Dunlap@eu.citrix.com>
    >     CC: Ian Jackson <ian.jackson@citrix.com>
    >     CC: Jan Beulich <JBeulich@suse.com>
    >     CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
    >     CC: Stefano Stabellini <sstabellini@kernel.org>
    >     CC: Tim Deegan <tim@xen.org>
    >     CC: Wei Liu <wl@xen.org>
    >     CC: Julien Grall <julien@xen.org>
    >     CC: Rich Persaud <persaur@gmail.com>
    >     CC: Juergen Gross <jgross@suse.com>
    >     ---
    >      COPYING                                 |  3 +++
    >      docs/README.source                      | 32 ++++++++++++++++++++++++++++++++
    >      docs/admin-guide/index.rst              |  2 ++
    >      docs/admin-guide/microcode-loading.rst  |  2 ++
    >      docs/conf.py                            |  1 +
    >      docs/guest-guide/index.rst              |  2 ++
    >      docs/guest-guide/x86/hypercall-abi.rst  |  2 ++
    >      docs/guest-guide/x86/index.rst          |  2 ++
    >      docs/hypervisor-guide/code-coverage.rst |  2 ++
    >      docs/hypervisor-guide/index.rst         |  2 ++
    >      docs/index.rst                          |  2 ++
    >      11 files changed, 52 insertions(+)
    >      create mode 100644 docs/README.source
    >     
    >     diff --git a/COPYING b/COPYING
    >     index 310fd52c27..80fac091d3 100644
    >     --- a/COPYING
    >     +++ b/COPYING
    >     @@ -47,6 +47,9 @@ various drivers, support functions and header files within Xen-aware
    >      Linux source trees. In all such cases, license terms are stated at the
    >      top of the file or in a COPYING file in the same directory.
    >      
    >     +Sphinx documentation is licensed under CC-BY 4.0.  See
    >     +docs/README.source for more specific information.
    >     +
    >      In some cases, compatible 3rd party code has been imported into the
    >      Xen tree, retaining the original license, such as
    >        - AES-128 3.0
    >     diff --git a/docs/README.source b/docs/README.source
    >     new file mode 100644
    >     index 0000000000..f20fa92c28
    >     --- /dev/null
    >     +++ b/docs/README.source
    >     @@ -0,0 +1,32 @@
    >     +Sphinx documentation:
    >     +
    >     +All source rendered by Sphinx is licensed under CC-BY-4.0.
    >
    > Sorry for opening this can of worms. 
    >
    > Although I had seen the discussion between Rich and you about this, I had 
    > not actually done any groundwork on the licensing. 
    >
    > So, we have to look at two things:
    >
    > * Compatibility:
    >    See https://creativecommons.org/2015/10/08/cc-by-sa-4-0-now-one-way-compatible-with-gplv3/ 
    >    This makes CC-BY-4.0 inbound compatible with GPLv3
    >    It's not clear to me whether GPLv2 is compatible with CC-BY 4.0: lack of publicly
    >    available information implies this is not the case 
    >
    > * Output License
    >    But even if it is, the produced sphinx output would be GPLv2, not CC-BY 4.0
    >    This would even be true if none of the older GPLv2 docs portions were included, as
    >    the API docs generated from source are GPLv2
    >
    > As such the statement "All source rendered by Sphinx is licensed under
    > CC-BY-4.0" is wrong.
    
    At the moment, I (and therefore Citrix in practice) holds the copyright
    on all rst content which exists in Xen.
    
    The point of this patch is to get it licensed sensibly (and in
    particular, not falling back to the GPL default) before 4.13 goes out of
    the door.
    
    The result therefore is uniformly CC-BY-4.0, with no GPL in sight.

Having looked at this again, you are correct. I was assuming that 
https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/x86/hypercall-abi.html
was linking to the doxygen generated ABI docs, which it seems not to do.
    
    > Although it is probably correct to say "All CC-BY 4.0 source rendered by
    > Sphinx is licensed under CC-BY-4.0", because Sphinx retains the source file
    > to html mapping and linkage in docs generation works differently
    > to linkage in code. 
    >
    > I am wondering whether anyone else has come across this. This question in
    > particular goes back to Rich who made a very strong case for CC-BY-4.0 based
    > documentation. I don't think we would have an issue if the entire sphinx doc-set
    > is GPLv2 if most content is licensed under CC-BY-4.0, except that such an
    > approach would make re-using the entire sphinx generated docset messy.
    >
    > We probably also want to maintain the capability to copy text from some
    > documentation freely into the source tree and vice versa, if needed. This is
    > particularly true for content in Technical Debt, user content (may end up in
    > man pages), etc.
    
    I disagree, insofar as blindly copying notes out of source code and into
    the sphinx documentation is liable to get a -1 from me.

I was actually looking at this the other way round: let's say we have
good information in a section about Technical Debt in the sphinx docs
and some of it may end up in the tree, then we could have an issue.
The dual license approach would enable us to do this.
    
    The types of style/language which are appropriate for these two cases
    are a disjoint set.

I think we are agreed. 
    
    >
    > Maybe the right approach would be to dually license the documentation
    > files using both GPLv2 and CC-BY 4.0 and quantifying this in the COPYING
    > file of the docs directory (starting from a specific date). We could eventually
    > re-license all the other stuff over time, which should be relatively straightforward
    > and/or exclude specific problematic directories.
    
    I don't see how that helps for blindly copying things in.  They'd still
    be GPL2 strictly, and would need explicit relicensing to become dual
    licensed, at which point, why not go straight for consistency?

It doesn't help with this case: it does help with copying information from the
sphinx docs to the tree 

I think we need to have a discussion/proposal after this series is committed covering
topics such as
* File formats for say man pages (or other docs)
* What are the acceptable use-cases of copying information from existing documentation to sphinx docs
   - Copying information from the Wiki to sphinx: the Wiki is CC-BY 3.0 and forward compatible, while (c) are retained - we should be OK
   - Would we want to link to man pages from the sphinx docset
   - Would we want to link to ABI docs generated from the sphinx docset in future
* What type of content may end up in a GPLv2 code with the sphinx docs as source

What this says to me is that we probably need to put a bit more thought into this

However, you have convinced me, that this does not have to happen before 4.13 
as long as we make sure we can still change licensing if we need to after 4.13
is released.    

I think we need to have a wider discussion about this set of topics
as use-case, format and license are potentially entangled for what we may
want to do in future. I can run with this: I already promised to cover the man 
page format portion and can cast the net a bit more widely

    >
    > Things like standardising say man pages to rst, would potentially also
    > create complexities with this patch, because of 
    >     +This includes:
    >     +  * All ReStructured Text files:          docs/*/*.rst
    >
    > I don't want this to become a long-winded conversation during the 4.13 freeze.
    > Please keep this in mind when responding.
    >
    > It may mean though, that we can't resolve this before 4.13 is released
    
    Getting this fixed is a blocker for 4.13.
    
I think you have shown that my concern was not valid for this particular
patch at this time. 

Note that my aim is NOT to blindly copy notes out of source code and into
the sphinx documentation. But I would like to end up in a position where
contributors to docs do not have to think about licensing nor have to deal
with a plethora of different file formats. I am however worried that we
end up with something fragile and complex, creating barriers that are
unnecessary unless we give this some more thought.

Regards
Lars




    

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Xen-devel] [PATCH 2/4] docs/sphinx: Indent cleanup
  2019-10-03 20:56 ` [Xen-devel] [PATCH 2/4] docs/sphinx: Indent cleanup Andrew Cooper
@ 2019-10-08 12:09   ` Lars Kurth
  0 siblings, 0 replies; 10+ messages in thread
From: Lars Kurth @ 2019-10-08 12:09 UTC (permalink / raw)
  To: Andrew Cooper, Xen-devel
  Cc: Juergen Gross, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, Tim \(Xen.org\),
	George Dunlap, Jan Beulich, Ian Jackson



On 03/10/2019, 21:56, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:

    Sphinx, its linters, and RST modes in common editors, expect 3 spaces of
    indentation.  Some bits already conform to this expectation.  Update the
    rest to match.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Lars Kurth <lars.kurth@citrix.com>
    

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0
  2019-10-03 20:56 ` [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0 Andrew Cooper
  2019-10-07 12:01   ` Lars Kurth
@ 2019-10-08 13:09   ` Lars Kurth
  1 sibling, 0 replies; 10+ messages in thread
From: Lars Kurth @ 2019-10-08 13:09 UTC (permalink / raw)
  To: Andrew Cooper, Xen-devel
  Cc: Juergen Gross, Stefano Stabellini, Julien Grall, Wei Liu,
	Konrad Rzeszutek Wilk, Tim \(Xen.org\),
	George Dunlap, Rich Persaud, Jan Beulich, Ian Jackson



On 03/10/2019, 21:56, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:

    Creative Commons is a more common license than GPL for documentation purposes.
    Switch to using CC-BY-4.0 to explicitly permit re-purposing and remixing of
    the content.
    
    Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Lars Kurth <lars.kurth@citrix.com>

Also, I need to clarify that one of the statements I made yesterday is wrong
CC-BY-4 is compatible with all versions with of the GPL, see
https://www.gnu.org/licenses/license-list.en.html#OtherLicenses

I mixed up CC-BY-4 with CC-BY-SA-4
 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, back to index

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-03 20:56 [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Andrew Cooper
2019-10-03 20:56 ` [Xen-devel] [PATCH 1/4] docs/sphinx: License content with CC-BY-4.0 Andrew Cooper
2019-10-07 12:01   ` Lars Kurth
2019-10-07 12:29     ` Andrew Cooper
2019-10-07 13:09       ` Lars Kurth
2019-10-08 13:09   ` Lars Kurth
2019-10-03 20:56 ` [Xen-devel] [PATCH 2/4] docs/sphinx: Indent cleanup Andrew Cooper
2019-10-08 12:09   ` Lars Kurth
2019-10-03 20:56 ` [Xen-devel] [PATCH 4/4] docs/sphinx: Technical Debt Andrew Cooper
2019-10-04  4:53 ` [Xen-devel] [PATCH for-4.13 0/4] docs/sphinx Jürgen Groß

Xen-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/xen-devel/0 xen-devel/git/0.git
	git clone --mirror https://lore.kernel.org/xen-devel/1 xen-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 xen-devel xen-devel/ https://lore.kernel.org/xen-devel \
		xen-devel@lists.xenproject.org xen-devel@lists.xen.org xen-devel@archiver.kernel.org
	public-inbox-index xen-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.xenproject.lists.xen-devel


AGPL code for this site: git clone https://public-inbox.org/ public-inbox