Re: [Xen-devel] [RFC] Documentation formats, licenses and file system structure

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 10/10/2019 13:34, Lars Kurth wrote:
> Hi all,
>
> following on from a discussion on IRC and on various other places, I think we need to try and rationalize how we handle documentation.
>
> What we have now and what we may get in future
> * http://xenbits.xen.org/docs/unstable/ (GPL-2)
> * http://xenbits.xen.org/docs/sphinx-unstable-staging/ (CC-BY-4)
> * Additional API documentation (with a view to enabling safety) 
> * Any future documentation related to safety (requirements, designs, test cases, tracability)
>
> Desired licenses
> * There is a desire to keep http://xenbits.xen.org/docs/sphinx-unstable-staging/ CC-BY-4 only
> * There is a desire to publish future documentation related to safety as CC-BY-4

Its probably worth nothing that the
http://xenbits.xen.org/docs/sphinx-unstable-staging/ URL is only
transitional.

When Sphinx is more ready for primetime, I was thinking of using
http://xenbits.xen.org/docs/xen/, and using the Sphinx support for
multiple versions, which would end up becoming docs/xen/{4.13,...,latest}/

> Existing formats and licenses
> * Hypercall ABI Documentation generated from Xen public headers
> Format: kerndoc
> License: typically BSD-3-Clause (documentation is generated from public headers)

Its homegrown markup&perl, superimposed on what used to be doxygen in
the past.

I wasn't planning on reusing any of the markup, and wasn't expecting to
use much of the text either.  I'm still considering the option of
defining that xen/public/* isn't the canonical description of the ABI,
because C is the wrong tool for the job.

Its fine to provide a C set of headers implementing an ABI, but there is
a very deliberate reason why the canonical migration v2 spec is in a
text document.

> * docs/designs, docs/features, docs/specs
> Formats: primarily pandoc, with some files md
> License: GPL-2
> * docs/processs - covers internal processes
> Formats: txt, with some pandoc
> License: GPL-2
> * docs/figs
> Formats: misc
> License: GPL-2
> * docs/misc
> Formats: txt, with some large number of pandoc, some other docs
> License: GPL-2
> * docs/man
> Formats: pod
> License: GPL-2
> * Sphinx docs: docs, docs/guest-guide, docs/hypervisor-guide
> Formats: rst
> License: CC-BY-4

This is the intention, but hasn't taken effect while my series is still
pending.  For now, strictly speaking it is still GPL-2.

>
> * Wiki: 
> Formats: mediawiki markdown
> License: CC-BY-SA-3 which has an automatic update to CC-BY-SA-4
> (c) of Wiki contributions are kept by the authors
>
> This means that the 3 most common file formats in use are
> * pod
> * pandoc (with some md) - these are essentially identical
> * txt for legacy and old stuff
> * rst
>
> License compatibility
> * GPL-2 and CC-BY-4 are compatible, but mixing means that the complete docset is GPL-2
> * GPL-2 and BSD-3-Clause are compatible, but mixing means that the complete docset is GPL-2
> * BSD-3-Clause and CC-BY-4 I am not 100% sure, but should not be an issue
> * CC-BY-SA-4 is only one way compatible with GPLv3 (affecting content on the wiki)
>
> The first question is whether we should convert pod to rst
> * https://metacpan.org/pod/pod2rst provides a conversion tool
> * man pages can be generated by rst2man
> Thus, technically this should be easy and should make contributions to docs/man easier
> If we do this, we should add a CONTRIBUTING file, clarifying the license in this directory

One thing I have done is put SPDX tags on every *.rst file.  What I
haven't found is a nice way to insert one into the *.drawio.svg files,
but I should probably finish off some of my experimentation TODOs.

An easy way out is to just say "look at the SPDX tag", but then we end
up with a docset which is a mess of licenses, still can't be easily
built upon.

> There are a set of related questions on what we would eventually merge into the sphinx
> docset. I believe there is agreement that most of what is in docs today is not really
> suitable, however there are a few possible exceptions
> * man pages - with a variety of different contributors from different orgs. Changing license would be hard

But certainly not impossible.

> * API docs generated from PUBLIC headers - changing license would be impossible, but would be BSD-3-Clause

The code, yes, but I'm expecting that to be orthogonal in the long run.

> * Some wiki content (e.g. https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches and friends) 
>    More than 95% of changes were from Citrix staff, so we could convert to CC-BY-4
>    Most non-Citrix changes are one-line changes and could be covered by fair use
> * Possibly stuff such as https://xenbits.xen.org/docs/unstable/support-matrix.html (which is currently GPL-2,
>    but we could relicense to say GPL-2 and CC-BY-4 if we had to)
> The implication is that the sphinx docs would not be fully CC-BY-4, but the bulk of the pages would be

Would be what?

~Andrew

>
> * Would we ever include API docs generated from GPLv2 code? E.g. for safety use-cases?
> @Stefano, @Artem: I guess this one is for you. 
> I suppose if we would have a similar issue for a safety manual
> I am also assuming we would want to use sphinx docs and rst to generate a future safety manual
>
> Other pages in docs that may be useful for the sphinx docs should essentially be re-written, 
> so we would be fine from a licensing perspective. That means that over time, we could get rid of 
> pandoc and text files in docs/misc, docs/designs, docs/features, docs/specs which
> have not really built a lot of traction.
>
> Related to this is the general question, whether we would ever copy code from source to docs
> and vice versa and to which degree. This is an unknown to me: I think in practice we have not seen
> much or even any of this in the past.
>
> On licensing, we should try and make the docs directory clean, e.g.
> * We should set the default to CC-BY-4 (e.g. through a contributing file in docs)
> * And specifically use GPL-2 for directories such as docs/misc, docs/man, ...
>
> In any case, this seems all a little bit of a mess at the moment and I think we need to
> agree on a foundation to get us to a better state. This mail is a start and intends to gather
> input and will eventually lead to a more concrete proposal.
>
> If I have missed anything, feel free to add
>
> Best Regards
> Lars

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel