[Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Lars Kurth]

Hi all,

since Andy created https://xenbits.xen.org/docs/sphinx-unstable/ and https://xenbits.xen.org/docs/sphinx-unstable-staging/ (sources in docs/hypervisor-guide, docs/guest-guide, docs/admin-guide) I was wondering whether it would not make sense to migrate some key documents in the wiki (and possible some docs elsewhere in the tree) into the new structure and refresh and update the documentation in the process. I am volunteering to do some of the leg-work.

I started looking into what some other projects do and the following seem to be sensible and lightweight examples of Information Architecture to structure the content:
* https://docs.openstack.org/infra/manual/developers.html
* https://github.com/kubernetes/community/blob/master/contributors/devel/README.md

Both are fairly lightweight and seem to be sensible guides. But before we get to that level of detail, I thought it makes sense to look at what we have and candidates for moving/improving/creation. However, it is clear that we need two broad categories under docs/hypervisor-guide (Hypervisor developer documentation): note that I don’t much care about the actual labels
* The process of developing and contributing code
* Setting up your dev environment, coding, and debugging

Documentation which may be worth moving or creating

Text files in tree (which are close to RST file format)
* Xen.git:CONTRIBUTING – it seems to me that this is a candidate for moving (with a note in the original file which outlines where to look for the source/rendered output) – Even committers occasionally don’t seem to be aware of some of the licensing related common practices we have. Giving some of that content a more prominent place in a new more user friendly and modern looking docset seems sensible  
* Xen.git:INSTALL may be a good candidate which could live in the admin guide and/or developer guide
* Xen.git:CODING_STYLE and Xen.git:tools/libxl/CODING_STYLE - Note that in the community call discussion Jan raised the point that we tend to not document precedents for many things which are coding style related. Maybe we can get a bit better 
* Xen.git:MAINTAINERS should stay as it is, but should probably be referred to appropriately in the docset

Content from the wiki (the idea would be to redirect those pages in the wiki to the new locations)
* https://wiki.xenproject.org/wiki/Asking_Developer_Questions - could do with a refresh. Possibly there is also a tie in with https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html
* https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source - there seems to be some overlap with Xen.git/INSTALL which may be worth cleaning up
* https://wiki.xenproject.org/wiki/Xen_Project_Repositories 
* https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches - This has become a bit unwieldy and could do with some clean-up
   * Slight overlap with Xen.git:CONTRIBUTING (around DCO and Sign-off) 
   * Making good patches probably needs some work and maybe should be broken out. It should include good examples and setting expectations of what is deemed good and bad around areas where we have higher standards than other projects (such as commit messages, explaining rationale for a change, technical debt, ...). It should probably also cover things such as Design Documentation and where to find templates, highlight existing documentation and where to find it/update it, the same with text, SUPPORT.md (aka add a new feature), etc.
   * Setting up git send-email: should probably go into a section related to the development environment set-up and just be referred to
   * Using git send-email alone - we should nuke this section and focus on the next section 
   * Simplest workflow: Git format-patch, add_maintainers.pl/get_maintainer.pl and git send-email - I would build the bulk of the doc around this, but maybe move the add_maintainers.pl/get_maintainer.pl  file into a separate document under a Xen specific development tools section and just refer to it
    * Sending patches manually - we should nuke this section and focus on the next section
    * I would move the bulk of this into a Contribution Workflow section, which gives an overall workflow and just highlight the reroll count. We should define the tags and conventions such as RFC somewhere in an introductory section of this document
   * All the QEMU, Linux, ... stuff can either stay on the Wiki or could be broken out
* https://wiki.xenproject.org/wiki/Reporting_Bugs_against_Xen_Project - strip all the XAPI stuff. NBot sure whether https://wiki.xenproject.org/wiki/XenParavirtOpsHelp is still applicable: nuke if not

Key content that is missing
* An overview for testing, which should include
   - OSSTEST
   - XTF
   - The GitLab CI
* Outcome from the Minimum Standards and Best Practices discussion at https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html depending on outcome. I was thinking about a Community Standards section, which would point to a Code of Conduct/Minimum and Best Practices (maybe written by role: aka contributor, reviewer and maybe committer)
* Release Process Related documentation (from a contributor's perspective)
* Maybe a description of the source tree
* Some of the information in SUPPORT.md in a feature lifecycle document
* Maybe some of the things people need to know about KCONFIG

Let me know what you think. I will start with some of the easier bits next week if I can find some time, unless there are major objections.

Best Regards
Lars  

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 6259 bytes --]

On Mon, 24 Jun 2019, Lars Kurth wrote:
> Hi all,
> 
> since Andy created https://xenbits.xen.org/docs/sphinx-unstable/ and https://xenbits.xen.org/docs/sphinx-unstable-staging/ (sources in docs/hypervisor-guide, docs/guest-guide, docs/admin-guide) I was wondering whether it would not make sense to migrate some key documents in the wiki (and possible some docs elsewhere in the tree) into the new structure and refresh and update the documentation in the process. I am volunteering to do some of the leg-work.
> 
> I started looking into what some other projects do and the following seem to be sensible and lightweight examples of Information Architecture to structure the content:
> * https://docs.openstack.org/infra/manual/developers.html
> * https://github.com/kubernetes/community/blob/master/contributors/devel/README.md
> 
> Both are fairly lightweight and seem to be sensible guides. But before we get to that level of detail, I thought it makes sense to look at what we have and candidates for moving/improving/creation. However, it is clear that we need two broad categories under docs/hypervisor-guide (Hypervisor developer documentation): note that I don’t much care about the actual labels
> * The process of developing and contributing code
> * Setting up your dev environment, coding, and debugging
> 
> Documentation which may be worth moving or creating
> 
> Text files in tree (which are close to RST file format)
> * Xen.git:CONTRIBUTING – it seems to me that this is a candidate for moving (with a note in the original file which outlines where to look for the source/rendered output) – Even committers occasionally don’t seem to be aware of some of the licensing related common practices we have. Giving some of that content a more prominent place in a new more user friendly and modern looking docset seems sensible  
> * Xen.git:INSTALL may be a good candidate which could live in the admin guide and/or developer guide
> * Xen.git:CODING_STYLE and Xen.git:tools/libxl/CODING_STYLE - Note that in the community call discussion Jan raised the point that we tend to not document precedents for many things which are coding style related. Maybe we can get a bit better 
> * Xen.git:MAINTAINERS should stay as it is, but should probably be referred to appropriately in the docset
> 
> Content from the wiki (the idea would be to redirect those pages in the wiki to the new locations)
> * https://wiki.xenproject.org/wiki/Asking_Developer_Questions - could do with a refresh. Possibly there is also a tie in with https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html
> * https://wiki.xenproject.org/wiki/Compiling_Xen_From_Source - there seems to be some overlap with Xen.git/INSTALL which may be worth cleaning up
> * https://wiki.xenproject.org/wiki/Xen_Project_Repositories 
> * https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches - This has become a bit unwieldy and could do with some clean-up
>    * Slight overlap with Xen.git:CONTRIBUTING (around DCO and Sign-off) 
>    * Making good patches probably needs some work and maybe should be broken out. It should include good examples and setting expectations of what is deemed good and bad around areas where we have higher standards than other projects (such as commit messages, explaining rationale for a change, technical debt, ...). It should probably also cover things such as Design Documentation and where to find templates, highlight existing documentation and where to find it/update it, the same with text, SUPPORT.md (aka add a new feature), etc.
>    * Setting up git send-email: should probably go into a section related to the development environment set-up and just be referred to
>    * Using git send-email alone - we should nuke this section and focus on the next section 
>    * Simplest workflow: Git format-patch, add_maintainers.pl/get_maintainer.pl and git send-email - I would build the bulk of the doc around this, but maybe move the add_maintainers.pl/get_maintainer.pl  file into a separate document under a Xen specific development tools section and just refer to it
>     * Sending patches manually - we should nuke this section and focus on the next section
>     * I would move the bulk of this into a Contribution Workflow section, which gives an overall workflow and just highlight the reroll count. We should define the tags and conventions such as RFC somewhere in an introductory section of this document
>    * All the QEMU, Linux, ... stuff can either stay on the Wiki or could be broken out
> * https://wiki.xenproject.org/wiki/Reporting_Bugs_against_Xen_Project - strip all the XAPI stuff. NBot sure whether https://wiki.xenproject.org/wiki/XenParavirtOpsHelp is still applicable: nuke if not
> 
> Key content that is missing
> * An overview for testing, which should include
>    - OSSTEST
>    - XTF
>    - The GitLab CI
> * Outcome from the Minimum Standards and Best Practices discussion at https://lists.xenproject.org/archives/html/xen-devel/2019-06/msg01518.html depending on outcome. I was thinking about a Community Standards section, which would point to a Code of Conduct/Minimum and Best Practices (maybe written by role: aka contributor, reviewer and maybe committer)
> * Release Process Related documentation (from a contributor's perspective)
> * Maybe a description of the source tree
> * Some of the information in SUPPORT.md in a feature lifecycle document
> * Maybe some of the things people need to know about KCONFIG
> 
> Let me know what you think. I will start with some of the easier bits next week if I can find some time, unless there are major objections.

I think we all agree by now that maintaining up-to-date docs on the wiki
and keeping them in sync with code changes is hard. I see moving things
from the wiki to xen.git as a great improvement. We have a few Xen on
ARM docs that are worth importing from the wiki:

https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions

And all the board specific docs linked from it, such as:

https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels
https://wiki.xenproject.org/wiki/HiKey960

etc.

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Julien Grall]

Hi,

On 24/06/2019 19:03, Stefano Stabellini wrote:
> On Mon, 24 Jun 2019, Lars Kurth wrote:
> I think we all agree by now that maintaining up-to-date docs on the wiki
> and keeping them in sync with code changes is hard. I see moving things
> from the wiki to xen.git as a great improvement. We have a few Xen on
> ARM docs that are worth importing from the wiki:
> 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions

I agree for this but ...

> 
> And all the board specific docs linked from it, such as:
> 
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels
> https://wiki.xenproject.org/wiki/HiKey960

... I think it is a pretty bad idea to import board specific docs. There 
are a lot of way to build components for a given board and I am worry of 
the overheard for the maintainers to look/maintain the documentation. It 
also brings the question of the acceptance/removal of
a board documentation.

Instead we should provide generic guidance/troubleshoot to the user. 
Anything board specific could be maintain on the wiki by someone caring 
about the board without having us to gate it.

We could try to revive a discussion we had a couple of years ago where 
someone is responsible for a given board to test and document it. Today 
it is informal, but I have been pushing for it recently for new boards.

Cheers,

-- 
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Stefano Stabellini]

On Mon, 24 Jun 2019, Julien Grall wrote:
> Hi,
> 
> On 24/06/2019 19:03, Stefano Stabellini wrote:
> > On Mon, 24 Jun 2019, Lars Kurth wrote:
> > I think we all agree by now that maintaining up-to-date docs on the wiki
> > and keeping them in sync with code changes is hard. I see moving things
> > from the wiki to xen.git as a great improvement. We have a few Xen on
> > ARM docs that are worth importing from the wiki:
> > 
> > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions
> 
> I agree for this but ...
> 
> > 
> > And all the board specific docs linked from it, such as:
> > 
> > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
> > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels
> > https://wiki.xenproject.org/wiki/HiKey960
> 
> ... I think it is a pretty bad idea to import board specific docs. There 
> are a lot of way to build components for a given board and I am worry of 
> the overheard for the maintainers to look/maintain the documentation. It 
> also brings the question of the acceptance/removal of
> a board documentation.

That problem can be solved by specifying an appropriate maintenance
model for those documents.


> Instead we should provide generic guidance/troubleshoot to the user. 
> Anything board specific could be maintain on the wiki by someone caring 
> about the board without having us to gate it.

If we move the docs to xen.git it doesn't immediately imply that the
REST maintainers need to "gate" them. We could make the existing
curators of those pages the maintainers for those files, for example. We
can come up with mode ideas. We could even leave them unmaintained.

The point here is that we can be flexible and creative about the way to
maintain the docs on xen.git. But as a technology is certainly better
than the wiki: we don't have to keep them all up-to-date with the code,
but at least this way we have a chance (if we want to). If we leave them
on the wiki, there is no chance.


> We could try to revive a discussion we had a couple of years ago where 
> someone is responsible for a given board to test and document it. Today 
> it is informal, but I have been pushing for it recently for new boards.

Yes, this is a good idea and fit nicely with the maintenance model I
was suggesting.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Julien Grall]

Hi Stefano,

On 6/24/19 9:23 PM, Stefano Stabellini wrote:
> On Mon, 24 Jun 2019, Julien Grall wrote:
>> Hi,
>>
>> On 24/06/2019 19:03, Stefano Stabellini wrote:
>>> On Mon, 24 Jun 2019, Lars Kurth wrote:
>>> I think we all agree by now that maintaining up-to-date docs on the wiki
>>> and keeping them in sync with code changes is hard. I see moving things
>>> from the wiki to xen.git as a great improvement. We have a few Xen on
>>> ARM docs that are worth importing from the wiki:
>>>
>>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions
>>
>> I agree for this but ...
>>
>>>
>>> And all the board specific docs linked from it, such as:
>>>
>>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
>>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels
>>> https://wiki.xenproject.org/wiki/HiKey960
>>
>> ... I think it is a pretty bad idea to import board specific docs. There
>> are a lot of way to build components for a given board and I am worry of
>> the overheard for the maintainers to look/maintain the documentation. It
>> also brings the question of the acceptance/removal of
>> a board documentation.
> 
> That problem can be solved by specifying an appropriate maintenance
> model for those documents.
> 
> 
>> Instead we should provide generic guidance/troubleshoot to the user.
>> Anything board specific could be maintain on the wiki by someone caring
>> about the board without having us to gate it.
> 
> If we move the docs to xen.git it doesn't immediately imply that the
> REST maintainers need to "gate" them. We could make the existing
> curators of those pages the maintainers for those files, for example. We
> can come up with mode ideas. We could even leave them unmaintained.

I don't think I want to add a random person as a maintainer in xen.git. 
So at best we would need a new role.

> 
> The point here is that we can be flexible and creative about the way to
> maintain the docs on xen.git. But as a technology is certainly better
> than the wiki: we don't have to keep them all up-to-date with the code,
> but at least this way we have a chance (if we want to). If we leave them
> on the wiki, there is no chance.

I can't see how xen.git is going to be better if "we don't have to keep 
them all up-to-date".

But my point here is most of the board should be trivial. The most of 
the non-trivial setup require non-upstream patch. While I am happy to 
see that on the wiki, I think xen.git should not promote such 
configuration at all. We are working upstream, not with 
unknown/untrusted stack.

For some working fully upstream, I don't think xen.git should promote 
any distros/versions of the kernel. However, this is ok on the wiki.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Stefano Stabellini]

On Mon, 24 Jun 2019, Julien Grall wrote:
> Hi Stefano,
> 
> On 6/24/19 9:23 PM, Stefano Stabellini wrote:
> > On Mon, 24 Jun 2019, Julien Grall wrote:
> > > Hi,
> > > 
> > > On 24/06/2019 19:03, Stefano Stabellini wrote:
> > > > On Mon, 24 Jun 2019, Lars Kurth wrote:
> > > > I think we all agree by now that maintaining up-to-date docs on the wiki
> > > > and keeping them in sync with code changes is hard. I see moving things
> > > > from the wiki to xen.git as a great improvement. We have a few Xen on
> > > > ARM docs that are worth importing from the wiki:
> > > > 
> > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions
> > > 
> > > I agree for this but ...
> > > 
> > > > 
> > > > And all the board specific docs linked from it, such as:
> > > > 
> > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
> > > > https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels
> > > > https://wiki.xenproject.org/wiki/HiKey960
> > > 
> > > ... I think it is a pretty bad idea to import board specific docs. There
> > > are a lot of way to build components for a given board and I am worry of
> > > the overheard for the maintainers to look/maintain the documentation. It
> > > also brings the question of the acceptance/removal of
> > > a board documentation.
> > 
> > That problem can be solved by specifying an appropriate maintenance
> > model for those documents.
> > 
> > 
> > > Instead we should provide generic guidance/troubleshoot to the user.
> > > Anything board specific could be maintain on the wiki by someone caring
> > > about the board without having us to gate it.
> > 
> > If we move the docs to xen.git it doesn't immediately imply that the
> > REST maintainers need to "gate" them. We could make the existing
> > curators of those pages the maintainers for those files, for example. We
> > can come up with mode ideas. We could even leave them unmaintained.
> 
> I don't think I want to add a random person as a maintainer in xen.git. So at
> best we would need a new role.

This is a good point, taking into account the current governance model.
We could use R: for that?


> > The point here is that we can be flexible and creative about the way to
> > maintain the docs on xen.git. But as a technology is certainly better
> > than the wiki: we don't have to keep them all up-to-date with the code,
> > but at least this way we have a chance (if we want to). If we leave them
> > on the wiki, there is no chance.
> 
> I can't see how xen.git is going to be better if "we don't have to keep them
> all up-to-date".

That's because a contributor could add a patch at the end of a series to
update one of the docs, even if the doc in question comes with no
promises of being up-to-date.


> But my point here is most of the board should be trivial. The most of the
> non-trivial setup require non-upstream patch. While I am happy to see that on
> the wiki, I think xen.git should not promote such configuration at all. We are
> working upstream, not with unknown/untrusted stack.
> 
> For some working fully upstream, I don't think xen.git should promote any
> distros/versions of the kernel. However, this is ok on the wiki.

I would like to see the wiki disappear completely in the long term. As
we are moving more content to xen.git, it is not a good idea to have two
places where we keep information, for similar reasons why you suggested
to use in-code comments instead of docs to document interfaces. It
just takes more efforts to maintain information in two places and they
tend to get out of sync with each others.

If we make the wiki go away (I hope so), we'll need a place to store the
Arm board-specific documents, and other tutorials.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Julien Grall]

Hi Stefano,

On 25/06/2019 01:02, Stefano Stabellini wrote:
> On Mon, 24 Jun 2019, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 6/24/19 9:23 PM, Stefano Stabellini wrote:
>>> On Mon, 24 Jun 2019, Julien Grall wrote:
>>>> Hi,
>>>>
>>>> On 24/06/2019 19:03, Stefano Stabellini wrote:
>>>>> On Mon, 24 Jun 2019, Lars Kurth wrote:
>>>>> I think we all agree by now that maintaining up-to-date docs on the wiki
>>>>> and keeping them in sync with code changes is hard. I see moving things
>>>>> from the wiki to xen.git as a great improvement. We have a few Xen on
>>>>> ARM docs that are worth importing from the wiki:
>>>>>
>>>>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions
>>>>
>>>> I agree for this but ...
>>>>
>>>>>
>>>>> And all the board specific docs linked from it, such as:
>>>>>
>>>>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64
>>>>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/FastModels
>>>>> https://wiki.xenproject.org/wiki/HiKey960
>>>>
>>>> ... I think it is a pretty bad idea to import board specific docs. There
>>>> are a lot of way to build components for a given board and I am worry of
>>>> the overheard for the maintainers to look/maintain the documentation. It
>>>> also brings the question of the acceptance/removal of
>>>> a board documentation.
>>>
>>> That problem can be solved by specifying an appropriate maintenance
>>> model for those documents.
>>>
>>>
>>>> Instead we should provide generic guidance/troubleshoot to the user.
>>>> Anything board specific could be maintain on the wiki by someone caring
>>>> about the board without having us to gate it.
>>>
>>> If we move the docs to xen.git it doesn't immediately imply that the
>>> REST maintainers need to "gate" them. We could make the existing
>>> curators of those pages the maintainers for those files, for example. We
>>> can come up with mode ideas. We could even leave them unmaintained.
>>
>> I don't think I want to add a random person as a maintainer in xen.git. So at
>> best we would need a new role.
> 
> This is a good point, taking into account the current governance model.
> We could use R: for that?

I am not entirely sure, this still mean "THE REST" will be in charge of it. 
Anyway, this is not the biggest problem here.

> 
> 
>>> The point here is that we can be flexible and creative about the way to
>>> maintain the docs on xen.git. But as a technology is certainly better
>>> than the wiki: we don't have to keep them all up-to-date with the code,
>>> but at least this way we have a chance (if we want to). If we leave them
>>> on the wiki, there is no chance.
>>
>> I can't see how xen.git is going to be better if "we don't have to keep them
>> all up-to-date".
> 
> That's because a contributor could add a patch at the end of a series to
> update one of the docs, even if the doc in question comes with no
> promises of being up-to-date.

I think this is going the wrong direction. The goal of using xen.git is to try 
to keep the documentation up-to-date.

> 
> 
>> But my point here is most of the board should be trivial. The most of the
>> non-trivial setup require non-upstream patch. While I am happy to see that on
>> the wiki, I think xen.git should not promote such configuration at all. We are
>> working upstream, not with unknown/untrusted stack.
>>
>> For some working fully upstream, I don't think xen.git should promote any
>> distros/versions of the kernel. However, this is ok on the wiki.
> 
> I would like to see the wiki disappear completely in the long term. As
> we are moving more content to xen.git, it is not a good idea to have two
> places where we keep information, for similar reasons why you suggested
> to use in-code comments instead of docs to document interfaces. It
> just takes more efforts to maintain information in two places and they
> tend to get out of sync with each others.
> 
> If we make the wiki go away (I hope so), we'll need a place to store the
> Arm board-specific documents, and other tutorials.

Removing the wiki is an honorable goal, however I don't think all the wiki is 
suitable for xen.git. The Arm board-specific documents is an example.

You actually haven't addressed my concern above. If you look at the wiki, a lot 
of them ([1], [2], [3]) contains non-upstreamed work or non-upstreamable hack.

For those containing only upstream work [4], the example is focusing on one set 
of distros. In the case of QEMU, I already had some people asking whether it is 
possible to use without U-boot. Why would we promote Ubuntu and not something else?

Overall, there are so many configurations possible (kernel, u-boot, 
distributions) that it does not makes sense to keep track of that in xen.git.

Instead, I think we should write generic doc on how to boot Xen on a U-boot/UEFI 
setup and inviting the users to look into more details for his board.

Cheers,

[1] 
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-X
[2] https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Stout
[3] https://wiki.xenproject.org/wiki/HiKey
[4] 
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/qemu-system-aarch64


> 

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Lars Kurth]

On 25/06/2019, 10:03, "Julien Grall" <julien.grall@arm.com> wrote:

    >>> The point here is that we can be flexible and creative about the way to
    >>> maintain the docs on xen.git. But as a technology is certainly better
    >>> than the wiki: we don't have to keep them all up-to-date with the code,
    >>> but at least this way we have a chance (if we want to). If we leave them
    >>> on the wiki, there is no chance.
    >>
    >> I can't see how xen.git is going to be better if "we don't have to keep them
    >> all up-to-date".
    > 
    > That's because a contributor could add a patch at the end of a series to
    > update one of the docs, even if the doc in question comes with no
    > promises of being up-to-date.
    
    I think this is going the wrong direction. The goal of using xen.git is to try 
    to keep the documentation up-to-date.
    
I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format.

I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these. 

What happens today for many of these type of docs and/or processes is that:
a) We have discussion about a process / working practice on the list until we come to a conclusion
b) Then we take it and copy it to the wiki
Why not merge this into one activity

Both of you are interested in Arm docs, but this is something I will let you fight out. 
Maybe you want to chat about this some more at the summit

    >> But my point here is most of the board should be trivial. The most of the
    >> non-trivial setup require non-upstream patch. While I am happy to see that on
    >> the wiki, I think xen.git should not promote such configuration at all. We are
    >> working upstream, not with unknown/untrusted stack.
    >>
    >> For some working fully upstream, I don't think xen.git should promote any
    >> distros/versions of the kernel. However, this is ok on the wiki.
    > 
    > I would like to see the wiki disappear completely in the long term. As
    > we are moving more content to xen.git, it is not a good idea to have two
    > places where we keep information, for similar reasons why you suggested
    > to use in-code comments instead of docs to document interfaces. It
    > just takes more efforts to maintain information in two places and they
    > tend to get out of sync with each others.
    > 
    > If we make the wiki go away (I hope so), we'll need a place to store the
    > Arm board-specific documents, and other tutorials.
    
    Removing the wiki is an honorable goal, however I don't think all the wiki is 
    suitable for xen.git. The Arm board-specific documents is an example.

Removing the wiki was not my aim. The wiki is useful in some cases, but not in others.

Lars
    
    

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Andrew Cooper]

On 25/06/2019 13:15, Lars Kurth wrote:
> On 25/06/2019, 10:03, "Julien Grall" <julien.grall@arm.com> wrote:
>
>     >>> The point here is that we can be flexible and creative about the way to
>     >>> maintain the docs on xen.git. But as a technology is certainly better
>     >>> than the wiki: we don't have to keep them all up-to-date with the code,
>     >>> but at least this way we have a chance (if we want to). If we leave them
>     >>> on the wiki, there is no chance.
>     >>
>     >> I can't see how xen.git is going to be better if "we don't have to keep them
>     >> all up-to-date".
>     > 
>     > That's because a contributor could add a patch at the end of a series to
>     > update one of the docs, even if the doc in question comes with no
>     > promises of being up-to-date.
>     
>     I think this is going the wrong direction. The goal of using xen.git is to try 
>     to keep the documentation up-to-date.
>     
> I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format.

So the plan here is to get a consistent and uniform set of high quality
docs.

As fair warning, I'm intending to be fairly strict with what goes in
(quality wise), because I'm going to do my best to ensure that the
sphinx documentation doesn't devolve into the mess that wiki or the
majority of docs/ currently is.

> I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these.

The key point is for maintainers/reviewers to be aware of whether
documentation exists for the area of code being modified, and if so,
whether the submitted patch should also patch the documentation.

Reviewers tend to be fairly good at noticing patches which also need to
patch docs/misc/xen-command-line.pandoc (submitters, less so), but this
approach needs extending to the whole of the sphinx docs (which in turn
requires the docs to stay high quality so its easy for maintainers to
know what is where).

~Andrew


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Lars Kurth]

On 25/06/2019, 14:47, "Andrew Cooper" <Andrew.Cooper3@citrix.com> wrote:

    On 25/06/2019 13:15, Lars Kurth wrote:
    > On 25/06/2019, 10:03, "Julien Grall" <julien.grall@arm.com> wrote:
    >
    >     >>> The point here is that we can be flexible and creative about the way to
    >     >>> maintain the docs on xen.git. But as a technology is certainly better
    >     >>> than the wiki: we don't have to keep them all up-to-date with the code,
    >     >>> but at least this way we have a chance (if we want to). If we leave them
    >     >>> on the wiki, there is no chance.
    >     >>
    >     >> I can't see how xen.git is going to be better if "we don't have to keep them
    >     >> all up-to-date".
    >     > 
    >     > That's because a contributor could add a patch at the end of a series to
    >     > update one of the docs, even if the doc in question comes with no
    >     > promises of being up-to-date.
    >     
    >     I think this is going the wrong direction. The goal of using xen.git is to try 
    >     to keep the documentation up-to-date.
    >     
    > I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format.
    
    So the plan here is to get a consistent and uniform set of high quality
    docs.
    
    As fair warning, I'm intending to be fairly strict with what goes in
    (quality wise), because I'm going to do my best to ensure that the
    sphinx documentation doesn't devolve into the mess that wiki or the
    majority of docs/ currently is.

I wholeheartedly agree
    
    > I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these.
    
    The key point is for maintainers/reviewers to be aware of whether
    documentation exists for the area of code being modified, and if so,
    whether the submitted patch should also patch the documentation.

I am wondering whether this is something which could be addressed. One possibility may be to have SUPPORT.md point to documentation. But that is kind of assuming that SUPPORT.md works and is widely used. There may be better or orthogonal ways to point to relevant docs (e.g. by pointing to docs in header files and other source files). 
    
    Reviewers tend to be fairly good at noticing patches which also need to
    patch docs/misc/xen-command-line.pandoc (submitters, less so), but this
    approach needs extending to the whole of the sphinx docs (which in turn
    requires the docs to stay high quality so its easy for maintainers to
    know what is where).
    
Although this does not apply in my proposal, I think the key issue has been that reviewers and submitters of code often don't use our documentation. The wiki is seen as a separate thing and also has the disadvantage that it doesn't lend itself to supporting different versions of Xen. And most of the time, developers do not use it and neither contribute to it.

My hope was that by hosting documentation related to contribution workflow and other essential tasks close to other useful documentation this would enable change.

@Andy and others: I need to know whether you agree with my proposal and whether anyone has other suggestions.

Regards
Lars
 


 
    
    

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Andrew Cooper]

On 25/06/2019 17:34, Lars Kurth wrote:
>
> On 25/06/2019, 14:47, "Andrew Cooper" <Andrew.Cooper3@citrix.com> wrote:
>
>     On 25/06/2019 13:15, Lars Kurth wrote:
>     > On 25/06/2019, 10:03, "Julien Grall" <julien.grall@arm.com> wrote:
>     >
>     >     >>> The point here is that we can be flexible and creative about the way to
>     >     >>> maintain the docs on xen.git. But as a technology is certainly better
>     >     >>> than the wiki: we don't have to keep them all up-to-date with the code,
>     >     >>> but at least this way we have a chance (if we want to). If we leave them
>     >     >>> on the wiki, there is no chance.
>     >     >>
>     >     >> I can't see how xen.git is going to be better if "we don't have to keep them
>     >     >> all up-to-date".
>     >     > 
>     >     > That's because a contributor could add a patch at the end of a series to
>     >     > update one of the docs, even if the doc in question comes with no
>     >     > promises of being up-to-date.
>     >     
>     >     I think this is going the wrong direction. The goal of using xen.git is to try 
>     >     to keep the documentation up-to-date.
>     >     
>     > I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format.
>     
>     So the plan here is to get a consistent and uniform set of high quality
>     docs.
>     
>     As fair warning, I'm intending to be fairly strict with what goes in
>     (quality wise), because I'm going to do my best to ensure that the
>     sphinx documentation doesn't devolve into the mess that wiki or the
>     majority of docs/ currently is.
>
> I wholeheartedly agree
>     
>     > I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these.
>     
>     The key point is for maintainers/reviewers to be aware of whether
>     documentation exists for the area of code being modified, and if so,
>     whether the submitted patch should also patch the documentation.
>
> I am wondering whether this is something which could be addressed. One possibility may be to have SUPPORT.md point to documentation. But that is kind of assuming that SUPPORT.md works and is widely used. There may be better or orthogonal ways to point to relevant docs (e.g. by pointing to docs in header files and other source files).
>     
>     Reviewers tend to be fairly good at noticing patches which also need to
>     patch docs/misc/xen-command-line.pandoc (submitters, less so), but this
>     approach needs extending to the whole of the sphinx docs (which in turn
>     requires the docs to stay high quality so its easy for maintainers to
>     know what is where).
>     
> Although this does not apply in my proposal, I think the key issue has been that reviewers and submitters of code often don't use our documentation. The wiki is seen as a separate thing and also has the disadvantage that it doesn't lend itself to supporting different versions of Xen. And most of the time, developers do not use it and neither contribute to it.

It is a positive feedback cycle.  Noone uses our documentation because
its terrible, so the documentation stays in a terrible state.

As soon as the docs start to improve, the awareness will improve.

That said, there is still a responsibility for maintainers to be aware
of the documentation for their own areas, and to ensure the
documentation is maintained to the same high standard as we expect from
our code.

>
> My hope was that by hosting documentation related to contribution workflow and other essential tasks close to other useful documentation this would enable change.
>
> @Andy and others: I need to know whether you agree with my proposal and whether anyone has other suggestions.

Absolutely.  "how to submit changes" is one of the most cited wiki
pages, and having that in sphinx will cause loads of people to be aware
that we do have other docs in sphinx as well.  The fact that everything
is properly indexed is a massive benefit here.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[P S]

> On Jun 25, 2019, at 12:34, Lars Kurth <lars.kurth@citrix.com> wrote:
> 
> 
> 
> On 25/06/2019, 14:47, "Andrew Cooper" <Andrew.Cooper3@citrix.com> wrote:
> 
>>    On 25/06/2019 13:15, Lars Kurth wrote:
>> On 25/06/2019, 10:03, "Julien Grall" <julien.grall@arm.com> wrote:
>> 
>>>>> The point here is that we can be flexible and creative about the way to
>>>>> maintain the docs on xen.git. But as a technology is certainly better
>>>>> than the wiki: we don't have to keep them all up-to-date with the code,
>>>>> but at least this way we have a chance (if we want to). If we leave them
>>>>> on the wiki, there is no chance.
>>>> 
>>>> I can't see how xen.git is going to be better if "we don't have to keep them
>>>> all up-to-date".
>>> 
>>> That's because a contributor could add a patch at the end of a series to
>>> update one of the docs, even if the doc in question comes with no
>>> promises of being up-to-date.
>> 
>>    I think this is going the wrong direction. The goal of using xen.git is to try 
>>    to keep the documentation up-to-date.
>> 
>> I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format.
> 
>    So the plan here is to get a consistent and uniform set of high quality
>    docs.
> 
>    As fair warning, I'm intending to be fairly strict with what goes in
>    (quality wise), because I'm going to do my best to ensure that the
>    sphinx documentation doesn't devolve into the mess that wiki or the
>    majority of docs/ currently is.
> 
> I wholeheartedly agree
> 
>> I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these.
> 
>    The key point is for maintainers/reviewers to be aware of whether
>    documentation exists for the area of code being modified, and if so,
>    whether the submitted patch should also patch the documentation.
> 
> I am wondering whether this is something which could be addressed. One possibility may be to have SUPPORT.md point to documentation. But that is kind of assuming that SUPPORT.md works and is widely used. There may be better or orthogonal ways to point to relevant docs (e.g. by pointing to docs in header files and other source files). 
> 
>    Reviewers tend to be fairly good at noticing patches which also need to
>    patch docs/misc/xen-command-line.pandoc (submitters, less so), but this
>    approach needs extending to the whole of the sphinx docs (which in turn
>    requires the docs to stay high quality so its easy for maintainers to
>    know what is where).
> 
> Although this does not apply in my proposal, I think the key issue has been that reviewers and submitters of code often don't use our documentation. The wiki is seen as a separate thing and also has the disadvantage that it doesn't lend itself to supporting different versions of Xen. And most of the time, developers do not use it and neither contribute to it.
> 
> My hope was that by hosting documentation related to contribution workflow and other essential tasks close to other useful documentation this would enable change.
> 
> @Andy and others: I need to know whether you agree with my proposal and whether anyone has other suggestions.

If not already present in the release manager process checklist, we could specify documentation-related updates for each release, e.g. minimum text for new features, revisions to modified features, SUPPORT.md updates.

Rich
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Rich Persaud]

> On Jun 25, 2019, at 12:34, Lars Kurth <lars.kurth@citrix.com> wrote:
> 
> On 25/06/2019, 14:47, "Andrew Cooper" <Andrew.Cooper3@citrix.com> wrote:
> 
>>   On 25/06/2019 13:15, Lars Kurth wrote:
>> On 25/06/2019, 10:03, "Julien Grall" <julien.grall@arm.com> wrote:
>> 
>>>>> The point here is that we can be flexible and creative about the way to
>>>>> maintain the docs on xen.git. But as a technology is certainly better
>>>>> than the wiki: we don't have to keep them all up-to-date with the code,
>>>>> but at least this way we have a chance (if we want to). If we leave them
>>>>> on the wiki, there is no chance.
>>>> 
>>>> I can't see how xen.git is going to be better if "we don't have to keep them
>>>> all up-to-date".
>>> 
>>> That's because a contributor could add a patch at the end of a series to
>>> update one of the docs, even if the doc in question comes with no
>>> promises of being up-to-date.
>> 
>>   I think this is going the wrong direction. The goal of using xen.git is to try 
>>   to keep the documentation up-to-date.
>> 
>> I agree with Julien and this was also not my intention. The reason why I brought this up now is that the in-tree docs are pretty much a mess today and are stale in many ways. And they look TERRIBLE and are not easily searchable. However, Andy's latest set of patches provide an opportunity to consolidate some of the in-tree docs in a nicely rendered and searchable format.
> 
>   So the plan here is to get a consistent and uniform set of high quality
>   docs.
> 
>   As fair warning, I'm intending to be fairly strict with what goes in
>   (quality wise), because I'm going to do my best to ensure that the
>   sphinx documentation doesn't devolve into the mess that wiki or the
>   majority of docs/ currently is.
> 
> I wholeheartedly agree
> 
>> I have been focussing on process related and key developer related docs, because who maintains them is not actually an issue in theory. Everyone really ought to care, because everyone is impacted by these.
> 
>   The key point is for maintainers/reviewers to be aware of whether
>   documentation exists for the area of code being modified, and if so,
>   whether the submitted patch should also patch the documentation.
> 
> I am wondering whether this is something which could be addressed. One possibility may be to have SUPPORT.md point to documentation. But that is kind of assuming that SUPPORT.md works and is widely used. There may be better or orthogonal ways to point to relevant docs (e.g. by pointing to docs in header files and other source files). 
> 
>   Reviewers tend to be fairly good at noticing patches which also need to
>   patch docs/misc/xen-command-line.pandoc (submitters, less so), but this
>   approach needs extending to the whole of the sphinx docs (which in turn
>   requires the docs to stay high quality so its easy for maintainers to
>   know what is where).
> 
> Although this does not apply in my proposal, I think the key issue has been that reviewers and submitters of code often don't use our documentation. The wiki is seen as a separate thing and also has the disadvantage that it doesn't lend itself to supporting different versions of Xen. And most of the time, developers do not use it and neither contribute to it.
> 
> My hope was that by hosting documentation related to contribution workflow and other essential tasks close to other useful documentation this would enable change.
> 
> @Andy and others: I need to know whether you agree with my proposal and whether anyone has other suggestions.

If not already present in the release manager process checklist, we could specify documentation-related updates for each release, e.g. minimum text for new features, revisions to modified features, SUPPORT.md updates.

Rich

(resend with non-bulkmail address)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Stefano Stabellini]

On Tue, 25 Jun 2019, Julien Grall wrote:
> > > > The point here is that we can be flexible and creative about the way to
> > > > maintain the docs on xen.git. But as a technology is certainly better
> > > > than the wiki: we don't have to keep them all up-to-date with the code,
> > > > but at least this way we have a chance (if we want to). If we leave them
> > > > on the wiki, there is no chance.
> > > 
> > > I can't see how xen.git is going to be better if "we don't have to keep
> > > them
> > > all up-to-date".
> > 
> > That's because a contributor could add a patch at the end of a series to
> > update one of the docs, even if the doc in question comes with no
> > promises of being up-to-date.
> 
> I think this is going the wrong direction. The goal of using xen.git is to try
> to keep the documentation up-to-date.

Is the goal to keep the documentation fully up-to-date, or more
up-to-date than what we currently have on the wiki? I don't see this as
black and white. There are lot of stages of "up-to-dateness" in between.


> > > But my point here is most of the board should be trivial. The most of the
> > > non-trivial setup require non-upstream patch. While I am happy to see that
> > > on
> > > the wiki, I think xen.git should not promote such configuration at all. We
> > > are
> > > working upstream, not with unknown/untrusted stack.
> > > 
> > > For some working fully upstream, I don't think xen.git should promote any
> > > distros/versions of the kernel. However, this is ok on the wiki.
> > 
> > I would like to see the wiki disappear completely in the long term. As
> > we are moving more content to xen.git, it is not a good idea to have two
> > places where we keep information, for similar reasons why you suggested
> > to use in-code comments instead of docs to document interfaces. It
> > just takes more efforts to maintain information in two places and they
> > tend to get out of sync with each others.
> > 
> > If we make the wiki go away (I hope so), we'll need a place to store the
> > Arm board-specific documents, and other tutorials.
> 
> Removing the wiki is an honorable goal, however I don't think all the wiki is
> suitable for xen.git. The Arm board-specific documents is an example.
> 
> You actually haven't addressed my concern above. If you look at the wiki, a
> lot of them ([1], [2], [3]) contains non-upstreamed work or non-upstreamable
> hack.
> 
> For those containing only upstream work [4], the example is focusing on one
> set of distros. In the case of QEMU, I already had some people asking whether
> it is possible to use without U-boot. Why would we promote Ubuntu and not
> something else?
> 
> Overall, there are so many configurations possible (kernel, u-boot,
> distributions) that it does not makes sense to keep track of that in xen.git.
> 
> Instead, I think we should write generic doc on how to boot Xen on a
> U-boot/UEFI setup and inviting the users to look into more details for his
> board.

I don't think that we should host non-upstreamable hacks in general
either in xen.git. However, some boards require specific Linux kernel
trees to boot that are not upstream, so there is a fine line between
"non-upstreamable hack" and "following the docs and code provided with a
given board".

Similarly, I don't think we should have distro-specific information in
xen.git (or the wiki) either but sometimes we need to pick one as an
example. Otherwise, the tutorial wouldn't be complete.

In the case of the board specific files, we do both these things, but
because they are unavoidable, not because somebody wanted to advertise
Ubuntu, or preferred to use a private Linux kernel tree over upstream.
If that is the case, we should not import them into xen.git.

I also agree with you that if we are going to host only docs 100%
accurate, generic, and fully maintained by us on xen.git, then we should
NOT have the board specific docs there. But I would say that there is a
benefit in having docs not maintained by us and potentially not fully
up-to-date on xen.git that are useful to the community, like the board
specific docs. It would be easier to keep them "somewhat up to date"
release by release compared to the wiki, and in the future turn them
into "fully up to date" docs if we get more engagement.

If we leave them in the wiki I have the impression that their only
possible future is to rot and die...

This is not something that is going to happen anytime soon, so I think
we should take Lars' suggestion and talk about it at the Summit.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Migrating key developer docs to xen.git sphinx docs and refreshing them in the process

[Julien Grall]

Hi Stefano,

On 6/25/19 10:18 PM, Stefano Stabellini wrote:
> On Tue, 25 Jun 2019, Julien Grall wrote:
>>>>> The point here is that we can be flexible and creative about the way to
>>>>> maintain the docs on xen.git. But as a technology is certainly better
>>>>> than the wiki: we don't have to keep them all up-to-date with the code,
>>>>> but at least this way we have a chance (if we want to). If we leave them
>>>>> on the wiki, there is no chance.
>>>>
>>>> I can't see how xen.git is going to be better if "we don't have to keep
>>>> them
>>>> all up-to-date".
>>>
>>> That's because a contributor could add a patch at the end of a series to
>>> update one of the docs, even if the doc in question comes with no
>>> promises of being up-to-date.
>>
>> I think this is going the wrong direction. The goal of using xen.git is to try
>> to keep the documentation up-to-date.
> 
> Is the goal to keep the documentation fully up-to-date, or more
> up-to-date than what we currently have on the wiki? I don't see this as
> black and white. There are lot of stages of "up-to-dateness" in between.

 From the discussion it is pretty that we want the doc fully up-to-date.

> 
>>>> But my point here is most of the board should be trivial. The most of the
>>>> non-trivial setup require non-upstream patch. While I am happy to see that
>>>> on
>>>> the wiki, I think xen.git should not promote such configuration at all. We
>>>> are
>>>> working upstream, not with unknown/untrusted stack.
>>>>
>>>> For some working fully upstream, I don't think xen.git should promote any
>>>> distros/versions of the kernel. However, this is ok on the wiki.
>>>
>>> I would like to see the wiki disappear completely in the long term. As
>>> we are moving more content to xen.git, it is not a good idea to have two
>>> places where we keep information, for similar reasons why you suggested
>>> to use in-code comments instead of docs to document interfaces. It
>>> just takes more efforts to maintain information in two places and they
>>> tend to get out of sync with each others.
>>>
>>> If we make the wiki go away (I hope so), we'll need a place to store the
>>> Arm board-specific documents, and other tutorials.
>>
>> Removing the wiki is an honorable goal, however I don't think all the wiki is
>> suitable for xen.git. The Arm board-specific documents is an example.
>>
>> You actually haven't addressed my concern above. If you look at the wiki, a
>> lot of them ([1], [2], [3]) contains non-upstreamed work or non-upstreamable
>> hack.
>>
>> For those containing only upstream work [4], the example is focusing on one
>> set of distros. In the case of QEMU, I already had some people asking whether
>> it is possible to use without U-boot. Why would we promote Ubuntu and not
>> something else?
>>
>> Overall, there are so many configurations possible (kernel, u-boot,
>> distributions) that it does not makes sense to keep track of that in xen.git.
>>
>> Instead, I think we should write generic doc on how to boot Xen on a
>> U-boot/UEFI setup and inviting the users to look into more details for his
>> board.
> 
> I don't think that we should host non-upstreamable hacks in general
> either in xen.git. However, some boards require specific Linux kernel
> trees to boot that are not upstream, so there is a fine line between
> "non-upstreamable hack" and "following the docs and code provided with a
> given board".
> 
> Similarly, I don't think we should have distro-specific information in
> xen.git (or the wiki) either but sometimes we need to pick one as an
> example. Otherwise, the tutorial wouldn't be complete.
> 
> In the case of the board specific files, we do both these things, but
> because they are unavoidable, not because somebody wanted to advertise
> Ubuntu, or preferred to use a private Linux kernel tree over upstream.
> If that is the case, we should not import them into xen.git.

That's not correct, it is unavoidable because we never worked with the 
distros to integrate Xen on Arm. So we pile up crap as we can't say 
"Please select your distros and install Xen from the package".

Basically, we are doing the job of a distro. Instead, we should focus at 
better integration in distros such as Yocto, Zephyr.

> 
> I also agree with you that if we are going to host only docs 100%
> accurate, generic, and fully maintained by us on xen.git, then we should
> NOT have the board specific docs there. But I would say that there is a
> benefit in having docs not maintained by us and potentially not fully
> up-to-date on xen.git that are useful to the community, like the board
> specific docs. It would be easier to keep them "somewhat up to date"
> release by release compared to the wiki, and in the future turn them
> into "fully up to date" docs if we get more engagement.
> 
> If we leave them in the wiki I have the impression that their only
> possible future is to rot and die...
> 
> This is not something that is going to happen anytime soon, so I think
> we should take Lars' suggestion and talk about it at the Summit.

Sounds good to me.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel