[Draft Design v3] ACPI/IORT Support in Xen.

[Draft Design v3] ACPI/IORT Support in Xen.

[Manish Jaggi]

  ACPI/IORT Support in Xen.
  --------------------------
       Draft 3

  Revision History:

  Changes since v2:
  - Modified as per comments from Julien /Sameer/Andre

  Changes since v1:
  - Modified IORT Parsing data structures.
  - Added RID-StreamID and RID-DeviceID map as per Andre's suggestion.
  - Added reference code which can be read along with this document.
  - Removed domctl for DomU, it would be covered in PCI-PT design.

  Introduction:
  -------------

  I had sent out patch series [0] to hide smmu from Dom0 IORT.
  This document is a rework of the series as it:
  (a) extends scope by adding parsing of IORT table once
  and storing it in in-memory data structures, which can then be used
  for querying. This would eliminate the need to parse complete iort
  table multiple times.

  (b) Generation of IORT for domains be independent using a set of
  helper routines.

  Index
  --------
  1. What is IORT. What are its components ?
  2. Current Support in Xen
  3. IORT for Dom0
  4. IORT for DomU
  5. Parsing of IORT in Xen
  6. Generation of IORT
7. Implementation Phases
  8. References

  1. IORT Structure ?
  --------------------------------------------
  IORT refers to Input Output remapping table. It is essentially used to 
find
  information about the IO topology (PCIRC-SMMU-ITS) and relationships 
between
  devices.

  A general structure of IORT [1]:
  It has nodes for PCI RC, SMMU, ITS and Platform devices. Using an IORT 
table
  relationship between RID -> StreamID -> Deviceid can be obtained.
  Which device is behind which SMMU and which interrupt controller, topology
  is described in IORT Table.

  Some PCI RC may be not behind an SMMU, and directly map RID-DeviceID.

  RID is a requester ID in PCI context,
  StreamID is the ID of the device in SMMU context,
  DeviceID is the ID programmed in ITS.

  Each iort_node contains an ID map array to translate one ID into another.
  IDmap Entry {input_range, output_range, output_node_ref, id_count}
  This array is associated with PCI RC node, SMMU node, Named component 
node.
  and can reference to a SMMU or ITS node.

  2. Support of IORT
  ---------------------------
  It is proposed in this document to parse iort once and use the information
  to translate RID without traversing IORT again and again.

  Also Xen prepares an IORT table for dom0 based on host IORT.
  For DomU IORT table is required only in case of device passthrough.

  3. IORT for Dom0
  -----------------
  IORT for Dom0 is based on host iort. Few nodes could be removed or 
modified.
    For instance
  - Host SMMU nodes should not be present as Xen should only touch it.
  - platform devices (named components) would be passed as is. The 
visibility
    criterion for DOM0 is TDB.

  4. IORT for DomU
  -----------------
  IORT for DomU should be generated by toolstack. IORT table is only present
  in case of device passthrough.

  At a minimum domU IORT should include a single PCIRC and ITS Group.
  Similar PCIRC can be added in DSDT.
  The exact structure of DomU IORT would be covered along with PCI PT 
design.

  5. Parsing of IORT in Xen
  --------------------------
  IORT nodes can be saved in structures so that IORT table parsing can 
be done
  once and is reused by all xen subsystems like ITS / SMMU etc, domain 
creation.
  Proposed are the structures to hold IORT information. [4]

  struct rid_map_struct {
     void *pcirc_node;
     u16 inpute_base;
     u32 output_base;
     u16 id_ccount;
     struct list_head entry;
  };

Two global variables would hold the maps.
       struct list_head rid_streamid_map;
       struct list_head rid_deviceid_map;

  5.1 Functions to query StreamID and DeviceID from RID.

  void query_streamid(void *pcirc_node, u16 rid, u32 *streamid);
  void query_deviceid(void *pcirc_node, u16 rid, u32 *deviceid);

  Adding a mapping is done via helper functions

  int add_rid_streamid_map(void *pcirc_node, u32 ib, u32 ob, u32 idc)
  int add_rid_deviceid_map(void *pcirc_node, u32 ib, u32 ob, u32 idc)
  - rid-streamid map is straight forward and is created using pci_rc's idmap
  - rid-deviceid map is created by translating streamids to deviceids.
       fixup_rid_deviceid_map function does that. (See [6])

To keep the api similar to linux iort_node_map_rid be mapped to
query_streamid

6. IORT Generation
-------------------
It is proposed to have a common helper library to generate IORT for dom0/U.
Note: it is desired to have IORT generation code sharing between toolstack
and Xen.

a. For Dom0
  rid_deviceId_map can be used directly to generate dom0 IORT table.
  Exclusions of nodes is still open for suggestions.

b. For DomU
  Minimal structure is discussed in section 4. It will be further discussed
  in the context of PCI PT design.

7. Implementation Phases
-------------------------
a. IORT Parsing and RID Query
b. IORT Generation for Dom0
c. IORT Generation for DomU.

8. References:
-------------
[0] https://www.mail-archive.com/xen-devel@lists.xen.org/msg121667.html
[1] ARM DEN0049C: 
http://infocenter.arm.com/help/topic/com.arm.doc.den0049c/DEN0049C_IO_Remapping_Table.pdf
[2] https://www.mail-archive.com/xen-devel@lists.xen.org/msg123082.html
[3] NA.
[4] https://www.mail-archive.com/xen-devel@lists.xen.org/msg123434.html
[5] https://www.mail-archive.com/xen-devel@lists.xen.org/msg123080.html
[6] 
https://github.com/mjaggi-cavium/xen-wip/commit/df006d64bdbb5c8344de5a710da8bf64c9e8edd5 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Draft Design v3] ACPI/IORT Support in Xen.

[Julien Grall]

Hi Manish,

On 11/21/2017 07:22 AM, Manish Jaggi wrote:
> 
>   ACPI/IORT Support in Xen.
>   --------------------------
>        Draft 3
> 
>   Revision History:
> 
>   Changes since v2:
>   - Modified as per comments from Julien /Sameer/Andre
> 
>   Changes since v1:
>   - Modified IORT Parsing data structures.
>   - Added RID-StreamID and RID-DeviceID map as per Andre's suggestion.
>   - Added reference code which can be read along with this document.
>   - Removed domctl for DomU, it would be covered in PCI-PT design.
> 
>   Introduction:
>   -------------
> 
>   I had sent out patch series [0] to hide smmu from Dom0 IORT.
>   This document is a rework of the series as it:
>   (a) extends scope by adding parsing of IORT table once

Can you stick with one name. Either IORT or "IORT table" but not mixing 
both. Technically the T in IORT stands for Table. So it would be better 
if you only use "IORT" everywhere in that document.

>   and storing it in in-memory data structures, which can then be used
>   for querying. This would eliminate the need to parse complete iort

s/iort/IORT/

>   table multiple times.
> 
>   (b) Generation of IORT for domains be independent using a set of
>   helper routines.
> 
>   Index
>   --------
>   1. What is IORT. What are its components ?
>   2. Current Support in Xen
>   3. IORT for Dom0
>   4. IORT for DomU
>   5. Parsing of IORT in Xen
>   6. Generation of IORT
> 7. Implementation Phases
>   8. References
> 
>   1. IORT Structure ?
>   --------------------------------------------
>   IORT refers to Input Output remapping table. It is essentially used to 
> find
>   information about the IO topology (PCIRC-SMMU-ITS) and relationships 
> between
>   devices.
> 
>   A general structure of IORT [1]:
>   It has nodes for PCI RC, SMMU, ITS and Platform devices. Using an IORT 
> table
>   relationship between RID -> StreamID -> Deviceid can be obtained.
>   Which device is behind which SMMU and which interrupt controller, 
> topology
>   is described in IORT Table.
> 
>   Some PCI RC may be not behind an SMMU, and directly map RID-DeviceID.
> 
>   RID is a requester ID in PCI context,
>   StreamID is the ID of the device in SMMU context,
>   DeviceID is the ID programmed in ITS.
> 
>   Each iort_node contains an ID map array to translate one ID into another.
>   IDmap Entry {input_range, output_range, output_node_ref, id_count}
>   This array is associated with PCI RC node, SMMU node, Named component 
> node.
>   and can reference to a SMMU or ITS node.
> 
>   2. Support of IORT
>   ---------------------------
>   It is proposed in this document to parse iort once and use the 
> information
>   to translate RID without traversing IORT again and again.
> 
>   Also Xen prepares an IORT table for dom0 based on host IORT.
>   For DomU IORT table is required only in case of device passthrough.
> 
>   3. IORT for Dom0
>   -----------------
>   IORT for Dom0 is based on host iort. Few nodes could be removed or 
> modified.
>     For instance
>   - Host SMMU nodes should not be present as Xen should only touch it.
>   - platform devices (named components) would be passed as is. The 
> visibility
>     criterion for DOM0 is TDB.

s/TDB/TBD/

> 
>   4. IORT for DomU
>   -----------------
>   IORT for DomU should be generated by toolstack. IORT table is only 
> present
>   in case of device passthrough.
> 
>   At a minimum domU IORT should include a single PCIRC and ITS Group.

s/domU/DomU/

>   Similar PCIRC can be added in DSDT.
>   The exact structure of DomU IORT would be covered along with PCI PT 
> design.
> 
>   5. Parsing of IORT in Xen
>   --------------------------
>   IORT nodes can be saved in structures so that IORT table parsing can 
> be done
>   once and is reused by all xen subsystems like ITS / SMMU etc, domain 

s/xen/Xen/

> creation.
>   Proposed are the structures to hold IORT information. [4]
> 
>   struct rid_map_struct {
>      void *pcirc_node;
>      u16 inpute_base;
>      u32 output_base;
>      u16 id_ccount;

s/id_ccount/id_count/

>      struct list_head entry;
>   };
> 
> Two global variables would hold the maps.
>        struct list_head rid_streamid_map;
>        struct list_head rid_deviceid_map;
> 
>   5.1 Functions to query StreamID and DeviceID from RID.
> 
>   void query_streamid(void *pcirc_node, u16 rid, u32 *streamid);
>   void query_deviceid(void *pcirc_node, u16 rid, u32 *deviceid);

I don't fully understand what is the "void *pcirc_node" and how a caller 
would find it.

> 
>   Adding a mapping is done via helper functions
> 
>   int add_rid_streamid_map(void *pcirc_node, u32 ib, u32 ob, u32 idc)
>   int add_rid_deviceid_map(void *pcirc_node, u32 ib, u32 ob, u32 idc)

What does ib, ob, idc mean? I thought we asked to avoid unknown acronym.

>   - rid-streamid map is straight forward and is created using pci_rc's 
> idmap
>   - rid-deviceid map is created by translating streamids to deviceids.
>        fixup_rid_deviceid_map function does that. (See [6])

I will look at the code when it is formally sent on the mailing list. 
The design documents looks good, thought it may need to be refine during 
the code review.

> 
> To keep the api similar to linux iort_node_map_rid be mapped to
> query_streamid
> 
> 6. IORT Generation
> -------------------
> It is proposed to have a common helper library to generate IORT for dom0/U.
> Note: it is desired to have IORT generation code sharing between toolstack
> and Xen.
> 
> a. For Dom0
>   rid_deviceId_map can be used directly to generate dom0 IORT table.
>   Exclusions of nodes is still open for suggestions.
> 
> b. For DomU
>   Minimal structure is discussed in section 4. It will be further discussed
>   in the context of PCI PT design.
> 
> 7. Implementation Phases
> -------------------------
> a. IORT Parsing and RID Query
> b. IORT Generation for Dom0
> c. IORT Generation for DomU.
> 
> 8. References:
> -------------
> [0] https://www.mail-archive.com/xen-devel@lists.xen.org/msg121667.html
> [1] ARM DEN0049C: 
> http://infocenter.arm.com/help/topic/com.arm.doc.den0049c/DEN0049C_IO_Remapping_Table.pdf 
> 
> [2] https://www.mail-archive.com/xen-devel@lists.xen.org/msg123082.html
> [3] NA.
> [4] https://www.mail-archive.com/xen-devel@lists.xen.org/msg123434.html
> [5] https://www.mail-archive.com/xen-devel@lists.xen.org/msg123080.html
> [6] 
> https://github.com/mjaggi-cavium/xen-wip/commit/df006d64bdbb5c8344de5a710da8bf64c9e8edd5 

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel