[PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Paul Durrant]

This patch adds a new 'designs' subdirectory under docs as a repository
for this and future design proposals.

Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
---

For convenience this document can also be viewed in PDF at:

http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
---
 docs/designs/hvmmem_ioreq_server.md | 63 +++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)
 create mode 100755 docs/designs/hvmmem_ioreq_server.md

diff --git a/docs/designs/hvmmem_ioreq_server.md b/docs/designs/hvmmem_ioreq_server.md
new file mode 100755
index 0000000..47fa715
--- /dev/null
+++ b/docs/designs/hvmmem_ioreq_server.md
@@ -0,0 +1,63 @@
+HVMMEM\_ioreq\_serverX
+----------------------
+
+Background
+==========
+
+The concept of the IOREQ server was introduced to allow multiple distinct
+device emulators to a single VM. The XenGT project uses an IOREQ server to
+provide mediated pass-through of Intel GPUs to guests and, as part of the
+mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
+reside in guest RAM.
+
+The current implementation of this sets the type of GTT pages to type
+HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to such pages,
+and then maps the guest physical addresses of those pages to the XenGT
+IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server hypercall.
+However, because the number of GTTs is potentially large, using this
+approach does not scale well.
+
+Proposal
+========
+
+Because the number of spare types available in the P2M type-space is
+currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
+replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
+P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
+and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
+can be added.
+
+Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
+HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore
+it should only be possible to set the type of a page to
+HVMMEM\_ioreq\_serverX if that page is currently of type HVMMEM\_ram\_rw.
+
+To allow an IOREQ server to claim or release a claim to a type a new pair
+of hypercalls will be introduced:
+
+- HVMOP\_map\_mem\_type\_to\_ioreq\_server
+- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
+
+and an associated argument structure:
+
+		struct hvm_ioreq_mem_type {
+			domid_t domid;      /* IN - domain to be serviced */
+			ioservid_t id;      /* IN - server id */
+			hvmmem_type_t type; /* IN - memory type */
+			uint32_t flags;     /* IN - types of access to be
+	                                    intercepted */
+
+	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
+	#define HVMOP_IOREQ_MEM_ACCESS_READ \
+	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
+
+	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
+	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
+	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
+
+		};
+
+
+Once the type has been claimed then the requested types of access to any
+page of the claimed type will be passed to the IOREQ server for handling.
+Only HVMMEM\_ioreq\_serverX types may be claimed.
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Andrew Cooper]

On 25/02/16 15:49, Paul Durrant wrote:
> This patch adds a new 'designs' subdirectory under docs as a repository
> for this and future design proposals.
>
> Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> ---
>
> For convenience this document can also be viewed in PDF at:
>
> http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
> ---
>  docs/designs/hvmmem_ioreq_server.md | 63 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 63 insertions(+)
>  create mode 100755 docs/designs/hvmmem_ioreq_server.md

If you name it .markdown, the docs buildsystem will be able to publish
it automatically.  Alternatively, teach the build system about .md.

On the other hand, .pandoc tends to end up making nicer PDFs.

>
> diff --git a/docs/designs/hvmmem_ioreq_server.md b/docs/designs/hvmmem_ioreq_server.md
> new file mode 100755
> index 0000000..47fa715
> --- /dev/null
> +++ b/docs/designs/hvmmem_ioreq_server.md
> @@ -0,0 +1,63 @@
> +HVMMEM\_ioreq\_serverX
> +----------------------
> +
> +Background
> +==========
> +
> +The concept of the IOREQ server was introduced to allow multiple distinct
> +device emulators to a single VM. The XenGT project uses an IOREQ server to
> +provide mediated pass-through of Intel GPUs to guests and, as part of the
> +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
> +reside in guest RAM.
> +
> +The current implementation of this sets the type of GTT pages to type
> +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to such pages,
> +and then maps the guest physical addresses of those pages to the XenGT

"then sends the guest physical" surely?

> +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server hypercall.
> +However, because the number of GTTs is potentially large, using this
> +approach does not scale well.
> +
> +Proposal
> +========
> +
> +Because the number of spare types available in the P2M type-space is
> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
> +P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
> +can be added.
> +
> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore
> +it should only be possible to set the type of a page to
> +HVMMEM\_ioreq\_serverX if that page is currently of type HVMMEM\_ram\_rw.
> +
> +To allow an IOREQ server to claim or release a claim to a type a new pair
> +of hypercalls will be introduced:
> +
> +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
> +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
> +
> +and an associated argument structure:
> +
> +		struct hvm_ioreq_mem_type {
> +			domid_t domid;      /* IN - domain to be serviced */
> +			ioservid_t id;      /* IN - server id */
> +			hvmmem_type_t type; /* IN - memory type */
> +			uint32_t flags;     /* IN - types of access to be
> +	                                    intercepted */
> +
> +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
> +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)

(1U << ...)

> +
> +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
> +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
> +
> +		};
> +
> +
> +Once the type has been claimed then the requested types of access to any
> +page of the claimed type will be passed to the IOREQ server for handling.
> +Only HVMMEM\_ioreq\_serverX types may be claimed.

LGTM.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Jan Beulich]

>>> On 25.02.16 at 16:49, <paul.durrant@citrix.com> wrote:
> +To allow an IOREQ server to claim or release a claim to a type a new pair
> +of hypercalls will be introduced:
> +
> +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
> +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
> +
> +and an associated argument structure:
> +
> +		struct hvm_ioreq_mem_type {
> +			domid_t domid;      /* IN - domain to be serviced */
> +			ioservid_t id;      /* IN - server id */
> +			hvmmem_type_t type; /* IN - memory type */
> +			uint32_t flags;     /* IN - types of access to be
> +	                                    intercepted */
> +
> +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
> +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
> +
> +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
> +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
> +
> +		};

How about having just one new hypercall, with flags being zero
meaning "unmap"?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Paul Durrant]

> -----Original Message-----
> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com]
> Sent: 25 February 2016 16:28
> To: Paul Durrant; xen-devel@lists.xenproject.org
> Subject: Re: [Xen-devel] [PATCH] docs/design: introduce
> HVMMEM_ioreq_serverX types
> 
> On 25/02/16 15:49, Paul Durrant wrote:
> > This patch adds a new 'designs' subdirectory under docs as a repository
> > for this and future design proposals.
> >
> > Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> > ---
> >
> > For convenience this document can also be viewed in PDF at:
> >
> > http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
> > ---
> >  docs/designs/hvmmem_ioreq_server.md | 63
> +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 63 insertions(+)
> >  create mode 100755 docs/designs/hvmmem_ioreq_server.md
> 
> If you name it .markdown, the docs buildsystem will be able to publish
> it automatically.  Alternatively, teach the build system about .md.
> 
> On the other hand, .pandoc tends to end up making nicer PDFs.
> 

Ok. Good idea. I'll look at doing that.

> >
> > diff --git a/docs/designs/hvmmem_ioreq_server.md
> b/docs/designs/hvmmem_ioreq_server.md
> > new file mode 100755
> > index 0000000..47fa715
> > --- /dev/null
> > +++ b/docs/designs/hvmmem_ioreq_server.md
> > @@ -0,0 +1,63 @@
> > +HVMMEM\_ioreq\_serverX
> > +----------------------
> > +
> > +Background
> > +==========
> > +
> > +The concept of the IOREQ server was introduced to allow multiple distinct
> > +device emulators to a single VM. The XenGT project uses an IOREQ server
> to
> > +provide mediated pass-through of Intel GPUs to guests and, as part of the
> > +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
> > +reside in guest RAM.
> > +
> > +The current implementation of this sets the type of GTT pages to type
> > +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to
> such pages,
> > +and then maps the guest physical addresses of those pages to the XenGT
> 
> "then sends the guest physical" surely?
> 
> > +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server
> hypercall.
> > +However, because the number of GTTs is potentially large, using this
> > +approach does not scale well.
> > +
> > +Proposal
> > +========
> > +
> > +Because the number of spare types available in the P2M type-space is
> > +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm
> be
> > +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
> > +P2M type-space is increased, this can be renamed to
> HVMMEM\_ioreq\_server0
> > +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc.
> types
> > +can be added.
> > +
> > +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the
> same as
> > +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server.
> Furthermore
> > +it should only be possible to set the type of a page to
> > +HVMMEM\_ioreq\_serverX if that page is currently of type
> HVMMEM\_ram\_rw.
> > +
> > +To allow an IOREQ server to claim or release a claim to a type a new pair
> > +of hypercalls will be introduced:
> > +
> > +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
> > +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
> > +
> > +and an associated argument structure:
> > +
> > +		struct hvm_ioreq_mem_type {
> > +			domid_t domid;      /* IN - domain to be serviced */
> > +			ioservid_t id;      /* IN - server id */
> > +			hvmmem_type_t type; /* IN - memory type */
> > +			uint32_t flags;     /* IN - types of access to be
> > +	                                    intercepted */
> > +
> > +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
> > +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
> > +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
> 
> (1U << ...)
> 

Yep. Should be.

> > +
> > +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
> > +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
> > +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
> > +
> > +		};
> > +
> > +
> > +Once the type has been claimed then the requested types of access to
> any
> > +page of the claimed type will be passed to the IOREQ server for handling.
> > +Only HVMMEM\_ioreq\_serverX types may be claimed.
> 
> LGTM.

Thanks :-)

  Paul

> 
> ~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Paul Durrant]

> -----Original Message-----
> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Jan
> Beulich
> Sent: 25 February 2016 16:48
> To: Paul Durrant
> Cc: xen-devel@lists.xenproject.org
> Subject: Re: [Xen-devel] [PATCH] docs/design: introduce
> HVMMEM_ioreq_serverX types
> 
> >>> On 25.02.16 at 16:49, <paul.durrant@citrix.com> wrote:
> > +To allow an IOREQ server to claim or release a claim to a type a new pair
> > +of hypercalls will be introduced:
> > +
> > +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
> > +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
> > +
> > +and an associated argument structure:
> > +
> > +		struct hvm_ioreq_mem_type {
> > +			domid_t domid;      /* IN - domain to be serviced */
> > +			ioservid_t id;      /* IN - server id */
> > +			hvmmem_type_t type; /* IN - memory type */
> > +			uint32_t flags;     /* IN - types of access to be
> > +	                                    intercepted */
> > +
> > +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
> > +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
> > +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
> > +
> > +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
> > +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
> > +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
> > +
> > +		};
> 
> How about having just one new hypercall, with flags being zero
> meaning "unmap"?

Hmm, yes we could do it that way. I saves an op code so I'll re-spin the doc.

Thanks,

  Paul

> 
> Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Tian, Kevin]

> From: Paul Durrant
> Sent: Thursday, February 25, 2016 11:49 PM
> 
> This patch adds a new 'designs' subdirectory under docs as a repository
> for this and future design proposals.
> 
> Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> ---
> 
> For convenience this document can also be viewed in PDF at:
> 
> http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
> ---
>  docs/designs/hvmmem_ioreq_server.md | 63
> +++++++++++++++++++++++++++++++++++++
>  1 file changed, 63 insertions(+)
>  create mode 100755 docs/designs/hvmmem_ioreq_server.md
> 
> diff --git a/docs/designs/hvmmem_ioreq_server.md
> b/docs/designs/hvmmem_ioreq_server.md
> new file mode 100755
> index 0000000..47fa715
> --- /dev/null
> +++ b/docs/designs/hvmmem_ioreq_server.md
> @@ -0,0 +1,63 @@
> +HVMMEM\_ioreq\_serverX
> +----------------------
> +
> +Background
> +==========
> +
> +The concept of the IOREQ server was introduced to allow multiple distinct
> +device emulators to a single VM. The XenGT project uses an IOREQ server to
> +provide mediated pass-through of Intel GPUs to guests and, as part of the
> +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
> +reside in guest RAM.
> +
> +The current implementation of this sets the type of GTT pages to type
> +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to such pages,
> +and then maps the guest physical addresses of those pages to the XenGT
> +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server hypercall.
> +However, because the number of GTTs is potentially large, using this
> +approach does not scale well.
> +
> +Proposal
> +========
> +
> +Because the number of spare types available in the P2M type-space is
> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
> +P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
> +can be added.
> +
> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore
> +it should only be possible to set the type of a page to
> +HVMMEM\_ioreq\_serverX if that page is currently of type HVMMEM\_ram\_rw.

Is there similar assumption on the opposite change, i.e. from ioreq_serverX
only to ram_rw?

> +
> +To allow an IOREQ server to claim or release a claim to a type a new pair
> +of hypercalls will be introduced:
> +
> +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
> +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
> +
> +and an associated argument structure:
> +
> +		struct hvm_ioreq_mem_type {
> +			domid_t domid;      /* IN - domain to be serviced */
> +			ioservid_t id;      /* IN - server id */
> +			hvmmem_type_t type; /* IN - memory type */
> +			uint32_t flags;     /* IN - types of access to be
> +	                                    intercepted */
> +
> +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
> +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
> +
> +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
> +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
> +
> +		};
> +
> +
> +Once the type has been claimed then the requested types of access to any
> +page of the claimed type will be passed to the IOREQ server for handling.
> +Only HVMMEM\_ioreq\_serverX types may be claimed.
> --

It'd good to also add how to handle multiple ioreq servers claiming
one same type for a given domain.

Thanks
Kevin
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Yu, Zhang]

Hi Paul,

   Thanks a lot for your help on this! And below are my questions.

On 2/25/2016 11:49 PM, Paul Durrant wrote:
> This patch adds a new 'designs' subdirectory under docs as a repository
> for this and future design proposals.
>
> Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> ---
>
> For convenience this document can also be viewed in PDF at:
>
> http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
> ---
>   docs/designs/hvmmem_ioreq_server.md | 63 +++++++++++++++++++++++++++++++++++++
>   1 file changed, 63 insertions(+)
>   create mode 100755 docs/designs/hvmmem_ioreq_server.md
>
> diff --git a/docs/designs/hvmmem_ioreq_server.md b/docs/designs/hvmmem_ioreq_server.md
> new file mode 100755
> index 0000000..47fa715
> --- /dev/null
> +++ b/docs/designs/hvmmem_ioreq_server.md
> @@ -0,0 +1,63 @@
> +HVMMEM\_ioreq\_serverX
> +----------------------
> +
> +Background
> +==========
> +
> +The concept of the IOREQ server was introduced to allow multiple distinct
> +device emulators to a single VM. The XenGT project uses an IOREQ server to
> +provide mediated pass-through of Intel GPUs to guests and, as part of the
> +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
> +reside in guest RAM.
> +
> +The current implementation of this sets the type of GTT pages to type
> +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to such pages,
> +and then maps the guest physical addresses of those pages to the XenGT
> +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server hypercall.
> +However, because the number of GTTs is potentially large, using this
> +approach does not scale well.
> +
> +Proposal
> +========
> +
> +Because the number of spare types available in the P2M type-space is
> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
> +P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
> +can be added.
> +
> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore

Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX type,
its access rights in P2M still remains unchanged? So the new hypercall
pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also responsible for
the PTE updates on the access bits?

If it is true, I'm afraid this would be time consuming, because the
map/unmap will have to traverse all P2M structures to detect the PTEs
with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
triggered dynamically with the construction/destruction of shadow PPGTT.
But I'm not sure to which degree the performance casualties will be,
with frequent EPT table walk and EPT tlb flush.

If it is not, I guess we can(e.g. when trying to WP a gfn):
1> use HVMOP_set_mem_type to set the HVMMEM_ioreq_serverX flag, which
for the write protected case works the same as HVMMEM_mmio_write_dm;
If successful, accesses to a page of type HVMMEM_ioreq_serverX should
trigger the ioreq server selection path, but will be discarded.
2> after HVMOP_map_mem_type_to_ioreq_server is called, all accesses to
this pages of type HVMMEM_ioreq_serverX would be forwarded to a
specified ioreq server.

As to XenGT backend device model, we only need to use the map hypercall
once when trying to contruct the first shadow PPGTT, and use the unmap
hypercall when a VM is going to be torn down.

Any suggestions? :)

Thanks
Yu

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Jan Beulich]

>>> On 26.02.16 at 05:24, <kevin.tian@intel.com> wrote:
>> +Proposal
>> +========
>> +
>> +Because the number of spare types available in the P2M type-space is
>> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
>> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
>> +P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
>> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
>> +can be added.
>> +
>> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
>> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore
>> +it should only be possible to set the type of a page to
>> +HVMMEM\_ioreq\_serverX if that page is currently of type HVMMEM\_ram\_rw.
> 
> Is there similar assumption on the opposite change, i.e. from ioreq_serverX
> only to ram_rw?

Yes. The above should be re-worded to make clear this is a
bi-directional requirement.

>> +To allow an IOREQ server to claim or release a claim to a type a new pair
>> +of hypercalls will be introduced:
>> +
>> +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
>> +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
>> +
>> +and an associated argument structure:
>> +
>> +		struct hvm_ioreq_mem_type {
>> +			domid_t domid;      /* IN - domain to be serviced */
>> +			ioservid_t id;      /* IN - server id */
>> +			hvmmem_type_t type; /* IN - memory type */
>> +			uint32_t flags;     /* IN - types of access to be
>> +	                                    intercepted */
>> +
>> +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
>> +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
>> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
>> +
>> +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
>> +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
>> +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
>> +
>> +		};
>> +
>> +
>> +Once the type has been claimed then the requested types of access to any
>> +page of the claimed type will be passed to the IOREQ server for handling.
>> +Only HVMMEM\_ioreq\_serverX types may be claimed.
>> --
> 
> It'd good to also add how to handle multiple ioreq servers claiming
> one same type for a given domain.

It's going to fail, but I agree that perhaps this would better be
spelled out.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Paul Durrant]

> -----Original Message-----
> From: Tian, Kevin [mailto:kevin.tian@intel.com]
> Sent: 26 February 2016 04:25
> To: Paul Durrant; xen-devel@lists.xenproject.org
> Subject: RE: [Xen-devel] [PATCH] docs/design: introduce
> HVMMEM_ioreq_serverX types
> 
> > From: Paul Durrant
> > Sent: Thursday, February 25, 2016 11:49 PM
> >
> > This patch adds a new 'designs' subdirectory under docs as a repository
> > for this and future design proposals.
> >
> > Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> > ---
> >
> > For convenience this document can also be viewed in PDF at:
> >
> > http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
> > ---
> >  docs/designs/hvmmem_ioreq_server.md | 63
> > +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 63 insertions(+)
> >  create mode 100755 docs/designs/hvmmem_ioreq_server.md
> >
> > diff --git a/docs/designs/hvmmem_ioreq_server.md
> > b/docs/designs/hvmmem_ioreq_server.md
> > new file mode 100755
> > index 0000000..47fa715
> > --- /dev/null
> > +++ b/docs/designs/hvmmem_ioreq_server.md
> > @@ -0,0 +1,63 @@
> > +HVMMEM\_ioreq\_serverX
> > +----------------------
> > +
> > +Background
> > +==========
> > +
> > +The concept of the IOREQ server was introduced to allow multiple distinct
> > +device emulators to a single VM. The XenGT project uses an IOREQ server
> to
> > +provide mediated pass-through of Intel GPUs to guests and, as part of the
> > +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
> > +reside in guest RAM.
> > +
> > +The current implementation of this sets the type of GTT pages to type
> > +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to
> such pages,
> > +and then maps the guest physical addresses of those pages to the XenGT
> > +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server
> hypercall.
> > +However, because the number of GTTs is potentially large, using this
> > +approach does not scale well.
> > +
> > +Proposal
> > +========
> > +
> > +Because the number of spare types available in the P2M type-space is
> > +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm
> be
> > +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
> > +P2M type-space is increased, this can be renamed to
> HVMMEM\_ioreq\_server0
> > +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc.
> types
> > +can be added.
> > +
> > +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the
> same as
> > +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server.
> Furthermore
> > +it should only be possible to set the type of a page to
> > +HVMMEM\_ioreq\_serverX if that page is currently of type
> HVMMEM\_ram\_rw.
> 
> Is there similar assumption on the opposite change, i.e. from ioreq_serverX
> only to ram_rw?
> 

Yes, I will call that out.

> > +
> > +To allow an IOREQ server to claim or release a claim to a type a new pair
> > +of hypercalls will be introduced:
> > +
> > +- HVMOP\_map\_mem\_type\_to\_ioreq\_server
> > +- HVMOP\_unmap\_mem\_type\_from\_ioreq\_server
> > +
> > +and an associated argument structure:
> > +
> > +		struct hvm_ioreq_mem_type {
> > +			domid_t domid;      /* IN - domain to be serviced */
> > +			ioservid_t id;      /* IN - server id */
> > +			hvmmem_type_t type; /* IN - memory type */
> > +			uint32_t flags;     /* IN - types of access to be
> > +	                                    intercepted */
> > +
> > +	#define _HVMOP_IOREQ_MEM_ACCESS_READ 0
> > +	#define HVMOP_IOREQ_MEM_ACCESS_READ \
> > +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_READ)
> > +
> > +	#define _HVMOP_IOREQ_MEM_ACCESS_WRITE 1
> > +	#define HVMOP_IOREQ_MEM_ACCESS_WRITE \
> > +	        (1 << _HVMOP_IOREQ_MEM_ACCESS_WRITE)
> > +
> > +		};
> > +
> > +
> > +Once the type has been claimed then the requested types of access to
> any
> > +page of the claimed type will be passed to the IOREQ server for handling.
> > +Only HVMMEM\_ioreq\_serverX types may be claimed.
> > --
> 
> It'd good to also add how to handle multiple ioreq servers claiming
> one same type for a given domain.
> 

That would clearly be an error so I imagine -EBUSY would be an appropriate return value from the map hypercall.

  Paul

> Thanks
> Kevin
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Jan Beulich]

>>> On 26.02.16 at 07:59, <yu.c.zhang@linux.intel.com> wrote:
>> +Proposal
>> +========
>> +
>> +Because the number of spare types available in the P2M type-space is
>> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
>> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
>> +P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
>> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
>> +can be added.
>> +
>> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
>> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore
> 
> Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX type,
> its access rights in P2M still remains unchanged? So the new hypercall
> pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also responsible for
> the PTE updates on the access bits?
> 
> If it is true, I'm afraid this would be time consuming, because the
> map/unmap will have to traverse all P2M structures to detect the PTEs
> with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
> triggered dynamically with the construction/destruction of shadow PPGTT.
> But I'm not sure to which degree the performance casualties will be,
> with frequent EPT table walk and EPT tlb flush.

No walking of EPT trees will be necessary in that case, just like it
already has been made unnecessary for other changes resulting
in various PTE attributes needing re-calculation. We'll only need
to extend the p2m_memory_type_changed() mechanism to cover
changes like this one.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Paul Durrant]

> -----Original Message-----
> From: Yu, Zhang [mailto:yu.c.zhang@linux.intel.com]
> Sent: 26 February 2016 06:59
> To: Paul Durrant; xen-devel@lists.xenproject.org
> Subject: Re: [Xen-devel] [PATCH] docs/design: introduce
> HVMMEM_ioreq_serverX types
> 
> Hi Paul,
> 
>    Thanks a lot for your help on this! And below are my questions.
> 
> On 2/25/2016 11:49 PM, Paul Durrant wrote:
> > This patch adds a new 'designs' subdirectory under docs as a repository
> > for this and future design proposals.
> >
> > Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> > ---
> >
> > For convenience this document can also be viewed in PDF at:
> >
> > http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
> > ---
> >   docs/designs/hvmmem_ioreq_server.md | 63
> +++++++++++++++++++++++++++++++++++++
> >   1 file changed, 63 insertions(+)
> >   create mode 100755 docs/designs/hvmmem_ioreq_server.md
> >
> > diff --git a/docs/designs/hvmmem_ioreq_server.md
> b/docs/designs/hvmmem_ioreq_server.md
> > new file mode 100755
> > index 0000000..47fa715
> > --- /dev/null
> > +++ b/docs/designs/hvmmem_ioreq_server.md
> > @@ -0,0 +1,63 @@
> > +HVMMEM\_ioreq\_serverX
> > +----------------------
> > +
> > +Background
> > +==========
> > +
> > +The concept of the IOREQ server was introduced to allow multiple distinct
> > +device emulators to a single VM. The XenGT project uses an IOREQ server
> to
> > +provide mediated pass-through of Intel GPUs to guests and, as part of the
> > +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
> > +reside in guest RAM.
> > +
> > +The current implementation of this sets the type of GTT pages to type
> > +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to
> such pages,
> > +and then maps the guest physical addresses of those pages to the XenGT
> > +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server
> hypercall.
> > +However, because the number of GTTs is potentially large, using this
> > +approach does not scale well.
> > +
> > +Proposal
> > +========
> > +
> > +Because the number of spare types available in the P2M type-space is
> > +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm
> be
> > +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
> > +P2M type-space is increased, this can be renamed to
> HVMMEM\_ioreq\_server0
> > +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc.
> types
> > +can be added.
> > +
> > +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the
> same as
> > +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server.
> Furthermore
> 
> Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX
> type,
> its access rights in P2M still remains unchanged? So the new hypercall
> pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also responsible
> for
> the PTE updates on the access bits?

Yes, the access bits would not change *on existing* pages of this type until the type is claimed.

> 
> If it is true, I'm afraid this would be time consuming, because the
> map/unmap will have to traverse all P2M structures to detect the PTEs
> with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
> triggered dynamically with the construction/destruction of shadow PPGTT.
> But I'm not sure to which degree the performance casualties will be,
> with frequent EPT table walk and EPT tlb flush.

I don't see the concern. I am assuming XenGT would claim the type at start of day and then just to HVMOP_set_mem_type operations on PPGTT pages as necessary, which would not require p2m traversal (since that hypercall takes the gfn as an arg) and the EPT flush requirements are no different to setting the mmio_write_dm type in the current implementation.

> 
> If it is not, I guess we can(e.g. when trying to WP a gfn):
> 1> use HVMOP_set_mem_type to set the HVMMEM_ioreq_serverX flag,
> which
> for the write protected case works the same as HVMMEM_mmio_write_dm;
> If successful, accesses to a page of type HVMMEM_ioreq_serverX should
> trigger the ioreq server selection path, but will be discarded.
> 2> after HVMOP_map_mem_type_to_ioreq_server is called, all accesses to
> this pages of type HVMMEM_ioreq_serverX would be forwarded to a
> specified ioreq server.
> 
> As to XenGT backend device model, we only need to use the map hypercall
> once when trying to contruct the first shadow PPGTT, and use the unmap
> hypercall when a VM is going to be torn down.
> 

Yes, that's correct. A single claim at start of day and a single 'unclaim' at end of day. In between the current HVMOP_set_mem_type is pretty much as before (just with the new type name) and there is no longer a need for the hypercall to map the range to the ioreq server.

  Paul

> Any suggestions? :)
> 
> Thanks
> Yu
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Yu, Zhang]

Thanks, Paul.

On 2/26/2016 5:21 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: Yu, Zhang [mailto:yu.c.zhang@linux.intel.com]
>> Sent: 26 February 2016 06:59
>> To: Paul Durrant; xen-devel@lists.xenproject.org
>> Subject: Re: [Xen-devel] [PATCH] docs/design: introduce
>> HVMMEM_ioreq_serverX types
>>
>> Hi Paul,
>>
>>     Thanks a lot for your help on this! And below are my questions.
>>
>> On 2/25/2016 11:49 PM, Paul Durrant wrote:
>>> This patch adds a new 'designs' subdirectory under docs as a repository
>>> for this and future design proposals.
>>>
>>> Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
>>> ---
>>>
>>> For convenience this document can also be viewed in PDF at:
>>>
>>> http://xenbits.xen.org/people/pauldu/hvmmem_ioreq_server.pdf
>>> ---
>>>    docs/designs/hvmmem_ioreq_server.md | 63
>> +++++++++++++++++++++++++++++++++++++
>>>    1 file changed, 63 insertions(+)
>>>    create mode 100755 docs/designs/hvmmem_ioreq_server.md
>>>
>>> diff --git a/docs/designs/hvmmem_ioreq_server.md
>> b/docs/designs/hvmmem_ioreq_server.md
>>> new file mode 100755
>>> index 0000000..47fa715
>>> --- /dev/null
>>> +++ b/docs/designs/hvmmem_ioreq_server.md
>>> @@ -0,0 +1,63 @@
>>> +HVMMEM\_ioreq\_serverX
>>> +----------------------
>>> +
>>> +Background
>>> +==========
>>> +
>>> +The concept of the IOREQ server was introduced to allow multiple distinct
>>> +device emulators to a single VM. The XenGT project uses an IOREQ server
>> to
>>> +provide mediated pass-through of Intel GPUs to guests and, as part of the
>>> +mediation, needs to intercept accesses to GPU page-tables (or GTTs) that
>>> +reside in guest RAM.
>>> +
>>> +The current implementation of this sets the type of GTT pages to type
>>> +HVMMEM\_mmio\_write\_dm, which causes Xen to emulate writes to
>> such pages,
>>> +and then maps the guest physical addresses of those pages to the XenGT
>>> +IOREQ server using the HVMOP\_map\_io\_range\_to\_ioreq\_server
>> hypercall.
>>> +However, because the number of GTTs is potentially large, using this
>>> +approach does not scale well.
>>> +
>>> +Proposal
>>> +========
>>> +
>>> +Because the number of spare types available in the P2M type-space is
>>> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm
>> be
>>> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
>>> +P2M type-space is increased, this can be renamed to
>> HVMMEM\_ioreq\_server0
>>> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc.
>> types
>>> +can be added.
>>> +
>>> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the
>> same as
>>> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server.
>> Furthermore
>>
>> Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX
>> type,
>> its access rights in P2M still remains unchanged? So the new hypercall
>> pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also responsible
>> for
>> the PTE updates on the access bits?
>
> Yes, the access bits would not change *on existing* pages of this type until the type is claimed.
>
>>
>> If it is true, I'm afraid this would be time consuming, because the
>> map/unmap will have to traverse all P2M structures to detect the PTEs
>> with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
>> triggered dynamically with the construction/destruction of shadow PPGTT.
>> But I'm not sure to which degree the performance casualties will be,
>> with frequent EPT table walk and EPT tlb flush.
>
> I don't see the concern. I am assuming XenGT would claim the type at start of day and then just to HVMOP_set_mem_type operations on PPGTT pages as necessary, which would not require p2m traversal (since that hypercall takes the gfn as an arg) and the EPT flush requirements are no different to setting the mmio_write_dm type in the current implementation.

So this looks like my second interpretation, with step 1> and step 2>
changed?

>
>>
>> If it is not, I guess we can(e.g. when trying to WP a gfn):
>> 1> use HVMOP_set_mem_type to set the HVMMEM_ioreq_serverX flag,
>> which
>> for the write protected case works the same as HVMMEM_mmio_write_dm;
>> If successful, accesses to a page of type HVMMEM_ioreq_serverX should
>> trigger the ioreq server selection path, but will be discarded.
>> 2> after HVMOP_map_mem_type_to_ioreq_server is called, all accesses to
>> this pages of type HVMMEM_ioreq_serverX would be forwarded to a
>> specified ioreq server.
>>
>> As to XenGT backend device model, we only need to use the map hypercall
>> once when trying to contruct the first shadow PPGTT, and use the unmap
>> hypercall when a VM is going to be torn down.
>>
>
> Yes, that's correct. A single claim at start of day and a single 'unclaim' at end of day. In between the current HVMOP_set_mem_type is pretty much as before (just with the new type name) and there is no longer a need for the hypercall to map the range to the ioreq server.
>
>    Paul
>

B.R.
Yu

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Yu, Zhang]

On 2/26/2016 5:18 PM, Jan Beulich wrote:
>>>> On 26.02.16 at 07:59, <yu.c.zhang@linux.intel.com> wrote:
>>> +Proposal
>>> +========
>>> +
>>> +Because the number of spare types available in the P2M type-space is
>>> +currently very limited it is proposed that HVMMEM\_mmio\_write\_dm be
>>> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if the
>>> +P2M type-space is increased, this can be renamed to HVMMEM\_ioreq\_server0
>>> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc. types
>>> +can be added.
>>> +
>>> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the same as
>>> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server. Furthermore
>>
>> Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX type,
>> its access rights in P2M still remains unchanged? So the new hypercall
>> pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also responsible for
>> the PTE updates on the access bits?
>>
>> If it is true, I'm afraid this would be time consuming, because the
>> map/unmap will have to traverse all P2M structures to detect the PTEs
>> with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
>> triggered dynamically with the construction/destruction of shadow PPGTT.
>> But I'm not sure to which degree the performance casualties will be,
>> with frequent EPT table walk and EPT tlb flush.
>
> No walking of EPT trees will be necessary in that case, just like it
> already has been made unnecessary for other changes resulting
> in various PTE attributes needing re-calculation. We'll only need
> to extend the p2m_memory_type_changed() mechanism to cover
> changes like this one.

So you mean when the access bits are to be updated, we can leverage
something like p2m_memory_type_changed(which I guess only deals with
memory types, not access bits) to avoid the walking of EPT trees? I'll
need to study this part.
Anyway, thanks for your advice. :)

B.R.
Yu

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Paul Durrant]

> -----Original Message-----
> From: Yu, Zhang [mailto:yu.c.zhang@linux.intel.com]
> Sent: 26 February 2016 09:37
> To: Jan Beulich
> Cc: Paul Durrant; xen-devel@lists.xenproject.org
> Subject: Re: [Xen-devel] [PATCH] docs/design: introduce
> HVMMEM_ioreq_serverX types
> 
> On 2/26/2016 5:18 PM, Jan Beulich wrote:
> >>>> On 26.02.16 at 07:59, <yu.c.zhang@linux.intel.com> wrote:
> >>> +Proposal
> >>> +========
> >>> +
> >>> +Because the number of spare types available in the P2M type-space is
> >>> +currently very limited it is proposed that
> HVMMEM\_mmio\_write\_dm be
> >>> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if
> the
> >>> +P2M type-space is increased, this can be renamed to
> HVMMEM\_ioreq\_server0
> >>> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc.
> types
> >>> +can be added.
> >>> +
> >>> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the
> same as
> >>> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server.
> Furthermore
> >>
> >> Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX
> type,
> >> its access rights in P2M still remains unchanged? So the new hypercall
> >> pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also
> responsible for
> >> the PTE updates on the access bits?
> >>
> >> If it is true, I'm afraid this would be time consuming, because the
> >> map/unmap will have to traverse all P2M structures to detect the PTEs
> >> with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
> >> triggered dynamically with the construction/destruction of shadow
> PPGTT.
> >> But I'm not sure to which degree the performance casualties will be,
> >> with frequent EPT table walk and EPT tlb flush.
> >
> > No walking of EPT trees will be necessary in that case, just like it
> > already has been made unnecessary for other changes resulting
> > in various PTE attributes needing re-calculation. We'll only need
> > to extend the p2m_memory_type_changed() mechanism to cover
> > changes like this one.
> 
> So you mean when the access bits are to be updated, we can leverage
> something like p2m_memory_type_changed(which I guess only deals with
> memory types, not access bits) to avoid the walking of EPT trees? I'll
> need to study this part.

No, the P2M is walked when the map/unmap hypercall is issued but, in the XenGT use-case, that hypercall is issued once at start of day and - if everything is working as it I believe it should - there won't actually be any pages of type HVMMEM_ioreq_server at that point, so no EPT flush is required.

> Anyway, thanks for your advice. :)

I will post an implementation hopefully in the next few days once I've proved it works in my XenGT rig.

  Paul

> 
> B.R.
> Yu

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] docs/design: introduce HVMMEM_ioreq_serverX types

[Yu, Zhang]

On 2/26/2016 5:50 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: Yu, Zhang [mailto:yu.c.zhang@linux.intel.com]
>> Sent: 26 February 2016 09:37
>> To: Jan Beulich
>> Cc: Paul Durrant; xen-devel@lists.xenproject.org
>> Subject: Re: [Xen-devel] [PATCH] docs/design: introduce
>> HVMMEM_ioreq_serverX types
>>
>> On 2/26/2016 5:18 PM, Jan Beulich wrote:
>>>>>> On 26.02.16 at 07:59, <yu.c.zhang@linux.intel.com> wrote:
>>>>> +Proposal
>>>>> +========
>>>>> +
>>>>> +Because the number of spare types available in the P2M type-space is
>>>>> +currently very limited it is proposed that
>> HVMMEM\_mmio\_write\_dm be
>>>>> +replaced by a single new type HVMMEM\_ioreq\_server. In future, if
>> the
>>>>> +P2M type-space is increased, this can be renamed to
>> HVMMEM\_ioreq\_server0
>>>>> +and new HVMMEM\_ioreq\_server1, HVMMEM\_ioreq\_server2, etc.
>> types
>>>>> +can be added.
>>>>> +
>>>>> +Accesses to a page of type HVMMEM\_ioreq\_serverX should be the
>> same as
>>>>> +HVMMEM\_ram\_rw until the type is _claimed_ by an IOREQ server.
>> Furthermore
>>>>
>>>> Sorry, do you mean even when a gfn is set to HVMMEM_ioreq_serverX
>> type,
>>>> its access rights in P2M still remains unchanged? So the new hypercall
>>>> pair, HVMOP_[un]map_mem_type_to_ioreq_server, are also
>> responsible for
>>>> the PTE updates on the access bits?
>>>>
>>>> If it is true, I'm afraid this would be time consuming, because the
>>>> map/unmap will have to traverse all P2M structures to detect the PTEs
>>>> with HVMMEM_ioreq_serverX flag set. Yet in XenGT, setting this flag is
>>>> triggered dynamically with the construction/destruction of shadow
>> PPGTT.
>>>> But I'm not sure to which degree the performance casualties will be,
>>>> with frequent EPT table walk and EPT tlb flush.
>>>
>>> No walking of EPT trees will be necessary in that case, just like it
>>> already has been made unnecessary for other changes resulting
>>> in various PTE attributes needing re-calculation. We'll only need
>>> to extend the p2m_memory_type_changed() mechanism to cover
>>> changes like this one.
>>
>> So you mean when the access bits are to be updated, we can leverage
>> something like p2m_memory_type_changed(which I guess only deals with
>> memory types, not access bits) to avoid the walking of EPT trees? I'll
>> need to study this part.
>
> No, the P2M is walked when the map/unmap hypercall is issued but, in the XenGT use-case, that hypercall is issued once at start of day and - if everything is working as it I believe it should - there won't actually be any pages of type HVMMEM_ioreq_server at that point, so no EPT flush is required.
>
>> Anyway, thanks for your advice. :)
>
> I will post an implementation hopefully in the next few days once I've proved it works in my XenGT rig.

Great. Looking forward to this implementation, and thanks for your
help. :)

B.R.
Yu


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel