* [PATCH v2 01/21] xen/common: do not implicitly permit access to mapped I/O memory
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-07-31 15:22 ` Julien Grall
2014-07-31 15:00 ` [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest Julien Grall
` (20 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: Ian Campbell, Paolo Valente, Keir Fraser, ian.campbell,
Stefano Stabellini, Ian Jackson, Dario Faggioli, tim,
Julien Grall, Eric Trudeau, Andrew Cooper, stefano.stabellini,
Jan Beulich, Arianna Avanzini, Viktor Kleinik
From: Arianna Avanzini <avanzini.arianna@gmail.com>
Currently, the XEN_DOMCTL_memory_mapping hypercall implicitly grants
to a domain access permission to the I/O memory areas mapped in its
guest address space. This conflicts with the presence of a specific
hypercall (XEN_DOMCTL_iomem_permission) used to grant such a permission
to a domain.
This commit separates the functions of the two hypercalls by having only
the latter be able to permit I/O memory access to a domain, and the
former just performing the mapping after a permissions check on both the
granting and the grantee domains.
Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Paolo Valente <paolo.valente@unimore.it>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Julien Grall <julien.grall@citrix.com>
Cc: Ian Campbell <Ian.Campbell@eu.citrix.com>
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
Cc: Tim Deegan <tim@xen.org>
Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Eric Trudeau <etrudeau@broadcom.com>
Cc: Viktor Kleinik <viktor.kleinik@globallogic.com>
---
xen/common/domctl.c | 36 ++++++++++--------------------------
1 file changed, 10 insertions(+), 26 deletions(-)
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 80b7800..04ecd53 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -917,7 +917,8 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
break;
ret = -EPERM;
- if ( !iomem_access_permitted(current->domain, mfn, mfn_end) )
+ if ( !iomem_access_permitted(current->domain, mfn, mfn_end) ||
+ !iomem_access_permitted(d, mfn, mfn_end) )
break;
ret = xsm_iomem_mapping(XSM_HOOK, d, mfn, mfn_end, add);
@@ -930,40 +931,23 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
"memory_map:add: dom%d gfn=%lx mfn=%lx nr=%lx\n",
d->domain_id, gfn, mfn, nr_mfns);
- ret = iomem_permit_access(d, mfn, mfn_end);
- if ( !ret )
- {
- ret = map_mmio_regions(d, gfn, nr_mfns, mfn);
- if ( ret )
- {
- printk(XENLOG_G_WARNING
- "memory_map:fail: dom%d gfn=%lx mfn=%lx nr=%lx ret:%ld\n",
- d->domain_id, gfn, mfn, nr_mfns, ret);
- if ( iomem_deny_access(d, mfn, mfn_end) &&
- is_hardware_domain(current->domain) )
- printk(XENLOG_ERR
- "memory_map: failed to deny dom%d access to [%lx,%lx]\n",
- d->domain_id, mfn, mfn_end);
- }
- }
+ ret = map_mmio_regions(d, gfn, nr_mfns, mfn);
+ if ( ret )
+ printk(XENLOG_G_WARNING
+ "memory_map:fail: dom%d gfn=%lx mfn=%lx nr=%lx ret:%ld\n",
+ d->domain_id, gfn, mfn, nr_mfns, ret);
}
else
{
- int rc = 0;
-
printk(XENLOG_G_INFO
"memory_map:remove: dom%d gfn=%lx mfn=%lx nr=%lx\n",
d->domain_id, gfn, mfn, nr_mfns);
- rc = unmap_mmio_regions(d, gfn, nr_mfns, mfn);
- ret = iomem_deny_access(d, mfn, mfn_end);
- if ( !ret )
- ret = rc;
+ ret = unmap_mmio_regions(d, gfn, nr_mfns, mfn);
if ( ret && is_hardware_domain(current->domain) )
printk(XENLOG_ERR
- "memory_map: error %ld %s dom%d access to [%lx,%lx]\n",
- ret, rc ? "removing" : "denying", d->domain_id,
- mfn, mfn_end);
+ "memory_map: error %ld removing dom%d access to [%lx,%lx]\n",
+ ret, d->domain_id, mfn, mfn_end);
}
/* Do this unconditionally to cover errors on above failure paths. */
memory_type_changed(d);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 01/21] xen/common: do not implicitly permit access to mapped I/O memory
2014-07-31 15:00 ` [PATCH v2 01/21] xen/common: do not implicitly permit access to mapped I/O memory Julien Grall
@ 2014-07-31 15:22 ` Julien Grall
0 siblings, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:22 UTC (permalink / raw)
To: xen-devel
Cc: Ian Campbell, Paolo Valente, Keir Fraser, ian.campbell,
Stefano Stabellini, Ian Jackson, Dario Faggioli, tim,
Julien Grall, Eric Trudeau, Andrew Cooper, stefano.stabellini,
Jan Beulich, Arianna Avanzini, Viktor Kleinik
Hi all,
Please forgot this patch. I've added it by mistake in my series.
Regards,
On 07/31/2014 04:00 PM, Julien Grall wrote:
> From: Arianna Avanzini <avanzini.arianna@gmail.com>
>
> Currently, the XEN_DOMCTL_memory_mapping hypercall implicitly grants
> to a domain access permission to the I/O memory areas mapped in its
> guest address space. This conflicts with the presence of a specific
> hypercall (XEN_DOMCTL_iomem_permission) used to grant such a permission
> to a domain.
> This commit separates the functions of the two hypercalls by having only
> the latter be able to permit I/O memory access to a domain, and the
> former just performing the mapping after a permissions check on both the
> granting and the grantee domains.
>
> Signed-off-by: Arianna Avanzini <avanzini.arianna@gmail.com>
> Cc: Dario Faggioli <dario.faggioli@citrix.com>
> Cc: Paolo Valente <paolo.valente@unimore.it>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Julien Grall <julien.grall@citrix.com>
> Cc: Ian Campbell <Ian.Campbell@eu.citrix.com>
> Cc: Jan Beulich <JBeulich@suse.com>
> Cc: Keir Fraser <keir@xen.org>
> Cc: Tim Deegan <tim@xen.org>
> Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: Eric Trudeau <etrudeau@broadcom.com>
> Cc: Viktor Kleinik <viktor.kleinik@globallogic.com>
> ---
> xen/common/domctl.c | 36 ++++++++++--------------------------
> 1 file changed, 10 insertions(+), 26 deletions(-)
>
> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> index 80b7800..04ecd53 100644
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -917,7 +917,8 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> break;
>
> ret = -EPERM;
> - if ( !iomem_access_permitted(current->domain, mfn, mfn_end) )
> + if ( !iomem_access_permitted(current->domain, mfn, mfn_end) ||
> + !iomem_access_permitted(d, mfn, mfn_end) )
> break;
>
> ret = xsm_iomem_mapping(XSM_HOOK, d, mfn, mfn_end, add);
> @@ -930,40 +931,23 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> "memory_map:add: dom%d gfn=%lx mfn=%lx nr=%lx\n",
> d->domain_id, gfn, mfn, nr_mfns);
>
> - ret = iomem_permit_access(d, mfn, mfn_end);
> - if ( !ret )
> - {
> - ret = map_mmio_regions(d, gfn, nr_mfns, mfn);
> - if ( ret )
> - {
> - printk(XENLOG_G_WARNING
> - "memory_map:fail: dom%d gfn=%lx mfn=%lx nr=%lx ret:%ld\n",
> - d->domain_id, gfn, mfn, nr_mfns, ret);
> - if ( iomem_deny_access(d, mfn, mfn_end) &&
> - is_hardware_domain(current->domain) )
> - printk(XENLOG_ERR
> - "memory_map: failed to deny dom%d access to [%lx,%lx]\n",
> - d->domain_id, mfn, mfn_end);
> - }
> - }
> + ret = map_mmio_regions(d, gfn, nr_mfns, mfn);
> + if ( ret )
> + printk(XENLOG_G_WARNING
> + "memory_map:fail: dom%d gfn=%lx mfn=%lx nr=%lx ret:%ld\n",
> + d->domain_id, gfn, mfn, nr_mfns, ret);
> }
> else
> {
> - int rc = 0;
> -
> printk(XENLOG_G_INFO
> "memory_map:remove: dom%d gfn=%lx mfn=%lx nr=%lx\n",
> d->domain_id, gfn, mfn, nr_mfns);
>
> - rc = unmap_mmio_regions(d, gfn, nr_mfns, mfn);
> - ret = iomem_deny_access(d, mfn, mfn_end);
> - if ( !ret )
> - ret = rc;
> + ret = unmap_mmio_regions(d, gfn, nr_mfns, mfn);
> if ( ret && is_hardware_domain(current->domain) )
> printk(XENLOG_ERR
> - "memory_map: error %ld %s dom%d access to [%lx,%lx]\n",
> - ret, rc ? "removing" : "denying", d->domain_id,
> - mfn, mfn_end);
> + "memory_map: error %ld removing dom%d access to [%lx,%lx]\n",
> + ret, d->domain_id, mfn, mfn_end);
> }
> /* Do this unconditionally to cover errors on above failure paths. */
> memory_type_changed(d);
>
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
2014-07-31 15:00 ` [PATCH v2 01/21] xen/common: do not implicitly permit access to mapped I/O memory Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-01 8:40 ` Jan Beulich
` (2 more replies)
2014-07-31 15:00 ` [PATCH v2 03/21] xen/arm: vgic: Rename nr_lines into nr_spis Julien Grall
` (19 subsequent siblings)
21 siblings, 3 replies; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: Keir Fraser, ian.campbell, tim, Julien Grall, Ian Jackson,
stefano.stabellini, Jan Beulich, Daniel De Graaf
Flask code already provides an helper to copy a string from guest. In a later
patch, the new DT hypercalls will need a similar function.
To avoid code duplication, copy the flask helper (flask_copying_string) to
common code:
- Rename into safe_copy_string_from_guest
- Update arguments. The size provided by the hypercall is unsigned
not signed
- Add comment to explain the extra +1
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Keir Fraser <keir@xen.org>
Cc: Tim Deegan <tim@xen.org>
---
Changes in v2:
- Rename copy_string_from_guest into safe_copy_string_from_guest
- Update commit message and comment in the code
---
xen/common/Makefile | 1 +
xen/common/guestcopy.c | 29 +++++++++++++++++++++++++++++
xen/include/xen/guest_access.h | 5 +++++
xen/xsm/flask/flask_op.c | 29 +++--------------------------
4 files changed, 38 insertions(+), 26 deletions(-)
create mode 100644 xen/common/guestcopy.c
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 3683ae3..627b6e3 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -9,6 +9,7 @@ obj-y += event_2l.o
obj-y += event_channel.o
obj-y += event_fifo.o
obj-y += grant_table.o
+obj-y += guestcopy.o
obj-y += irq.o
obj-y += kernel.o
obj-y += keyhandler.o
diff --git a/xen/common/guestcopy.c b/xen/common/guestcopy.c
new file mode 100644
index 0000000..c01555c
--- /dev/null
+++ b/xen/common/guestcopy.c
@@ -0,0 +1,29 @@
+#include <xen/config.h>
+#include <xen/lib.h>
+#include <xen/guest_access.h>
+
+/* The function copies a string from the guest and add a NUL-terminated to
+ * make sure the string is correctly finished. */
+int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
+ unsigned long size, unsigned long max_size)
+{
+ char *tmp;
+
+ if ( size > max_size )
+ return -ENOENT;
+
+ /* Add an extra +1 to append \0 */
+ tmp = xmalloc_array(char, size + 1);
+ if ( !tmp )
+ return -ENOMEM;
+
+ if ( copy_from_guest(tmp, u_buf, size) )
+ {
+ xfree(tmp);
+ return -EFAULT;
+ }
+ tmp[size] = 0;
+
+ *buf = tmp;
+ return 0;
+}
diff --git a/xen/include/xen/guest_access.h b/xen/include/xen/guest_access.h
index 373454e..4c669ad 100644
--- a/xen/include/xen/guest_access.h
+++ b/xen/include/xen/guest_access.h
@@ -8,6 +8,8 @@
#define __XEN_GUEST_ACCESS_H__
#include <asm/guest_access.h>
+#include <xen/types.h>
+#include <public/xen.h>
#define copy_to_guest(hnd, ptr, nr) \
copy_to_guest_offset(hnd, 0, ptr, nr)
@@ -27,4 +29,7 @@
#define __clear_guest(hnd, nr) \
__clear_guest_offset(hnd, 0, nr)
+int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
+ unsigned long size, unsigned long max_size);
+
#endif /* __XEN_GUEST_ACCESS_H__ */
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index 7743aac..840a635 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -76,29 +76,6 @@ static int domain_has_security(struct domain *d, u32 perms)
perms, NULL);
}
-static int flask_copyin_string(XEN_GUEST_HANDLE(char) u_buf, char **buf,
- size_t size, size_t max_size)
-{
- char *tmp;
-
- if ( size > max_size )
- return -ENOENT;
-
- tmp = xmalloc_array(char, size + 1);
- if ( !tmp )
- return -ENOMEM;
-
- if ( copy_from_guest(tmp, u_buf, size) )
- {
- xfree(tmp);
- return -EFAULT;
- }
- tmp[size] = 0;
-
- *buf = tmp;
- return 0;
-}
-
#endif /* COMPAT */
static int flask_security_user(struct xen_flask_userlist *arg)
@@ -112,7 +89,7 @@ static int flask_security_user(struct xen_flask_userlist *arg)
if ( rv )
return rv;
- rv = flask_copyin_string(arg->u.user, &user, arg->size, PAGE_SIZE);
+ rv = safe_copy_string_from_guest(arg->u.user, &user, arg->size, PAGE_SIZE);
if ( rv )
return rv;
@@ -227,7 +204,7 @@ static int flask_security_context(struct xen_flask_sid_context *arg)
if ( rv )
return rv;
- rv = flask_copyin_string(arg->context, &buf, arg->size, PAGE_SIZE);
+ rv = safe_copy_string_from_guest(arg->context, &buf, arg->size, PAGE_SIZE);
if ( rv )
return rv;
@@ -324,7 +301,7 @@ static int flask_security_resolve_bool(struct xen_flask_boolean *arg)
if ( arg->bool_id != -1 )
return 0;
- rv = flask_copyin_string(arg->name, &name, arg->size, bool_maxstr);
+ rv = safe_copy_string_from_guest(arg->name, &name, arg->size, bool_maxstr);
if ( rv )
return rv;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-07-31 15:00 ` [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest Julien Grall
@ 2014-08-01 8:40 ` Jan Beulich
2014-08-06 14:18 ` Julien Grall
2014-09-09 12:52 ` Ian Campbell
2014-08-06 13:56 ` Stefano Stabellini
2014-08-06 16:06 ` Daniel De Graaf
2 siblings, 2 replies; 119+ messages in thread
From: Jan Beulich @ 2014-08-01 8:40 UTC (permalink / raw)
To: Julien Grall
Cc: Keir Fraser, ian.campbell, tim, Ian Jackson, stefano.stabellini,
xen-devel, Daniel De Graaf
>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> --- /dev/null
> +++ b/xen/common/guestcopy.c
> @@ -0,0 +1,29 @@
> +#include <xen/config.h>
> +#include <xen/lib.h>
> +#include <xen/guest_access.h>
> +
> +/* The function copies a string from the guest and add a NUL-terminated to
> + * make sure the string is correctly finished. */
"... and adds a NUL-terminator ...". Also not sure about "finished"
here. And please fix the comment style.
> +int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
> + unsigned long size, unsigned long max_size)
size_t please.
Further I think this function would benefit from using the definitions
in xen/err.h, eliminating the need to return the buffer pointer via
indirection.
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-08-01 8:40 ` Jan Beulich
@ 2014-08-06 14:18 ` Julien Grall
2014-09-09 12:52 ` Ian Campbell
1 sibling, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-08-06 14:18 UTC (permalink / raw)
To: Jan Beulich
Cc: Keir Fraser, ian.campbell, tim, Ian Jackson, stefano.stabellini,
xen-devel, Daniel De Graaf
Hi Jan,
On 08/01/2014 09:40 AM, Jan Beulich wrote:
>>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
>> --- /dev/null
>> +++ b/xen/common/guestcopy.c
>> @@ -0,0 +1,29 @@
>> +#include <xen/config.h>
>> +#include <xen/lib.h>
>> +#include <xen/guest_access.h>
>> +
>> +/* The function copies a string from the guest and add a NUL-terminated to
>> + * make sure the string is correctly finished. */
>
> "... and adds a NUL-terminator ...". Also not sure about "finished"
> here. And please fix the comment style.
Will fix it in the next version.
>> +int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
>> + unsigned long size, unsigned long max_size)
>
> size_t please.
Will do.
> Further I think this function would benefit from using the definitions
> in xen/err.h, eliminating the need to return the buffer pointer via
> indirection.
Good idea. I will use this solution in the next version.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-08-01 8:40 ` Jan Beulich
2014-08-06 14:18 ` Julien Grall
@ 2014-09-09 12:52 ` Ian Campbell
2014-09-09 13:17 ` Jan Beulich
1 sibling, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 12:52 UTC (permalink / raw)
To: Jan Beulich
Cc: Keir Fraser, tim, Julien Grall, Ian Jackson, stefano.stabellini,
xen-devel, Daniel De Graaf
On Fri, 2014-08-01 at 09:40 +0100, Jan Beulich wrote:
> >>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> > --- /dev/null
> > +++ b/xen/common/guestcopy.c
> > @@ -0,0 +1,29 @@
> > +#include <xen/config.h>
> > +#include <xen/lib.h>
> > +#include <xen/guest_access.h>
> > +
> > +/* The function copies a string from the guest and add a NUL-terminated to
> > + * make sure the string is correctly finished. */
>
> "... and adds a NUL-terminator ...". Also not sure about "finished"
> here.
It does sound odd. "correctly terminated" would be more usual, but given
the context is repetitive. Perhaps: "and add a NULL to make sure that
the string is correctly terminated"?
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-09-09 12:52 ` Ian Campbell
@ 2014-09-09 13:17 ` Jan Beulich
2014-09-09 13:40 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Jan Beulich @ 2014-09-09 13:17 UTC (permalink / raw)
To: Ian Campbell
Cc: Keir Fraser, tim, Julien Grall, Ian Jackson, stefano.stabellini,
xen-devel, Daniel De Graaf
>>> On 09.09.14 at 14:52, <Ian.Campbell@citrix.com> wrote:
> On Fri, 2014-08-01 at 09:40 +0100, Jan Beulich wrote:
>> >>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
>> > --- /dev/null
>> > +++ b/xen/common/guestcopy.c
>> > @@ -0,0 +1,29 @@
>> > +#include <xen/config.h>
>> > +#include <xen/lib.h>
>> > +#include <xen/guest_access.h>
>> > +
>> > +/* The function copies a string from the guest and add a NUL-terminated to
>> > + * make sure the string is correctly finished. */
>>
>> "... and adds a NUL-terminator ...". Also not sure about "finished"
>> here.
>
> It does sound odd. "correctly terminated" would be more usual, but given
> the context is repetitive. Perhaps: "and add a NULL to make sure that
> the string is correctly terminated"?
With s/add/adds/, maybe. Albeit personally I dislike using NULL for
other than pointers - I'd prefer NUL or NIL for the purpose here.
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-09-09 13:17 ` Jan Beulich
@ 2014-09-09 13:40 ` Ian Campbell
0 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:40 UTC (permalink / raw)
To: Jan Beulich
Cc: Keir Fraser, tim, Julien Grall, Ian Jackson, stefano.stabellini,
xen-devel, Daniel De Graaf
On Tue, 2014-09-09 at 14:17 +0100, Jan Beulich wrote:
> >>> On 09.09.14 at 14:52, <Ian.Campbell@citrix.com> wrote:
> > On Fri, 2014-08-01 at 09:40 +0100, Jan Beulich wrote:
> >> >>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> >> > --- /dev/null
> >> > +++ b/xen/common/guestcopy.c
> >> > @@ -0,0 +1,29 @@
> >> > +#include <xen/config.h>
> >> > +#include <xen/lib.h>
> >> > +#include <xen/guest_access.h>
> >> > +
> >> > +/* The function copies a string from the guest and add a NUL-terminated to
> >> > + * make sure the string is correctly finished. */
> >>
> >> "... and adds a NUL-terminator ...". Also not sure about "finished"
> >> here.
> >
> > It does sound odd. "correctly terminated" would be more usual, but given
> > the context is repetitive. Perhaps: "and add a NULL to make sure that
> > the string is correctly terminated"?
>
> With s/add/adds/, maybe.
Yes.
> Albeit personally I dislike using NULL for
> other than pointers - I'd prefer NUL or NIL for the purpose here.
Me too, that was a typo, sorry!
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-07-31 15:00 ` [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest Julien Grall
2014-08-01 8:40 ` Jan Beulich
@ 2014-08-06 13:56 ` Stefano Stabellini
2014-08-06 14:22 ` Julien Grall
2014-08-06 16:06 ` Daniel De Graaf
2 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 13:56 UTC (permalink / raw)
To: Julien Grall
Cc: Keir Fraser, ian.campbell, tim, Ian Jackson, stefano.stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
On Thu, 31 Jul 2014, Julien Grall wrote:
> Flask code already provides an helper to copy a string from guest. In a later
> patch, the new DT hypercalls will need a similar function.
>
> To avoid code duplication, copy the flask helper (flask_copying_string) to
> common code:
> - Rename into safe_copy_string_from_guest
> - Update arguments. The size provided by the hypercall is unsigned
> not signed
> - Add comment to explain the extra +1
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Keir Fraser <keir@xen.org>
> Cc: Tim Deegan <tim@xen.org>
>
> ---
> Changes in v2:
> - Rename copy_string_from_guest into safe_copy_string_from_guest
> - Update commit message and comment in the code
> ---
> xen/common/Makefile | 1 +
> xen/common/guestcopy.c | 29 +++++++++++++++++++++++++++++
> xen/include/xen/guest_access.h | 5 +++++
> xen/xsm/flask/flask_op.c | 29 +++--------------------------
> 4 files changed, 38 insertions(+), 26 deletions(-)
> create mode 100644 xen/common/guestcopy.c
>
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 3683ae3..627b6e3 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -9,6 +9,7 @@ obj-y += event_2l.o
> obj-y += event_channel.o
> obj-y += event_fifo.o
> obj-y += grant_table.o
> +obj-y += guestcopy.o
> obj-y += irq.o
> obj-y += kernel.o
> obj-y += keyhandler.o
> diff --git a/xen/common/guestcopy.c b/xen/common/guestcopy.c
> new file mode 100644
> index 0000000..c01555c
> --- /dev/null
> +++ b/xen/common/guestcopy.c
> @@ -0,0 +1,29 @@
> +#include <xen/config.h>
> +#include <xen/lib.h>
> +#include <xen/guest_access.h>
> +
> +/* The function copies a string from the guest and add a NUL-terminated to
> + * make sure the string is correctly finished. */
> +int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
> + unsigned long size, unsigned long max_size)
Why did you change the size type from size_t to unsigned long?
If you really need to move away from size_t, wouldn't it be better to
still be using an explicitly sized type, such as uint32_t or uint64_t?
> +{
> + char *tmp;
> +
> + if ( size > max_size )
> + return -ENOENT;
> +
> + /* Add an extra +1 to append \0 */
> + tmp = xmalloc_array(char, size + 1);
> + if ( !tmp )
> + return -ENOMEM;
> +
> + if ( copy_from_guest(tmp, u_buf, size) )
> + {
> + xfree(tmp);
> + return -EFAULT;
> + }
> + tmp[size] = 0;
> +
> + *buf = tmp;
> + return 0;
> +}
> diff --git a/xen/include/xen/guest_access.h b/xen/include/xen/guest_access.h
> index 373454e..4c669ad 100644
> --- a/xen/include/xen/guest_access.h
> +++ b/xen/include/xen/guest_access.h
> @@ -8,6 +8,8 @@
> #define __XEN_GUEST_ACCESS_H__
>
> #include <asm/guest_access.h>
> +#include <xen/types.h>
> +#include <public/xen.h>
>
> #define copy_to_guest(hnd, ptr, nr) \
> copy_to_guest_offset(hnd, 0, ptr, nr)
> @@ -27,4 +29,7 @@
> #define __clear_guest(hnd, nr) \
> __clear_guest_offset(hnd, 0, nr)
>
> +int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
> + unsigned long size, unsigned long max_size);
> +
> #endif /* __XEN_GUEST_ACCESS_H__ */
> diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
> index 7743aac..840a635 100644
> --- a/xen/xsm/flask/flask_op.c
> +++ b/xen/xsm/flask/flask_op.c
> @@ -76,29 +76,6 @@ static int domain_has_security(struct domain *d, u32 perms)
> perms, NULL);
> }
>
> -static int flask_copyin_string(XEN_GUEST_HANDLE(char) u_buf, char **buf,
> - size_t size, size_t max_size)
> -{
> - char *tmp;
> -
> - if ( size > max_size )
> - return -ENOENT;
> -
> - tmp = xmalloc_array(char, size + 1);
> - if ( !tmp )
> - return -ENOMEM;
> -
> - if ( copy_from_guest(tmp, u_buf, size) )
> - {
> - xfree(tmp);
> - return -EFAULT;
> - }
> - tmp[size] = 0;
> -
> - *buf = tmp;
> - return 0;
> -}
> -
> #endif /* COMPAT */
>
> static int flask_security_user(struct xen_flask_userlist *arg)
> @@ -112,7 +89,7 @@ static int flask_security_user(struct xen_flask_userlist *arg)
> if ( rv )
> return rv;
>
> - rv = flask_copyin_string(arg->u.user, &user, arg->size, PAGE_SIZE);
> + rv = safe_copy_string_from_guest(arg->u.user, &user, arg->size, PAGE_SIZE);
> if ( rv )
> return rv;
>
> @@ -227,7 +204,7 @@ static int flask_security_context(struct xen_flask_sid_context *arg)
> if ( rv )
> return rv;
>
> - rv = flask_copyin_string(arg->context, &buf, arg->size, PAGE_SIZE);
> + rv = safe_copy_string_from_guest(arg->context, &buf, arg->size, PAGE_SIZE);
> if ( rv )
> return rv;
>
> @@ -324,7 +301,7 @@ static int flask_security_resolve_bool(struct xen_flask_boolean *arg)
> if ( arg->bool_id != -1 )
> return 0;
>
> - rv = flask_copyin_string(arg->name, &name, arg->size, bool_maxstr);
> + rv = safe_copy_string_from_guest(arg->name, &name, arg->size, bool_maxstr);
> if ( rv )
> return rv;
>
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-08-06 13:56 ` Stefano Stabellini
@ 2014-08-06 14:22 ` Julien Grall
0 siblings, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-08-06 14:22 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Keir Fraser, ian.campbell, tim, Ian Jackson, stefano.stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
Hi Stefano,
On 08/06/2014 02:56 PM, Stefano Stabellini wrote:
>> diff --git a/xen/common/guestcopy.c b/xen/common/guestcopy.c
>> new file mode 100644
>> index 0000000..c01555c
>> --- /dev/null
>> +++ b/xen/common/guestcopy.c
>> @@ -0,0 +1,29 @@
>> +#include <xen/config.h>
>> +#include <xen/lib.h>
>> +#include <xen/guest_access.h>
>> +
>> +/* The function copies a string from the guest and add a NUL-terminated to
>> + * make sure the string is correctly finished. */
>> +int safe_copy_string_from_guest(XEN_GUEST_HANDLE(char) u_buf, char **buf,
>> + unsigned long size, unsigned long max_size)
>
> Why did you change the size type from size_t to unsigned long?
I don't remember why. I will reuse size_t here.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest
2014-07-31 15:00 ` [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest Julien Grall
2014-08-01 8:40 ` Jan Beulich
2014-08-06 13:56 ` Stefano Stabellini
@ 2014-08-06 16:06 ` Daniel De Graaf
2 siblings, 0 replies; 119+ messages in thread
From: Daniel De Graaf @ 2014-08-06 16:06 UTC (permalink / raw)
To: Julien Grall, xen-devel
Cc: Keir Fraser, ian.campbell, tim, Ian Jackson, stefano.stabellini,
Jan Beulich
On 07/31/2014 11:00 AM, Julien Grall wrote:
> Flask code already provides an helper to copy a string from guest. In a later
> patch, the new DT hypercalls will need a similar function.
>
> To avoid code duplication, copy the flask helper (flask_copying_string) to
> common code:
> - Rename into safe_copy_string_from_guest
> - Update arguments. The size provided by the hypercall is unsigned
> not signed
> - Add comment to explain the extra +1
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Keir Fraser <keir@xen.org>
> Cc: Tim Deegan <tim@xen.org>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 03/21] xen/arm: vgic: Rename nr_lines into nr_spis
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
2014-07-31 15:00 ` [PATCH v2 01/21] xen/common: do not implicitly permit access to mapped I/O memory Julien Grall
2014-07-31 15:00 ` [PATCH v2 02/21] xen: guestcopy: Provide an helper to safely copy string from guest Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 13:58 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq Julien Grall
` (18 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
The field nr_lines in the arch_domain vgic structure contains the number of
SPIs for the emulated GIC. Using the nr_lines make confusion with the GIC
code, where it means the number of IRQs. This can lead to coding error.
Also introduce vgic_nr_lines to get the number of IRQ handled by the emulated
GIC.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
This patch may be upstream via Stefano's interrupt series.
Changes in v2:
- Patch added.
---
xen/arch/arm/gic-v2.c | 2 --
xen/arch/arm/vgic-v2.c | 2 +-
xen/arch/arm/vgic.c | 15 ++++++---------
xen/include/asm-arm/domain.h | 2 +-
xen/include/asm-arm/vgic.h | 4 +++-
5 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 78ad4de..303a26a 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -431,8 +431,6 @@ static int gicv2v_setup(struct domain *d)
d->arch.vgic.cbase = GUEST_GICC_BASE;
}
- d->arch.vgic.nr_lines = 0;
-
/*
* Map the gic virtual cpu interface in the gic cpu interface
* region of the guest.
diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
index 8b21a13..14f52ed 100644
--- a/xen/arch/arm/vgic-v2.c
+++ b/xen/arch/arm/vgic-v2.c
@@ -54,7 +54,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
/* No secure world support for guests. */
vgic_lock(v);
*r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
- |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
+ |( ((v->domain->arch.vgic.nr_spis / 32)) & GICD_TYPE_LINES );
vgic_unlock(v);
return 1;
case GICD_IIDR:
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index e571ae0..ac34437 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -59,13 +59,10 @@ int domain_vgic_init(struct domain *d)
d->arch.vgic.ctlr = 0;
- /* Currently nr_lines in vgic and gic doesn't have the same meanings
- * Here nr_lines = number of SPIs
- */
if ( is_hardware_domain(d) )
- d->arch.vgic.nr_lines = gic_number_lines() - 32;
+ d->arch.vgic.nr_spis = gic_number_lines() - 32;
else
- d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
+ d->arch.vgic.nr_spis = 0; /* We don't need SPIs for the guest */
switch ( gic_hw_version() )
{
@@ -83,14 +80,14 @@ int domain_vgic_init(struct domain *d)
return -ENOMEM;
d->arch.vgic.pending_irqs =
- xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
+ xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
if ( d->arch.vgic.pending_irqs == NULL )
{
xfree(d->arch.vgic.shared_irqs);
return -ENOMEM;
}
- for (i=0; i<d->arch.vgic.nr_lines; i++)
+ for (i=0; i<d->arch.vgic.nr_spis; i++)
{
INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
@@ -213,7 +210,7 @@ void arch_move_irqs(struct vcpu *v)
struct vcpu *v_target;
int i;
- for ( i = 32; i < d->arch.vgic.nr_lines; i++ )
+ for ( i = 32; i < d->arch.vgic.nr_spis; i++ )
{
v_target = vgic_get_target_vcpu(v, i);
p = irq_to_pending(v_target, i);
@@ -339,7 +336,7 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
{
struct pending_irq *n;
- /* Pending irqs allocation strategy: the first vgic.nr_lines irqs
+ /* Pending irqs allocation strategy: the first vgic.nr_spis irqs
* are used for SPIs; the rests are used for per cpu irqs */
if ( irq < 32 )
n = &v->arch.vgic.pending_irqs[irq];
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 32d0554..5719fe5 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -89,7 +89,7 @@ struct arch_domain
*/
spinlock_t lock;
int ctlr;
- int nr_lines; /* Number of SPIs */
+ int nr_spis; /* Number of SPIs */
struct vgic_irq_rank *shared_irqs;
/*
* SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 338ba03..5ddc681 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -102,7 +102,7 @@ struct vgic_ops {
};
/* Number of ranks of interrupt registers for a domain */
-#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
+#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
#define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
#define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
@@ -156,6 +156,8 @@ enum gic_sgi_mode;
*/
#define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
+#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
+
extern int domain_vgic_init(struct domain *d);
extern void domain_vgic_free(struct domain *d);
extern int vcpu_vgic_init(struct vcpu *v);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 03/21] xen/arm: vgic: Rename nr_lines into nr_spis
2014-07-31 15:00 ` [PATCH v2 03/21] xen/arm: vgic: Rename nr_lines into nr_spis Julien Grall
@ 2014-08-06 13:58 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 13:58 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> The field nr_lines in the arch_domain vgic structure contains the number of
> SPIs for the emulated GIC. Using the nr_lines make confusion with the GIC
> code, where it means the number of IRQs. This can lead to coding error.
>
> Also introduce vgic_nr_lines to get the number of IRQ handled by the emulated
> GIC.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
> This patch may be upstream via Stefano's interrupt series.
>
> Changes in v2:
> - Patch added.
> ---
> xen/arch/arm/gic-v2.c | 2 --
> xen/arch/arm/vgic-v2.c | 2 +-
> xen/arch/arm/vgic.c | 15 ++++++---------
> xen/include/asm-arm/domain.h | 2 +-
> xen/include/asm-arm/vgic.h | 4 +++-
> 5 files changed, 11 insertions(+), 14 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
> index 78ad4de..303a26a 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -431,8 +431,6 @@ static int gicv2v_setup(struct domain *d)
> d->arch.vgic.cbase = GUEST_GICC_BASE;
> }
>
> - d->arch.vgic.nr_lines = 0;
> -
> /*
> * Map the gic virtual cpu interface in the gic cpu interface
> * region of the guest.
> diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c
> index 8b21a13..14f52ed 100644
> --- a/xen/arch/arm/vgic-v2.c
> +++ b/xen/arch/arm/vgic-v2.c
> @@ -54,7 +54,7 @@ static int vgic_v2_distr_mmio_read(struct vcpu *v, mmio_info_t *info)
> /* No secure world support for guests. */
> vgic_lock(v);
> *r = ( (v->domain->max_vcpus << 5) & GICD_TYPE_CPUS )
> - |( ((v->domain->arch.vgic.nr_lines / 32)) & GICD_TYPE_LINES );
> + |( ((v->domain->arch.vgic.nr_spis / 32)) & GICD_TYPE_LINES );
> vgic_unlock(v);
> return 1;
> case GICD_IIDR:
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index e571ae0..ac34437 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -59,13 +59,10 @@ int domain_vgic_init(struct domain *d)
>
> d->arch.vgic.ctlr = 0;
>
> - /* Currently nr_lines in vgic and gic doesn't have the same meanings
> - * Here nr_lines = number of SPIs
> - */
> if ( is_hardware_domain(d) )
> - d->arch.vgic.nr_lines = gic_number_lines() - 32;
> + d->arch.vgic.nr_spis = gic_number_lines() - 32;
> else
> - d->arch.vgic.nr_lines = 0; /* We don't need SPIs for the guest */
> + d->arch.vgic.nr_spis = 0; /* We don't need SPIs for the guest */
>
> switch ( gic_hw_version() )
> {
> @@ -83,14 +80,14 @@ int domain_vgic_init(struct domain *d)
> return -ENOMEM;
>
> d->arch.vgic.pending_irqs =
> - xzalloc_array(struct pending_irq, d->arch.vgic.nr_lines);
> + xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
> if ( d->arch.vgic.pending_irqs == NULL )
> {
> xfree(d->arch.vgic.shared_irqs);
> return -ENOMEM;
> }
>
> - for (i=0; i<d->arch.vgic.nr_lines; i++)
> + for (i=0; i<d->arch.vgic.nr_spis; i++)
> {
> INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
> INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
> @@ -213,7 +210,7 @@ void arch_move_irqs(struct vcpu *v)
> struct vcpu *v_target;
> int i;
>
> - for ( i = 32; i < d->arch.vgic.nr_lines; i++ )
> + for ( i = 32; i < d->arch.vgic.nr_spis; i++ )
> {
> v_target = vgic_get_target_vcpu(v, i);
> p = irq_to_pending(v_target, i);
> @@ -339,7 +336,7 @@ int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int
> struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
> {
> struct pending_irq *n;
> - /* Pending irqs allocation strategy: the first vgic.nr_lines irqs
> + /* Pending irqs allocation strategy: the first vgic.nr_spis irqs
> * are used for SPIs; the rests are used for per cpu irqs */
> if ( irq < 32 )
> n = &v->arch.vgic.pending_irqs[irq];
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 32d0554..5719fe5 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -89,7 +89,7 @@ struct arch_domain
> */
> spinlock_t lock;
> int ctlr;
> - int nr_lines; /* Number of SPIs */
> + int nr_spis; /* Number of SPIs */
> struct vgic_irq_rank *shared_irqs;
> /*
> * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 338ba03..5ddc681 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -102,7 +102,7 @@ struct vgic_ops {
> };
>
> /* Number of ranks of interrupt registers for a domain */
> -#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_lines+31)/32)
> +#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>
> #define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
> #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
> @@ -156,6 +156,8 @@ enum gic_sgi_mode;
> */
> #define REG_RANK_INDEX(b, n, s) ((((n) >> s) & ((b)-1)) % 32)
>
> +#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
> +
> extern int domain_vgic_init(struct domain *d);
> extern void domain_vgic_free(struct domain *d);
> extern int vcpu_vgic_init(struct vcpu *v);
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (2 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 03/21] xen/arm: vgic: Rename nr_lines into nr_spis Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 14:06 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO Julien Grall
` (17 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
The structure pending_irq is initialized on the same way in 2 differents
place. Introduce vgic_init_pending_irq to avoid code duplication.
Also move the setting of the irq field in this function as we need to
initialize it once rather than every time an IRQ is injected to the guest.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Patch added
---
xen/arch/arm/vgic.c | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index ac34437..17cde7a 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -53,6 +53,13 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
return vgic_rank_offset(v, 8, irq, DABT_WORD);
}
+static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
+{
+ INIT_LIST_HEAD(&p->inflight);
+ INIT_LIST_HEAD(&p->lr_queue);
+ p->irq = virq;
+}
+
int domain_vgic_init(struct domain *d)
{
int i;
@@ -88,10 +95,8 @@ int domain_vgic_init(struct domain *d)
}
for (i=0; i<d->arch.vgic.nr_spis; i++)
- {
- INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
- INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
- }
+ vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
+
for (i=0; i<DOMAIN_NR_RANKS(d); i++)
{
spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
@@ -131,10 +136,7 @@ int vcpu_vgic_init(struct vcpu *v)
memset(&v->arch.vgic.pending_irqs, 0, sizeof(v->arch.vgic.pending_irqs));
for (i = 0; i < 32; i++)
- {
- INIT_LIST_HEAD(&v->arch.vgic.pending_irqs[i].inflight);
- INIT_LIST_HEAD(&v->arch.vgic.pending_irqs[i].lr_queue);
- }
+ vgic_init_pending_irq(&v->arch.vgic.pending_irqs[i], i);
INIT_LIST_HEAD(&v->arch.vgic.inflight_irqs);
INIT_LIST_HEAD(&v->arch.vgic.lr_pending);
@@ -386,7 +388,6 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
goto out;
}
- n->irq = irq;
n->priority = priority;
/* the irq is enabled */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq
2014-07-31 15:00 ` [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq Julien Grall
@ 2014-08-06 14:06 ` Stefano Stabellini
2014-08-06 14:52 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 14:06 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> The structure pending_irq is initialized on the same way in 2 differents
> place. Introduce vgic_init_pending_irq to avoid code duplication.
>
> Also move the setting of the irq field in this function as we need to
> initialize it once rather than every time an IRQ is injected to the guest.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Changes in v2:
> - Patch added
> ---
> xen/arch/arm/vgic.c | 19 ++++++++++---------
> 1 file changed, 10 insertions(+), 9 deletions(-)
>
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index ac34437..17cde7a 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -53,6 +53,13 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
> return vgic_rank_offset(v, 8, irq, DABT_WORD);
> }
>
> +static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
> +{
> + INIT_LIST_HEAD(&p->inflight);
> + INIT_LIST_HEAD(&p->lr_queue);
> + p->irq = virq;
> +}
Reading this code made me realize that the irq field in pending_irq is a
signed integer, while virq is unsigned. We should be consistent.
In any case this patch is still an improvement over what we have now:
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> int domain_vgic_init(struct domain *d)
> {
> int i;
> @@ -88,10 +95,8 @@ int domain_vgic_init(struct domain *d)
> }
>
> for (i=0; i<d->arch.vgic.nr_spis; i++)
> - {
> - INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].inflight);
> - INIT_LIST_HEAD(&d->arch.vgic.pending_irqs[i].lr_queue);
> - }
> + vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i], i + 32);
> +
> for (i=0; i<DOMAIN_NR_RANKS(d); i++)
> {
> spin_lock_init(&d->arch.vgic.shared_irqs[i].lock);
> @@ -131,10 +136,7 @@ int vcpu_vgic_init(struct vcpu *v)
>
> memset(&v->arch.vgic.pending_irqs, 0, sizeof(v->arch.vgic.pending_irqs));
> for (i = 0; i < 32; i++)
> - {
> - INIT_LIST_HEAD(&v->arch.vgic.pending_irqs[i].inflight);
> - INIT_LIST_HEAD(&v->arch.vgic.pending_irqs[i].lr_queue);
> - }
> + vgic_init_pending_irq(&v->arch.vgic.pending_irqs[i], i);
>
> INIT_LIST_HEAD(&v->arch.vgic.inflight_irqs);
> INIT_LIST_HEAD(&v->arch.vgic.lr_pending);
> @@ -386,7 +388,6 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq)
> goto out;
> }
>
> - n->irq = irq;
> n->priority = priority;
>
> /* the irq is enabled */
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq
2014-08-06 14:06 ` Stefano Stabellini
@ 2014-08-06 14:52 ` Julien Grall
2014-08-06 14:57 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 14:52 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
On 08/06/2014 03:06 PM, Stefano Stabellini wrote:
> On Thu, 31 Jul 2014, Julien Grall wrote:
>> The structure pending_irq is initialized on the same way in 2 differents
>> place. Introduce vgic_init_pending_irq to avoid code duplication.
>>
>> Also move the setting of the irq field in this function as we need to
>> initialize it once rather than every time an IRQ is injected to the guest.
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
>
>
>> Changes in v2:
>> - Patch added
>> ---
>> xen/arch/arm/vgic.c | 19 ++++++++++---------
>> 1 file changed, 10 insertions(+), 9 deletions(-)
>>
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index ac34437..17cde7a 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -53,6 +53,13 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
>> return vgic_rank_offset(v, 8, irq, DABT_WORD);
>> }
>>
>> +static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
>> +{
>> + INIT_LIST_HEAD(&p->inflight);
>> + INIT_LIST_HEAD(&p->lr_queue);
>> + p->irq = virq;
>> +}
>
> Reading this code made me realize that the irq field in pending_irq is a
> signed integer, while virq is unsigned. We should be consistent.
I think we should use unsigned int everywhere. As p->irq is only used in
gic_dump_info, I will made the p->irq change type in this patch.
> In any case this patch is still an improvement over what we have now:
>
> Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Can I keep your keep with the p->irq change type?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq
2014-08-06 14:52 ` Julien Grall
@ 2014-08-06 14:57 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 14:57 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Wed, 6 Aug 2014, Julien Grall wrote:
> Hi Stefano,
>
> On 08/06/2014 03:06 PM, Stefano Stabellini wrote:
> > On Thu, 31 Jul 2014, Julien Grall wrote:
> >> The structure pending_irq is initialized on the same way in 2 differents
> >> place. Introduce vgic_init_pending_irq to avoid code duplication.
> >>
> >> Also move the setting of the irq field in this function as we need to
> >> initialize it once rather than every time an IRQ is injected to the guest.
> >>
> >> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >
> >
> >
> >> Changes in v2:
> >> - Patch added
> >> ---
> >> xen/arch/arm/vgic.c | 19 ++++++++++---------
> >> 1 file changed, 10 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> >> index ac34437..17cde7a 100644
> >> --- a/xen/arch/arm/vgic.c
> >> +++ b/xen/arch/arm/vgic.c
> >> @@ -53,6 +53,13 @@ struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
> >> return vgic_rank_offset(v, 8, irq, DABT_WORD);
> >> }
> >>
> >> +static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
> >> +{
> >> + INIT_LIST_HEAD(&p->inflight);
> >> + INIT_LIST_HEAD(&p->lr_queue);
> >> + p->irq = virq;
> >> +}
> >
> > Reading this code made me realize that the irq field in pending_irq is a
> > signed integer, while virq is unsigned. We should be consistent.
>
> I think we should use unsigned int everywhere. As p->irq is only used in
> gic_dump_info, I will made the p->irq change type in this patch.
>
> > In any case this patch is still an improvement over what we have now:
> >
> > Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>
> Can I keep your keep with the p->irq change type?
Yes
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (3 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 04/21] xen/arm: vgic: Introduce a function to initialize pending_irq Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-09-09 13:07 ` Ian Campbell
2014-07-31 15:00 ` [PATCH v2 06/21] xen/arm: Allow virq != irq Julien Grall
` (16 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
The commit 33233c2 "arch/arm: domain build: let dom0 access I/O memory of
mapped series" fill the iomem_caps to allow DOM0 managing MMIO of mapped
device.
A device can be disabled (i.e by adding a property status="disabled" in the
device tree) because the user may want to passthrough this device to a guest.
This will avoid DOM0 loading (and few minutes after unloading) the driver to
handle this device.
Even though, we don't want to let DOM0 using this device, the domain needs
to be able to manage the MMIO/IRQ range. This will allow the toolstack
to map MMIO/IRQ to a guest explicitly via "iomem" and "irqs" VM config
property or implicitly by passthrough a device.
Rework the function map_device (renamed into handle_device) to:
* For a given device node:
- Give permission to manage IRQ/MMIO for this device
- Retrieve the IRQ configuration (i.e edge/level) from the device
tree
* For available device (i.e status != disabled in the DT)
- Assign the device to the guest if it's protected by an IOMMU
- Map the IRQs and MMIOs regions to the guest
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Improve commit message
- Remove the third parameter of handle_device and get the status
of the device directly in the function.
---
xen/arch/arm/domain_build.c | 71 +++++++++++++++++++++++++++----------------
1 file changed, 45 insertions(+), 26 deletions(-)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 867dfd4..47d114f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -929,8 +929,14 @@ static int make_timer_node(const struct domain *d, void *fdt,
return res;
}
-/* Map the device in the domain */
-static int map_device(struct domain *d, struct dt_device_node *dev)
+/* For a given device node:
+ * - Give permission to the guest to manage IRQ and MMIO range
+ * - Retrieve the IRQ configuration (i.e edge/level) from device tree
+ * When the device is available:
+ * - Assign the device to the guest if it's protected by an IOMMU
+ * - Map the IRQs and iomem regions to DOM0
+ */
+static int handle_device(struct domain *d, struct dt_device_node *dev)
{
unsigned int nirq;
unsigned int naddr;
@@ -939,13 +945,15 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
unsigned int irq;
struct dt_raw_irq rirq;
u64 addr, size;
+ bool_t available = dt_device_is_available(dev);
nirq = dt_number_of_irq(dev);
naddr = dt_number_of_address(dev);
- DPRINT("%s nirq = %d naddr = %u\n", dt_node_full_name(dev), nirq, naddr);
+ DPRINT("%s available = %d nirq = %d naddr = %u\n", dt_node_full_name(dev),
+ available, nirq, naddr);
- if ( dt_device_is_protected(dev) )
+ if ( dt_device_is_protected(dev) && available )
{
DPRINT("%s setup iommu\n", dt_node_full_name(dev));
res = iommu_assign_dt_device(d, dev);
@@ -957,7 +965,7 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
}
}
- /* Map IRQs */
+ /* Give permission and map IRQs */
for ( i = 0; i < nirq; i++ )
{
res = dt_device_get_raw_irq(dev, i, &rirq);
@@ -990,16 +998,28 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
irq = res;
DPRINT("irq %u = %u\n", i, irq);
- res = route_irq_to_guest(d, irq, dt_node_name(dev));
+
+ res = irq_permit_access(d, irq);
if ( res )
{
- printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
- irq, d->domain_id);
+ printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
+ d->domain_id, irq);
return res;
}
+
+ if ( available )
+ {
+ res = route_irq_to_guest(d, irq, dt_node_name(dev));
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
+ irq, d->domain_id);
+ return res;
+ }
+ }
}
- /* Map the address ranges */
+ /* Give permission and map MMIOs */
for ( i = 0; i < naddr; i++ )
{
res = dt_device_get_address(dev, i, &addr, &size);
@@ -1023,17 +1043,21 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
return res;
}
- res = map_mmio_regions(d,
- paddr_to_pfn(addr & PAGE_MASK),
- DIV_ROUND_UP(size, PAGE_SIZE),
- paddr_to_pfn(addr & PAGE_MASK));
- if ( res )
+
+ if ( available )
{
- printk(XENLOG_ERR "Unable to map 0x%"PRIx64
- " - 0x%"PRIx64" in domain %d\n",
- addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1,
- d->domain_id);
- return res;
+ res = map_mmio_regions(d,
+ paddr_to_pfn(addr & PAGE_MASK),
+ DIV_ROUND_UP(size, PAGE_SIZE),
+ paddr_to_pfn(addr & PAGE_MASK));
+ if ( res )
+ {
+ printk(XENLOG_ERR "Unable to map 0x%"PRIx64
+ " - 0x%"PRIx64" in domain %d\n",
+ addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1,
+ d->domain_id);
+ return res;
+ }
}
}
@@ -1110,15 +1134,10 @@ static int handle_node(struct domain *d, struct kernel_info *kinfo,
* Some device doesn't need to be mapped in Xen:
* - Memory: the guest will see a different view of memory. It will
* be allocated later.
- * - Disabled device: Linux is able to cope with status="disabled"
- * property. Therefore these device doesn't need to be mapped. This
- * solution can be use later for pass through.
*/
- if ( !dt_device_type_is_equal(node, "memory") &&
- dt_device_is_available(node) )
+ if ( !dt_device_type_is_equal(node, "memory") )
{
- res = map_device(d, node);
-
+ res = handle_device(d, node);
if ( res )
return res;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO
2014-07-31 15:00 ` [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO Julien Grall
@ 2014-09-09 13:07 ` Ian Campbell
2014-09-11 22:32 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:07 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, stefano.stabellini
On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
Please make the subject line reflect what this patches function is, not
that it is a follow up to some other patch.
> The commit 33233c2 "arch/arm: domain build: let dom0 access I/O memory of
> mapped series" fill the iomem_caps to allow DOM0 managing MMIO of mapped
> device.
"fills in ... to allow DOM0 to manage MMIO of mapped devices" (I think
that is what was meant")
>
> A device can be disabled (i.e by adding a property status="disabled" in the
> device tree) because the user may want to passthrough this device to a guest.
> This will avoid DOM0 loading (and few minutes after unloading) the driver to
s/after/later/
> handle this device.
>
> Even though, we don't want to let DOM0 using this device, the domain needs
"use". And I think s/the domain/it/ since you are referring to DOM0
(just mentioned, so "it" is fine in the context) not the domain as in
the guest.
> to be able to manage the MMIO/IRQ range. This will allow the toolstack
> to map MMIO/IRQ to a guest explicitly via "iomem" and "irqs" VM config
> property or implicitly by passthrough a device.
"by passing through"
> Rework the function map_device (renamed into handle_device) to:
>
> * For a given device node:
> - Give permission to manage IRQ/MMIO for this device
> - Retrieve the IRQ configuration (i.e edge/level) from the device
> tree
> * For available device (i.e status != disabled in the DT)
> - Assign the device to the guest if it's protected by an IOMMU
> - Map the IRQs and MMIOs regions to the guest
So based on that I think a suitable $subject would be something like
"give domain 0 permission to manage IRQ/MMIO for disabled devices" or
something like that.
Is it a good idea to override status="disabled" in this way or should be
add our own xen,passthrough as a boolean DT property?
My concern is that we are also doing this for devices which are disabled
for some other reason (buggy? security sensitive? not present on this
partiocular board etc). It also leaves no way to mark a device as "not
for any domain, including dom0 or passthrough". Quite a few .dtsi files
define loads of stuff as disabled and then let the board files override
what they actually have as okay.
So I think keeping status="disabled" as more of a firmware thing and
having our own for devices which are to be passed through makes sense.
Since I expect the naming of the property isn't going to have a major
impact on the code itself outside of a few predicate functions I took a
look and it looks good, apart from a few nits which I'll mention.
> @@ -957,7 +965,7 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
> }
> }
>
> - /* Map IRQs */
> + /* Give permission and map IRQs */
double space.
> for ( i = 0; i < nirq; i++ )
> {
> res = dt_device_get_raw_irq(dev, i, &rirq);
> @@ -990,16 +998,28 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
> irq = res;
>
> DPRINT("irq %u = %u\n", i, irq);
> - res = route_irq_to_guest(d, irq, dt_node_name(dev));
> +
> + res = irq_permit_access(d, irq);
> if ( res )
> {
> - printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
> - irq, d->domain_id);
> + printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
> + d->domain_id, irq);
> return res;
> }
> +
> + if ( available )
if ( !avaiable ) \n continue would save you an indentation level.
> + {
> + res = route_irq_to_guest(d, irq, dt_node_name(dev));
> + if ( res )
> + {
> + printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
> + irq, d->domain_id);
> + return res;
> + }
> + }
> }
>
> - /* Map the address ranges */
> + /* Give permission and map MMIOs */
Permissions are now given above, not below, aren't they?
> for ( i = 0; i < naddr; i++ )
> {
> res = dt_device_get_address(dev, i, &addr, &size);
> @@ -1023,17 +1043,21 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
> addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
> return res;
> }
> - res = map_mmio_regions(d,
> - paddr_to_pfn(addr & PAGE_MASK),
> - DIV_ROUND_UP(size, PAGE_SIZE),
> - paddr_to_pfn(addr & PAGE_MASK));
> - if ( res )
> +
> + if ( available )
if ( !avaiable ) \n continue again.
> {
> - printk(XENLOG_ERR "Unable to map 0x%"PRIx64
> - " - 0x%"PRIx64" in domain %d\n",
> - addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1,
> - d->domain_id);
> - return res;
> + res = map_mmio_regions(d,
> + paddr_to_pfn(addr & PAGE_MASK),
> + DIV_ROUND_UP(size, PAGE_SIZE),
> + paddr_to_pfn(addr & PAGE_MASK));
> + if ( res )
> + {
> + printk(XENLOG_ERR "Unable to map 0x%"PRIx64
> + " - 0x%"PRIx64" in domain %d\n",
> + addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1,
> + d->domain_id);
> + return res;
> + }
> }
> }
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO
2014-09-09 13:07 ` Ian Campbell
@ 2014-09-11 22:32 ` Julien Grall
2014-09-12 10:13 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-11 22:32 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, tim, stefano.stabellini
Hi Ian,
On 09/09/14 06:07, Ian Campbell wrote:
> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
>
> Please make the subject line reflect what this patches function is, not
> that it is a follow up to some other patch.
>> The commit 33233c2 "arch/arm: domain build: let dom0 access I/O memory of
>> mapped series" fill the iomem_caps to allow DOM0 managing MMIO of mapped
>> device.
>
> "fills in ... to allow DOM0 to manage MMIO of mapped devices" (I think
> that is what was meant")
Yes, I will change in the next version.
>>
>> A device can be disabled (i.e by adding a property status="disabled" in the
>> device tree) because the user may want to passthrough this device to a guest.
>> This will avoid DOM0 loading (and few minutes after unloading) the driver to
>
> s/after/later/
>
>> handle this device.
>>
>> Even though, we don't want to let DOM0 using this device, the domain needs
>
> "use". And I think s/the domain/it/ since you are referring to DOM0
> (just mentioned, so "it" is fine in the context) not the domain as in
> the guest.
>
>> to be able to manage the MMIO/IRQ range. This will allow the toolstack
>> to map MMIO/IRQ to a guest explicitly via "iomem" and "irqs" VM config
>> property or implicitly by passthrough a device.
>
> "by passing through"
>
>> Rework the function map_device (renamed into handle_device) to:
>>
>> * For a given device node:
>> - Give permission to manage IRQ/MMIO for this device
>> - Retrieve the IRQ configuration (i.e edge/level) from the device
>> tree
>> * For available device (i.e status != disabled in the DT)
>> - Assign the device to the guest if it's protected by an IOMMU
>> - Map the IRQs and MMIOs regions to the guest
>
> So based on that I think a suitable $subject would be something like
> "give domain 0 permission to manage IRQ/MMIO for disabled devices" or
> something like that.
It's a follow-up in the sense that it improves Arianna patch to handle
MMIO and IRQ.
Your suggestion for the title sounds good. I will use it.
> Is it a good idea to override status="disabled" in this way or should be
> add our own xen,passthrough as a boolean DT property?
>
> My concern is that we are also doing this for devices which are disabled
> for some other reason (buggy? security sensitive? not present on this
> partiocular board etc). It also leaves no way to mark a device as "not
> for any domain, including dom0 or passthrough". Quite a few .dtsi files
> define loads of stuff as disabled and then let the board files override
> what they actually have as okay.
>
> So I think keeping status="disabled" as more of a firmware thing and
> having our own for devices which are to be passed through makes sense.
Using status="disabled" was an easy solution because Linux, and most of
the other OS, knows the device should not be used by DOM0.
The current solution doesn't map the device into DOM0 memory but the
device is still described in the device tree. That would let us the
possibility to retrieve from DOM0 the properties/compatible of the
device via a driver (and would avoid the bunch of hypercalls introduced
later).
With the new property "xen,passthrough", we would have to remove the
node from DOM0, or teach DOM0 that the device should not be used.
Overall, I don't think dropping the node in DOM0 device tree will impact
it. If it's the case that would mean the device should not be
passthrough to another guest. So I will give a look to introduce this
new property. Shall I send a patch to the device tree bindings ML?
BTW, I don't think the new property should be a boolean. Use only the
name should be enough here.
> Since I expect the naming of the property isn't going to have a major
> impact on the code itself outside of a few predicate functions I took a
> look and it looks good, apart from a few nits which I'll mention.
>
>> @@ -957,7 +965,7 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
>> }
>> }
>>
>> - /* Map IRQs */
>> + /* Give permission and map IRQs */
>
> double space.
>
>> for ( i = 0; i < nirq; i++ )
>> {
>> res = dt_device_get_raw_irq(dev, i, &rirq);
>> @@ -990,16 +998,28 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
>> irq = res;
>>
>> DPRINT("irq %u = %u\n", i, irq);
>> - res = route_irq_to_guest(d, irq, dt_node_name(dev));
>> +
>> + res = irq_permit_access(d, irq);
>> if ( res )
>> {
>> - printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
>> - irq, d->domain_id);
>> + printk(XENLOG_ERR "Unable to permit to dom%u access to IRQ %u\n",
>> + d->domain_id, irq);
>> return res;
>> }
>> +
>> + if ( available )
>
> if ( !avaiable ) \n continue would save you an indentation level.
Ok.
>> + {
>> + res = route_irq_to_guest(d, irq, dt_node_name(dev));
>> + if ( res )
>> + {
>> + printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
>> + irq, d->domain_id);
>> + return res;
>> + }
>> + }
>> }
>>
>> - /* Map the address ranges */
>> + /* Give permission and map MMIOs */
>
> Permissions are now given above, not below, aren't they?
Only permission for IRQ. The MMIO ones are given few lines below.
>> for ( i = 0; i < naddr; i++ )
>> {
>> res = dt_device_get_address(dev, i, &addr, &size);
>> @@ -1023,17 +1043,21 @@ static int map_device(struct domain *d, struct dt_device_node *dev)
>> addr & PAGE_MASK, PAGE_ALIGN(addr + size) - 1);
>> return res;
>> }
>> - res = map_mmio_regions(d,
>> - paddr_to_pfn(addr & PAGE_MASK),
>> - DIV_ROUND_UP(size, PAGE_SIZE),
>> - paddr_to_pfn(addr & PAGE_MASK));
>> - if ( res )
>> +
>> + if ( available )
>
> if ( !avaiable ) \n continue again.
Ok.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO
2014-09-11 22:32 ` Julien Grall
@ 2014-09-12 10:13 ` Ian Campbell
2014-09-12 19:04 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-12 10:13 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, stefano.stabellini
On Thu, 2014-09-11 at 15:32 -0700, Julien Grall wrote:
> The current solution doesn't map the device into DOM0 memory but the
> device is still described in the device tree.
I had some thoughts on that in the thread on the 00/21 mail BTW.
> With the new property "xen,passthrough", we would have to remove the
> node from DOM0, or teach DOM0 that the device should not be used.
You would translate it into status="disabled" for dom0, that's all, no
need to remove anything.
>
> Overall, I don't think dropping the node in DOM0 device tree will impact
> it. If it's the case that would mean the device should not be
> passthrough to another guest. So I will give a look to introduce this
> new property. Shall I send a patch to the device tree bindings ML?
>
> BTW, I don't think the new property should be a boolean. Use only the
> name should be enough here.
That's what boolean means in DT context, a name without a value.
>
> >> + {
> >> + res = route_irq_to_guest(d, irq, dt_node_name(dev));
> >> + if ( res )
> >> + {
> >> + printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
> >> + irq, d->domain_id);
> >> + return res;
> >> + }
> >> + }
> >> }
> >>
> >> - /* Map the address ranges */
> >> + /* Give permission and map MMIOs */
> >
> > Permissions are now given above, not below, aren't they?
>
> Only permission for IRQ. The MMIO ones are given few lines below.
Ah, right.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO
2014-09-12 10:13 ` Ian Campbell
@ 2014-09-12 19:04 ` Julien Grall
0 siblings, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-09-12 19:04 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, tim, stefano.stabellini
Hi Ian,
On 12/09/14 03:13, Ian Campbell wrote:
> On Thu, 2014-09-11 at 15:32 -0700, Julien Grall wrote:
>> The current solution doesn't map the device into DOM0 memory but the
>> device is still described in the device tree.
>
> I had some thoughts on that in the thread on the 00/21 mail BTW.
>
>> With the new property "xen,passthrough", we would have to remove the
>> node from DOM0, or teach DOM0 that the device should not be used.
>
> You would translate it into status="disabled" for dom0, that's all, no
> need to remove anything.
I didn't think about this. So this "xen.passthrough" property will be
only used by Xen.
I will implement it for the next version.
>>
>> Overall, I don't think dropping the node in DOM0 device tree will impact
>> it. If it's the case that would mean the device should not be
>> passthrough to another guest. So I will give a look to introduce this
>> new property. Shall I send a patch to the device tree bindings ML?
>>
>> BTW, I don't think the new property should be a boolean. Use only the
>> name should be enough here.
>
> That's what boolean means in DT context, a name without a value.
Oh right. I had in mind the status="disabled" stuff.
>>
>
>>>> + {
>>>> + res = route_irq_to_guest(d, irq, dt_node_name(dev));
>>>> + if ( res )
>>>> + {
>>>> + printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
>>>> + irq, d->domain_id);
>>>> + return res;
>>>> + }
>>>> + }
>>>> }
>>>>
>>>> - /* Map the address ranges */
>>>> + /* Give permission and map MMIOs */
>>>
>>> Permissions are now given above, not below, aren't they?
>>
>> Only permission for IRQ. The MMIO ones are given few lines below.
>
> Ah, right.
I can update the comment to make more clear.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (4 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 05/21] xen/arm: follow-up to allow DOM0 manage IRQ and MMIO Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 14:50 ` Stefano Stabellini
2014-09-09 13:29 ` Ian Campbell
2014-07-31 15:00 ` [PATCH v2 07/21] xen/arm: route_irq_to_guest: Check validity of the IRQ Julien Grall
` (15 subsequent siblings)
21 siblings, 2 replies; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
Actually Xen is assuming that the virtual IRQ will always be equal the IRQ.
Modify, route_guest_irq to take the virtual IRQ in parameter and let Xen
assigned a different IRQ number. Also store the vIRQ in the desc action to
retrieve easily the IRQ target when we need to inject the interrupt.
As DOM0 will get most the device, the vIRQ is equal to the IRQ.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Patch added
---
xen/arch/arm/domain_build.c | 2 +-
xen/arch/arm/gic.c | 5 +++--
xen/arch/arm/irq.c | 47 +++++++++++++++++++++++++++++++------------
xen/include/asm-arm/gic.h | 3 ++-
xen/include/asm-arm/irq.h | 4 ++--
5 files changed, 42 insertions(+), 19 deletions(-)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 47d114f..0dfe223 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1009,7 +1009,7 @@ static int handle_device(struct domain *d, struct dt_device_node *dev)
if ( available )
{
- res = route_irq_to_guest(d, irq, dt_node_name(dev));
+ res = route_irq_to_guest(d, irq, irq, dt_node_name(dev));
if ( res )
{
printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 6611ba0..8ef8764 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -126,7 +126,8 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
/* Program the GIC to route an interrupt to a guest
* - desc.lock must be held
*/
-void gic_route_irq_to_guest(struct domain *d, struct irq_desc *desc,
+void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc,
const cpumask_t *cpu_mask, unsigned int priority)
{
struct pending_irq *p;
@@ -139,7 +140,7 @@ void gic_route_irq_to_guest(struct domain *d, struct irq_desc *desc,
/* Use vcpu0 to retrieve the pending_irq struct. Given that we only
* route SPIs to guests, it doesn't make any difference. */
- p = irq_to_pending(d->vcpu[0], desc->irq);
+ p = irq_to_pending(d->vcpu[0], virq);
p->desc = desc;
}
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 25ecf1d..830832c 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -31,6 +31,13 @@
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
static DEFINE_SPINLOCK(local_irqs_type_lock);
+/* Describe an IRQ assigned to a guest */
+struct irq_guest
+{
+ struct domain *d;
+ unsigned int virq;
+};
+
static void ack_none(struct irq_desc *irq)
{
printk("unexpected IRQ trap at irq %02x\n", irq->irq);
@@ -122,18 +129,20 @@ void __cpuinit init_secondary_IRQ(void)
BUG_ON(init_local_irq_data() < 0);
}
-static inline struct domain *irq_get_domain(struct irq_desc *desc)
+static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
{
ASSERT(spin_is_locked(&desc->lock));
-
- if ( !test_bit(_IRQ_GUEST, &desc->status) )
- return dom_xen;
-
+ ASSERT(test_bit(_IRQ_GUEST, &desc->status));
ASSERT(desc->action != NULL);
return desc->action->dev_id;
}
+static inline struct domain *irq_get_domain(struct irq_desc *desc)
+{
+ return irq_get_guest_info(desc)->d;
+}
+
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
{
if ( desc != NULL )
@@ -197,7 +206,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
if ( test_bit(_IRQ_GUEST, &desc->status) )
{
- struct domain *d = irq_get_domain(desc);
+ struct irq_guest *info = irq_get_guest_info(desc);
desc->handler->end(desc);
@@ -206,7 +215,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
/* the irq cannot be a PPI, we only support delivery of SPIs to
* guests */
- vgic_vcpu_inject_spi(d, irq);
+ vgic_vcpu_inject_spi(info->d, info->virq);
goto out_no_end;
}
@@ -370,19 +379,30 @@ err:
return rc;
}
-int route_irq_to_guest(struct domain *d, unsigned int irq,
- const char * devname)
+int route_irq_to_guest(struct domain *d, unsigned int virq,
+ unsigned int irq, const char * devname)
{
struct irqaction *action;
- struct irq_desc *desc = irq_to_desc(irq);
+ struct irq_guest *info;
+ struct irq_desc *desc;
unsigned long flags;
int retval = 0;
action = xmalloc(struct irqaction);
- if (!action)
+ if ( !action )
+ return -ENOMEM;
+
+ info = xmalloc(struct irq_guest);
+ if ( !info )
+ {
+ xfree(action);
return -ENOMEM;
+ }
+
+ info->d = d;
+ info->virq = virq;
- action->dev_id = d;
+ action->dev_id = info;
action->name = devname;
action->free_on_release = 1;
@@ -413,7 +433,7 @@ int route_irq_to_guest(struct domain *d, unsigned int irq,
if ( retval )
goto out;
- gic_route_irq_to_guest(d, desc, cpumask_of(smp_processor_id()),
+ gic_route_irq_to_guest(d, virq, desc, cpumask_of(smp_processor_id()),
GIC_PRI_IRQ);
spin_unlock_irqrestore(&desc->lock, flags);
return 0;
@@ -421,6 +441,7 @@ int route_irq_to_guest(struct domain *d, unsigned int irq,
out:
spin_unlock_irqrestore(&desc->lock, flags);
xfree(action);
+ xfree(info);
return retval;
}
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index a0c07bf..89816cd 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -196,7 +196,8 @@ extern enum gic_version gic_hw_version(void);
/* Program the GIC to route an interrupt */
extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
unsigned int priority);
-extern void gic_route_irq_to_guest(struct domain *, struct irq_desc *desc,
+extern void gic_route_irq_to_guest(struct domain *, unsigned int virq,
+ struct irq_desc *desc,
const cpumask_t *cpu_mask,
unsigned int priority);
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index e877334..9bc3492 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -40,8 +40,8 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq);
void init_IRQ(void);
void init_secondary_IRQ(void);
-int route_irq_to_guest(struct domain *d, unsigned int irq,
- const char *devname);
+int route_irq_to_guest(struct domain *d, unsigned int virq,
+ unsigned int irq, const char *devname);
void arch_move_irqs(struct vcpu *v);
/* Set IRQ type for an SPI */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-07-31 15:00 ` [PATCH v2 06/21] xen/arm: Allow virq != irq Julien Grall
@ 2014-08-06 14:50 ` Stefano Stabellini
2014-08-06 15:07 ` Julien Grall
2014-09-09 13:29 ` Ian Campbell
1 sibling, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 14:50 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> Actually Xen is assuming that the virtual IRQ will always be equal the IRQ.
>
> Modify, route_guest_irq to take the virtual IRQ in parameter and let Xen
> assigned a different IRQ number. Also store the vIRQ in the desc action to
> retrieve easily the IRQ target when we need to inject the interrupt.
>
> As DOM0 will get most the device, the vIRQ is equal to the IRQ.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> ---
> Changes in v2:
> - Patch added
> ---
> xen/arch/arm/domain_build.c | 2 +-
> xen/arch/arm/gic.c | 5 +++--
> xen/arch/arm/irq.c | 47 +++++++++++++++++++++++++++++++------------
> xen/include/asm-arm/gic.h | 3 ++-
> xen/include/asm-arm/irq.h | 4 ++--
> 5 files changed, 42 insertions(+), 19 deletions(-)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 47d114f..0dfe223 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1009,7 +1009,7 @@ static int handle_device(struct domain *d, struct dt_device_node *dev)
>
> if ( available )
> {
> - res = route_irq_to_guest(d, irq, dt_node_name(dev));
> + res = route_irq_to_guest(d, irq, irq, dt_node_name(dev));
> if ( res )
> {
> printk(XENLOG_ERR "Unable to route IRQ %u to domain %u\n",
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 6611ba0..8ef8764 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -126,7 +126,8 @@ void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> /* Program the GIC to route an interrupt to a guest
> * - desc.lock must be held
> */
> -void gic_route_irq_to_guest(struct domain *d, struct irq_desc *desc,
> +void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc,
> const cpumask_t *cpu_mask, unsigned int priority)
> {
> struct pending_irq *p;
> @@ -139,7 +140,7 @@ void gic_route_irq_to_guest(struct domain *d, struct irq_desc *desc,
>
> /* Use vcpu0 to retrieve the pending_irq struct. Given that we only
> * route SPIs to guests, it doesn't make any difference. */
> - p = irq_to_pending(d->vcpu[0], desc->irq);
> + p = irq_to_pending(d->vcpu[0], virq);
> p->desc = desc;
> }
>
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 25ecf1d..830832c 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -31,6 +31,13 @@
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> static DEFINE_SPINLOCK(local_irqs_type_lock);
>
> +/* Describe an IRQ assigned to a guest */
> +struct irq_guest
> +{
> + struct domain *d;
> + unsigned int virq;
> +};
> +
> static void ack_none(struct irq_desc *irq)
> {
> printk("unexpected IRQ trap at irq %02x\n", irq->irq);
> @@ -122,18 +129,20 @@ void __cpuinit init_secondary_IRQ(void)
> BUG_ON(init_local_irq_data() < 0);
> }
>
> -static inline struct domain *irq_get_domain(struct irq_desc *desc)
> +static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
> {
> ASSERT(spin_is_locked(&desc->lock));
> -
> - if ( !test_bit(_IRQ_GUEST, &desc->status) )
> - return dom_xen;
> -
> + ASSERT(test_bit(_IRQ_GUEST, &desc->status));
> ASSERT(desc->action != NULL);
>
> return desc->action->dev_id;
> }
>
> +static inline struct domain *irq_get_domain(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->d;
> +}
> +
> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *cpu_mask)
> {
> if ( desc != NULL )
> @@ -197,7 +206,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>
> if ( test_bit(_IRQ_GUEST, &desc->status) )
> {
> - struct domain *d = irq_get_domain(desc);
> + struct irq_guest *info = irq_get_guest_info(desc);
>
> desc->handler->end(desc);
>
> @@ -206,7 +215,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>
> /* the irq cannot be a PPI, we only support delivery of SPIs to
> * guests */
> - vgic_vcpu_inject_spi(d, irq);
> + vgic_vcpu_inject_spi(info->d, info->virq);
> goto out_no_end;
> }
>
> @@ -370,19 +379,30 @@ err:
> return rc;
> }
>
> -int route_irq_to_guest(struct domain *d, unsigned int irq,
> - const char * devname)
> +int route_irq_to_guest(struct domain *d, unsigned int virq,
> + unsigned int irq, const char * devname)
> {
> struct irqaction *action;
> - struct irq_desc *desc = irq_to_desc(irq);
> + struct irq_guest *info;
> + struct irq_desc *desc;
> unsigned long flags;
> int retval = 0;
>
> action = xmalloc(struct irqaction);
> - if (!action)
> + if ( !action )
> + return -ENOMEM;
> +
> + info = xmalloc(struct irq_guest);
> + if ( !info )
> + {
> + xfree(action);
> return -ENOMEM;
> + }
Rather than xmalloc'ing another struct and storing the pointer in
dev_id, maybe we could simply expand struct arch_irq_desc?
> + info->d = d;
> + info->virq = virq;
>
> - action->dev_id = d;
> + action->dev_id = info;
> action->name = devname;
> action->free_on_release = 1;
>
> @@ -413,7 +433,7 @@ int route_irq_to_guest(struct domain *d, unsigned int irq,
> if ( retval )
> goto out;
>
> - gic_route_irq_to_guest(d, desc, cpumask_of(smp_processor_id()),
> + gic_route_irq_to_guest(d, virq, desc, cpumask_of(smp_processor_id()),
> GIC_PRI_IRQ);
> spin_unlock_irqrestore(&desc->lock, flags);
> return 0;
> @@ -421,6 +441,7 @@ int route_irq_to_guest(struct domain *d, unsigned int irq,
> out:
> spin_unlock_irqrestore(&desc->lock, flags);
> xfree(action);
> + xfree(info);
>
> return retval;
> }
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index a0c07bf..89816cd 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -196,7 +196,8 @@ extern enum gic_version gic_hw_version(void);
> /* Program the GIC to route an interrupt */
> extern void gic_route_irq_to_xen(struct irq_desc *desc, const cpumask_t *cpu_mask,
> unsigned int priority);
> -extern void gic_route_irq_to_guest(struct domain *, struct irq_desc *desc,
> +extern void gic_route_irq_to_guest(struct domain *, unsigned int virq,
> + struct irq_desc *desc,
> const cpumask_t *cpu_mask,
> unsigned int priority);
>
> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
> index e877334..9bc3492 100644
> --- a/xen/include/asm-arm/irq.h
> +++ b/xen/include/asm-arm/irq.h
> @@ -40,8 +40,8 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq);
> void init_IRQ(void);
> void init_secondary_IRQ(void);
>
> -int route_irq_to_guest(struct domain *d, unsigned int irq,
> - const char *devname);
> +int route_irq_to_guest(struct domain *d, unsigned int virq,
> + unsigned int irq, const char *devname);
> void arch_move_irqs(struct vcpu *v);
>
> /* Set IRQ type for an SPI */
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-08-06 14:50 ` Stefano Stabellini
@ 2014-08-06 15:07 ` Julien Grall
2014-08-06 16:48 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 15:07 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
On 08/06/2014 03:50 PM, Stefano Stabellini wrote:
>> -int route_irq_to_guest(struct domain *d, unsigned int irq,
>> - const char * devname)
>> +int route_irq_to_guest(struct domain *d, unsigned int virq,
>> + unsigned int irq, const char * devname)
>> {
>> struct irqaction *action;
>> - struct irq_desc *desc = irq_to_desc(irq);
>> + struct irq_guest *info;
>> + struct irq_desc *desc;
>> unsigned long flags;
>> int retval = 0;
>>
>> action = xmalloc(struct irqaction);
>> - if (!action)
>> + if ( !action )
>> + return -ENOMEM;
>> +
>> + info = xmalloc(struct irq_guest);
>> + if ( !info )
>> + {
>> + xfree(action);
>> return -ENOMEM;
>> + }
>
> Rather than xmalloc'ing another struct and storing the pointer in
> dev_id, maybe we could simply expand struct arch_irq_desc?
I though about it. If we add another field in arch_irq_desc, we will
likely use more memory than xmalloc. This is because most of the
platform doesn't use 1024 interrupts but about 256 interrupts.
As the new field will be a pointer (on ARM64, 8 bytes), that would make
Xen use statically about 8K more.
We could allocate irq_desc dynamically during Xen boot.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-08-06 15:07 ` Julien Grall
@ 2014-08-06 16:48 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:48 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Wed, 6 Aug 2014, Julien Grall wrote:
> Hi Stefano,
>
> On 08/06/2014 03:50 PM, Stefano Stabellini wrote:
> >> -int route_irq_to_guest(struct domain *d, unsigned int irq,
> >> - const char * devname)
> >> +int route_irq_to_guest(struct domain *d, unsigned int virq,
> >> + unsigned int irq, const char * devname)
> >> {
> >> struct irqaction *action;
> >> - struct irq_desc *desc = irq_to_desc(irq);
> >> + struct irq_guest *info;
> >> + struct irq_desc *desc;
> >> unsigned long flags;
> >> int retval = 0;
> >>
> >> action = xmalloc(struct irqaction);
> >> - if (!action)
> >> + if ( !action )
> >> + return -ENOMEM;
> >> +
> >> + info = xmalloc(struct irq_guest);
> >> + if ( !info )
> >> + {
> >> + xfree(action);
> >> return -ENOMEM;
> >> + }
> >
> > Rather than xmalloc'ing another struct and storing the pointer in
> > dev_id, maybe we could simply expand struct arch_irq_desc?
>
> I though about it. If we add another field in arch_irq_desc, we will
> likely use more memory than xmalloc. This is because most of the
> platform doesn't use 1024 interrupts but about 256 interrupts.
>
> As the new field will be a pointer (on ARM64, 8 bytes), that would make
> Xen use statically about 8K more.
OK
> We could allocate irq_desc dynamically during Xen boot.
Let's leave it as is.
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-07-31 15:00 ` [PATCH v2 06/21] xen/arm: Allow virq != irq Julien Grall
2014-08-06 14:50 ` Stefano Stabellini
@ 2014-09-09 13:29 ` Ian Campbell
2014-09-09 18:42 ` Julien Grall
1 sibling, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:29 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, stefano.stabellini
Mostly spelling/grammar nits, a couple of questions but mostly looks
good.
On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
> Actually Xen is assuming that the virtual IRQ will always be equal the IRQ.
"be equal to" or better "always be the same as".
>
> Modify, route_guest_irq to take the virtual IRQ in parameter and let Xen
Spurious comma.
> assigned a different IRQ number. Also store the vIRQ in the desc action to
"assign".
> retrieve easily the IRQ target when we need to inject the interrupt.
"easily retrieve" (or "be able to easily retrieve")
> As DOM0 will get most the device, the vIRQ is equal to the IRQ.
"most of the devices".
I suppose the sentence should end with "in that case" or something?
> @@ -122,18 +129,20 @@ void __cpuinit init_secondary_IRQ(void)
> BUG_ON(init_local_irq_data() < 0);
> }
>
> -static inline struct domain *irq_get_domain(struct irq_desc *desc)
> +static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
> {
> ASSERT(spin_is_locked(&desc->lock));
> -
> - if ( !test_bit(_IRQ_GUEST, &desc->status) )
> - return dom_xen;
> -
> + ASSERT(test_bit(_IRQ_GUEST, &desc->status));
> ASSERT(desc->action != NULL);
>
> return desc->action->dev_id;
> }
>
> +static inline struct domain *irq_get_domain(struct irq_desc *desc)
> +{
> + return irq_get_guest_info(desc)->d;
Previously irq_get_domain could return dom_xen for !_IRQ_GUEST domains.
What happened to this logic? If it is never possible to get here with a
Xen owned IRQ then I think that is worth mentioning in the commit log
and/or a comment.
> @@ -206,7 +215,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>
> /* the irq cannot be a PPI, we only support delivery of SPIs to
> * guests */
> - vgic_vcpu_inject_spi(d, irq);
> + vgic_vcpu_inject_spi(info->d, info->virq);
Could this function take an irq_guest* or are there other callers?
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-09-09 13:29 ` Ian Campbell
@ 2014-09-09 18:42 ` Julien Grall
2014-09-11 22:50 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-09 18:42 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, tim, stefano.stabellini
Hi Ian,
On 09/09/14 06:29, Ian Campbell wrote:
> Mostly spelling/grammar nits, a couple of questions but mostly looks
> good.
>
> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
>> Actually Xen is assuming that the virtual IRQ will always be equal the IRQ.
>
> "be equal to" or better "always be the same as".
>
>>
>> Modify, route_guest_irq to take the virtual IRQ in parameter and let Xen
>
> Spurious comma.
>
>> assigned a different IRQ number. Also store the vIRQ in the desc action to
>
> "assign".
>
>> retrieve easily the IRQ target when we need to inject the interrupt.
>
> "easily retrieve" (or "be able to easily retrieve")
>
>> As DOM0 will get most the device, the vIRQ is equal to the IRQ.
>
> "most of the devices".
>
> I suppose the sentence should end with "in that case" or something?
Yes. I will fix all the grammar nits in the next version.
>> @@ -122,18 +129,20 @@ void __cpuinit init_secondary_IRQ(void)
>> BUG_ON(init_local_irq_data() < 0);
>> }
>>
>> -static inline struct domain *irq_get_domain(struct irq_desc *desc)
>> +static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
>> {
>> ASSERT(spin_is_locked(&desc->lock));
>> -
>> - if ( !test_bit(_IRQ_GUEST, &desc->status) )
>> - return dom_xen;
>> -
>> + ASSERT(test_bit(_IRQ_GUEST, &desc->status));
>> ASSERT(desc->action != NULL);
>>
>> return desc->action->dev_id;
>> }
>>
>> +static inline struct domain *irq_get_domain(struct irq_desc *desc)
>> +{
>> + return irq_get_guest_info(desc)->d;
>
> Previously irq_get_domain could return dom_xen for !_IRQ_GUEST domains.
> What happened to this logic? If it is never possible to get here with a
> Xen owned IRQ then I think that is worth mentioning in the commit log
> and/or a comment.
This was a safe guard in case of developer were misusing the function.
Actually every callers of this function are checking that the IRQ is
assigned to a guest before calling irq_get_domain.
I will mention it in the commit log.
>
>> @@ -206,7 +215,7 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq)
>>
>> /* the irq cannot be a PPI, we only support delivery of SPIs to
>> * guests */
>> - vgic_vcpu_inject_spi(d, irq);
>> + vgic_vcpu_inject_spi(info->d, info->virq);
>
> Could this function take an irq_guest* or are there other callers?
I haven't thought about this possibility. This is the only caller, so I
will pass irq_guest.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-09-09 18:42 ` Julien Grall
@ 2014-09-11 22:50 ` Julien Grall
2014-09-12 10:13 ` Ian Campbell
2014-09-12 10:19 ` Ian Campbell
0 siblings, 2 replies; 119+ messages in thread
From: Julien Grall @ 2014-09-11 22:50 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, tim, stefano.stabellini
Hi Ian,
On 09/09/14 11:42, Julien Grall wrote:
>>> - vgic_vcpu_inject_spi(d, irq);
>>> + vgic_vcpu_inject_spi(info->d, info->virq);
>>
>> Could this function take an irq_guest* or are there other callers?
>
> I haven't thought about this possibility. This is the only caller, so I
> will pass irq_guest.
Looking again to this code, I've defined the structure irq_guest
directly in arch/arm/irq.c. And I don't really want to move the
definition in header as all the use are within this single file.
I will stay with the current solution.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-09-11 22:50 ` Julien Grall
@ 2014-09-12 10:13 ` Ian Campbell
2014-09-12 10:19 ` Ian Campbell
1 sibling, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-12 10:13 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, stefano.stabellini
On Thu, 2014-09-11 at 15:50 -0700, Julien Grall wrote:
> Hi Ian,
>
> On 09/09/14 11:42, Julien Grall wrote:
> >>> - vgic_vcpu_inject_spi(d, irq);
> >>> + vgic_vcpu_inject_spi(info->d, info->virq);
> >>
> >> Could this function take an irq_guest* or are there other callers?
> >
> > I haven't thought about this possibility. This is the only caller, so I
> > will pass irq_guest.
>
> Looking again to this code, I've defined the structure irq_guest
> directly in arch/arm/irq.c. And I don't really want to move the
> definition in header as all the use are within this single file.
>
> I will stay with the current solution.
OK.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 06/21] xen/arm: Allow virq != irq
2014-09-11 22:50 ` Julien Grall
2014-09-12 10:13 ` Ian Campbell
@ 2014-09-12 10:19 ` Ian Campbell
1 sibling, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-12 10:19 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, stefano.stabellini
On Thu, 2014-09-11 at 15:50 -0700, Julien Grall wrote:
> Hi Ian,
>
> On 09/09/14 11:42, Julien Grall wrote:
> >>> - vgic_vcpu_inject_spi(d, irq);
> >>> + vgic_vcpu_inject_spi(info->d, info->virq);
> >>
> >> Could this function take an irq_guest* or are there other callers?
> >
> > I haven't thought about this possibility. This is the only caller, so I
> > will pass irq_guest.
>
> Looking again to this code, I've defined the structure irq_guest
> directly in arch/arm/irq.c. And I don't really want to move the
> definition in header as all the use are within this single file.
>
> I will stay with the current solution.
OK
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 07/21] xen/arm: route_irq_to_guest: Check validity of the IRQ
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (5 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 06/21] xen/arm: Allow virq != irq Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 14:56 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later Julien Grall
` (14 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
Currently Xen only supports SPIs routing for guest, add a function
is_assign_irq to check if we can assign a given IRQ to the guest.
Secondly, make sure the vIRQ (which is currently the same as the pIRQ) is not
the greater that the number of IRQs handle to the vGIC.
Finally, desc->arch.type which contains the IRQ type (i.e level/edge) must
be correctly configured before. The IRQ type won't be configure when:
- the device has been blacklist for the current platform
- the IRQ has not been describe in the device tree
I think we can safely assume that a user won't never ask to route
as such IRQ to the guest.
Also, use XENLOG_G_ERR in the error message within the function as it will
be later called from a guest.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Rename is_routable_irq into is_assignable_irq
- Check if the IRQ is not greater than the number handled by the
number of IRQs handled by the gic
- Move is_assignable_irq in irq.c rather than defining in the
header irq.h
- Retrieve the irq descriptor after checking the validity of the
IRQ
- vgic_num_irqs has been moved in a separate patch
- Fix the irq check against vgic_num_irqs
- Use virq instead of irq for vGIC sanity check
---
xen/arch/arm/irq.c | 39 ++++++++++++++++++++++++++++++++++++---
xen/include/asm-arm/irq.h | 2 ++
2 files changed, 38 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 830832c..7eeb8dd 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -379,6 +379,15 @@ err:
return rc;
}
+bool_t is_assignable_irq(unsigned int irq)
+{
+ /* For now, we can only route SPIs to the guest */
+ return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
+}
+
+/* Route an IRQ to a specific guest.
+ * For now only SPIs are assignabled to the guest.
+ */
int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned int irq, const char * devname)
{
@@ -388,6 +397,22 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned long flags;
int retval = 0;
+ if ( !is_assignable_irq(irq) )
+ {
+ dprintk(XENLOG_G_ERR, "the IRQ%u is not routable\n", irq);
+ return -EINVAL;
+ }
+
+ desc = irq_to_desc(irq);
+
+ if ( virq >= vgic_num_irqs(d) )
+ {
+ dprintk(XENLOG_G_ERR,
+ "the IRQ number %u is too high for domain %u (max = %u)\n",
+ irq, d->domain_id, vgic_num_irqs(d));
+ return -EINVAL;
+ }
+
action = xmalloc(struct irqaction);
if ( !action )
return -ENOMEM;
@@ -408,6 +433,14 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
spin_lock_irqsave(&desc->lock, flags);
+ if ( desc->arch.type == DT_IRQ_TYPE_INVALID )
+ {
+ dprintk(XENLOG_G_ERR, "IRQ %u has not been configured\n",
+ irq);
+ retval = -EIO;
+ goto out;
+ }
+
/* If the IRQ is already used by someone
* - If it's the same domain -> Xen doesn't need to update the IRQ desc
* - Otherwise -> For now, don't allow the IRQ to be shared between
@@ -421,10 +454,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
goto out;
if ( test_bit(_IRQ_GUEST, &desc->status) )
- printk(XENLOG_ERR "ERROR: IRQ %u is already used by domain %u\n",
- irq, ad->domain_id);
+ dprintk(XENLOG_G_ERR, "IRQ %u is already used by domain %u\n",
+ irq, ad->domain_id);
else
- printk(XENLOG_ERR "ERROR: IRQ %u is already used by Xen\n", irq);
+ dprintk(XENLOG_G_ERR, "IRQ %u is already used by Xen\n", irq);
retval = -EBUSY;
goto out;
}
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index 9bc3492..a7174f3 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -37,6 +37,8 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq);
#define domain_pirq_to_irq(d, pirq) (pirq)
+bool_t is_assignable_irq(unsigned int irq);
+
void init_IRQ(void);
void init_secondary_IRQ(void);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 07/21] xen/arm: route_irq_to_guest: Check validity of the IRQ
2014-07-31 15:00 ` [PATCH v2 07/21] xen/arm: route_irq_to_guest: Check validity of the IRQ Julien Grall
@ 2014-08-06 14:56 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 14:56 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> Currently Xen only supports SPIs routing for guest, add a function
> is_assign_irq to check if we can assign a given IRQ to the guest.
^ is_assignable_irq
> Secondly, make sure the vIRQ (which is currently the same as the pIRQ) is not
> the greater that the number of IRQs handle to the vGIC.
>
> Finally, desc->arch.type which contains the IRQ type (i.e level/edge) must
> be correctly configured before. The IRQ type won't be configure when:
> - the device has been blacklist for the current platform
> - the IRQ has not been describe in the device tree
>
> I think we can safely assume that a user won't never ask to route
> as such IRQ to the guest.
>
> Also, use XENLOG_G_ERR in the error message within the function as it will
> be later called from a guest.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
> Changes in v2:
> - Rename is_routable_irq into is_assignable_irq
> - Check if the IRQ is not greater than the number handled by the
> number of IRQs handled by the gic
> - Move is_assignable_irq in irq.c rather than defining in the
> header irq.h
> - Retrieve the irq descriptor after checking the validity of the
> IRQ
> - vgic_num_irqs has been moved in a separate patch
> - Fix the irq check against vgic_num_irqs
> - Use virq instead of irq for vGIC sanity check
> ---
> xen/arch/arm/irq.c | 39 ++++++++++++++++++++++++++++++++++++---
> xen/include/asm-arm/irq.h | 2 ++
> 2 files changed, 38 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 830832c..7eeb8dd 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -379,6 +379,15 @@ err:
> return rc;
> }
>
> +bool_t is_assignable_irq(unsigned int irq)
> +{
> + /* For now, we can only route SPIs to the guest */
> + return ((irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines()));
> +}
> +
> +/* Route an IRQ to a specific guest.
> + * For now only SPIs are assignabled to the guest.
> + */
> int route_irq_to_guest(struct domain *d, unsigned int virq,
> unsigned int irq, const char * devname)
> {
> @@ -388,6 +397,22 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> unsigned long flags;
> int retval = 0;
>
> + if ( !is_assignable_irq(irq) )
> + {
> + dprintk(XENLOG_G_ERR, "the IRQ%u is not routable\n", irq);
> + return -EINVAL;
> + }
> +
> + desc = irq_to_desc(irq);
> +
> + if ( virq >= vgic_num_irqs(d) )
> + {
> + dprintk(XENLOG_G_ERR,
> + "the IRQ number %u is too high for domain %u (max = %u)\n",
> + irq, d->domain_id, vgic_num_irqs(d));
> + return -EINVAL;
> + }
> +
> action = xmalloc(struct irqaction);
> if ( !action )
> return -ENOMEM;
> @@ -408,6 +433,14 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
>
> spin_lock_irqsave(&desc->lock, flags);
>
> + if ( desc->arch.type == DT_IRQ_TYPE_INVALID )
> + {
> + dprintk(XENLOG_G_ERR, "IRQ %u has not been configured\n",
> + irq);
> + retval = -EIO;
> + goto out;
> + }
> +
> /* If the IRQ is already used by someone
> * - If it's the same domain -> Xen doesn't need to update the IRQ desc
> * - Otherwise -> For now, don't allow the IRQ to be shared between
> @@ -421,10 +454,10 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> goto out;
>
> if ( test_bit(_IRQ_GUEST, &desc->status) )
> - printk(XENLOG_ERR "ERROR: IRQ %u is already used by domain %u\n",
> - irq, ad->domain_id);
> + dprintk(XENLOG_G_ERR, "IRQ %u is already used by domain %u\n",
> + irq, ad->domain_id);
> else
> - printk(XENLOG_ERR "ERROR: IRQ %u is already used by Xen\n", irq);
> + dprintk(XENLOG_G_ERR, "IRQ %u is already used by Xen\n", irq);
> retval = -EBUSY;
> goto out;
> }
> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
> index 9bc3492..a7174f3 100644
> --- a/xen/include/asm-arm/irq.h
> +++ b/xen/include/asm-arm/irq.h
> @@ -37,6 +37,8 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq, int is_fiq);
>
> #define domain_pirq_to_irq(d, pirq) (pirq)
>
> +bool_t is_assignable_irq(unsigned int irq);
> +
> void init_IRQ(void);
> void init_secondary_IRQ(void);
>
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (6 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 07/21] xen/arm: route_irq_to_guest: Check validity of the IRQ Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 15:35 ` Stefano Stabellini
` (4 more replies)
2014-07-31 15:00 ` [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying Julien Grall
` (13 subsequent siblings)
21 siblings, 5 replies; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, tim, Julien Grall, Ian Jackson, stefano.stabellini,
Jan Beulich, Daniel De Graaf
The virtual GIC may differ between each guest (number of SPIs, emulate GIC
version...). Those informations may not be know when the domain is created
(for instance in case of migration). Therefore, move the VGIC initialization
in a separate function.
Introduce a new DOMCTL for ARM to configure the domain. This has to be called
before setting the maximum number of VCPUs for the guest.
For the moment, only configure the number SPIs. New informations could be
added later.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Cc: Jan Beulich <jbeulich@suse.com>
---
Changes in v2:
- Patch added
---
tools/libxc/xc_domain.c | 12 ++++++++++++
tools/libxc/xenctrl.h | 4 ++++
tools/libxl/libxl_arch.h | 3 +++
tools/libxl/libxl_arm.c | 19 +++++++++++++++++++
tools/libxl/libxl_dom.c | 4 ++++
tools/libxl/libxl_x86.c | 7 +++++++
xen/arch/arm/domain.c | 28 ++++++++++++++++++++++------
xen/arch/arm/domctl.c | 11 +++++++++++
xen/arch/arm/setup.c | 10 ++++++++--
xen/arch/arm/vgic.c | 10 +++++-----
xen/include/asm-arm/domain.h | 6 ++++++
xen/include/asm-arm/vgic.h | 4 +++-
xen/include/public/domctl.h | 14 ++++++++++++++
xen/xsm/flask/hooks.c | 3 +++
xen/xsm/flask/policy/access_vectors | 2 ++
15 files changed, 123 insertions(+), 14 deletions(-)
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index 27fe3b6..1348905 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -48,6 +48,18 @@ int xc_domain_create(xc_interface *xch,
return 0;
}
+#if defined(__arm__) || defined(__arch64__)
+int xc_domain_configure(xc_interface *xch, uint32_t domid,
+ uint32_t nr_spis)
+{
+ DECLARE_DOMCTL;
+ domctl.cmd = XEN_DOMCTL_configure_domain;
+ domctl.domain = (domid_t)domid;
+ domctl.u.configuredomain.nr_spis = nr_spis;
+ return do_domctl(xch, &domctl);
+}
+#endif
+
int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
xen_pfn_t start_pfn, xen_pfn_t nr_pfns)
{
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index 5beb846..cdda4e5 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -482,6 +482,10 @@ int xc_domain_create(xc_interface *xch,
uint32_t flags,
uint32_t *pdomid);
+#if defined(__arm__) || defined(__aarch64__)
+int xc_domain_configure(xc_interface *xch, uint32_t domid,
+ uint32_t nr_spis);
+#endif
/* Functions to produce a dump of a given domain
* xc_domain_dumpcore - produces a dump to a specified file
diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
index d3bc136..454f8db 100644
--- a/tools/libxl/libxl_arch.h
+++ b/tools/libxl/libxl_arch.h
@@ -16,6 +16,9 @@
#define LIBXL_ARCH_H
/* arch specific internal domain creation function */
+int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
+ libxl__domain_build_state *state,
+ uint32_t domid);
int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
uint32_t domid);
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index e19e2f4..b0491c3 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -23,6 +23,25 @@
#define DT_IRQ_TYPE_LEVEL_HIGH 0x00000004
#define DT_IRQ_TYPE_LEVEL_LOW 0x00000008
+int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
+ libxl__domain_build_state *state,
+ uint32_t domid)
+{
+ uint32_t nr_spis = 0;
+
+ nr_spis += d_config->b_info.num_irqs;
+
+ LOG(DEBUG, "Allocate %u SPIs\n", nr_spis);
+
+ if (xc_domain_configure(CTX->xch, domid, nr_spis) != 0) {
+ LOG(ERROR, "Couldn't configure the domain");
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+
int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
uint32_t domid)
{
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index c944804..a0e4e34 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -235,6 +235,10 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
char *xs_domid, *con_domid;
int rc;
+ rc = libxl__arch_domain_create_pre(gc, d_config, state, domid);
+ if (rc != 0)
+ return rc;
+
if (xc_domain_max_vcpus(ctx->xch, domid, info->max_vcpus) != 0) {
LOG(ERROR, "Couldn't set max vcpu count");
return ERROR_FAIL;
diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
index 7589060..0abc1aa 100644
--- a/tools/libxl/libxl_x86.c
+++ b/tools/libxl/libxl_x86.c
@@ -244,6 +244,13 @@ static int libxl__e820_alloc(libxl__gc *gc, uint32_t domid,
return 0;
}
+int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
+ libxl__domain_build_state *state,
+ uint32_t domid)
+{
+ return 0;
+}
+
int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
uint32_t domid)
{
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index aa4a8d7..fc97f8c 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -472,6 +472,13 @@ int vcpu_initialise(struct vcpu *v)
if ( is_idle_vcpu(v) )
return rc;
+ /* We can't initialize the VCPU if the VGIC has not been correctly
+ * initialized.
+ */
+ rc = -ENXIO;
+ if ( !domain_vgic_is_initialized(v->domain) )
+ goto fail;
+
v->arch.sctlr = SCTLR_GUEST_INIT;
/*
@@ -534,12 +541,6 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags)
if ( (rc = p2m_alloc_table(d)) != 0 )
goto fail;
- if ( (rc = gicv_setup(d)) != 0 )
- goto fail;
-
- if ( (rc = domain_vgic_init(d)) != 0 )
- goto fail;
-
if ( (rc = domain_vtimer_init(d)) != 0 )
goto fail;
@@ -580,6 +581,21 @@ void arch_domain_destroy(struct domain *d)
free_xenheap_page(d->shared_info);
}
+int domain_configure_vgic(struct domain *d, unsigned int nr_spis)
+{
+ int rc;
+
+ if ( (rc = gicv_setup(d)) != 0 )
+ return rc;
+
+ if ( (rc = domain_vgic_init(d, nr_spis)) != 0 )
+ return rc;
+
+ d->arch.vgic.initialized = 1;
+
+ return 0;
+}
+
static int is_guest_pv32_psr(uint32_t psr)
{
switch (psr & PSR_MODE_MASK)
diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index 45974e7..bab92b2 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -30,6 +30,17 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
return p2m_cache_flush(d, s, e);
}
+ case XEN_DOMCTL_configure_domain:
+ {
+ if ( domain_vgic_is_initialized(d) )
+ return -EBUSY;
+
+ /* Sanity check on the number of SPIs */
+ if ( domctl->u.configuredomain.nr_spis > (gic_number_lines() - 32) )
+ return -EINVAL;
+
+ return domain_configure_vgic(d, domctl->u.configuredomain.nr_spis);
+ }
default:
return subarch_do_domctl(domctl, d, u_domctl);
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 446b4dc..2be7dd1 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -840,8 +840,14 @@ void __init start_xen(unsigned long boot_phys_offset,
/* Create initial domain 0. */
dom0 = domain_create(0, 0, 0);
- if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
- panic("Error creating domain 0");
+ if ( IS_ERR(dom0) )
+ panic("Error creating domain 0");
+
+ if ( domain_configure_vgic(dom0, gic_number_lines() - 32) )
+ panic("Error configure the vgic for domain 0");
+
+ if ( alloc_dom0_vcpu0(dom0) == NULL )
+ panic("Error create vcpu0 for domain 0");
dom0->is_privileged = 1;
dom0->target = NULL;
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 17cde7a..a037ecc 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -60,16 +60,16 @@ static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
p->irq = virq;
}
-int domain_vgic_init(struct domain *d)
+int domain_vgic_init(struct domain *d, unsigned int nr_spis)
{
int i;
d->arch.vgic.ctlr = 0;
- if ( is_hardware_domain(d) )
- d->arch.vgic.nr_spis = gic_number_lines() - 32;
- else
- d->arch.vgic.nr_spis = 0; /* We don't need SPIs for the guest */
+ /* The number of SPIs as to be aligned to 32 see
+ * GICD_TYPER.ITLinesNumber definition
+ */
+ d->arch.vgic.nr_spis = ROUNDUP(nr_spis, 32);
switch ( gic_hw_version() )
{
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 5719fe5..44727b2 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -76,6 +76,10 @@ struct arch_domain
} virt_timer_base;
struct {
+ /* The VGIC initialization is defered to let the toolstack
+ * configure the emulated GIC (such as number of SPIs, version...)
+ */
+ bool_t initialized;
/* GIC HW version specific vGIC driver handler */
const struct vgic_ops *handler;
/*
@@ -241,6 +245,8 @@ struct arch_vcpu
void vcpu_show_execution_state(struct vcpu *);
void vcpu_show_registers(const struct vcpu *);
+int domain_configure_vgic(struct domain *d, unsigned int spis_number);
+
#endif /* __ASM_DOMAIN_H__ */
/*
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 5ddc681..84ae441 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -148,6 +148,8 @@ static inline void vgic_byte_write(uint32_t *reg, uint32_t var, int offset)
*reg |= var;
}
+#define domain_vgic_is_initialized(d) ((d)->arch.vgic.initialized)
+
enum gic_sgi_mode;
/*
@@ -158,7 +160,7 @@ enum gic_sgi_mode;
#define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
-extern int domain_vgic_init(struct domain *d);
+extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
extern void domain_vgic_free(struct domain *d);
extern int vcpu_vgic_init(struct vcpu *v);
extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 5b11bbf..b5f2ed7 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -67,6 +67,16 @@ struct xen_domctl_createdomain {
typedef struct xen_domctl_createdomain xen_domctl_createdomain_t;
DEFINE_XEN_GUEST_HANDLE(xen_domctl_createdomain_t);
+#if defined(__arm__) || defined(__aarch64__)
+/* XEN_DOMCTL_configure_domain */
+struct xen_domctl_configuredomain {
+ /* IN parameters */
+ uint32_t nr_spis;
+};
+typedef struct xen_domctl_configuredomain xen_domctl_configuredomain_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_configuredomain_t);
+#endif
+
/* XEN_DOMCTL_getdomaininfo */
struct xen_domctl_getdomaininfo {
/* OUT variables. */
@@ -1008,6 +1018,7 @@ struct xen_domctl {
#define XEN_DOMCTL_cacheflush 71
#define XEN_DOMCTL_get_vcpu_msrs 72
#define XEN_DOMCTL_set_vcpu_msrs 73
+#define XEN_DOMCTL_configure_domain 74
#define XEN_DOMCTL_gdbsx_guestmemio 1000
#define XEN_DOMCTL_gdbsx_pausevcpu 1001
#define XEN_DOMCTL_gdbsx_unpausevcpu 1002
@@ -1016,6 +1027,9 @@ struct xen_domctl {
domid_t domain;
union {
struct xen_domctl_createdomain createdomain;
+#if defined(__arm__) || defined(__aarch64__)
+ struct xen_domctl_configuredomain configuredomain;
+#endif
struct xen_domctl_getdomaininfo getdomaininfo;
struct xen_domctl_getmemlist getmemlist;
struct xen_domctl_getpageframeinfo getpageframeinfo;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index f2f59ea..4759461 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -715,6 +715,9 @@ static int flask_domctl(struct domain *d, int cmd)
case XEN_DOMCTL_cacheflush:
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__CACHEFLUSH);
+ case XEN_DOMCTL_configure_domain:
+ return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__CONFIGURE_DOMAIN);
+
default:
printk("flask_domctl: Unknown op %d\n", cmd);
return -EPERM;
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index 32371a9..33eec66 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -200,6 +200,8 @@ class domain2
cacheflush
# Creation of the hardware domain when it is not dom0
create_hardware_domain
+# XEN_DOMCTL_configure_domain
+ configure_domain
}
# Similar to class domain, but primarily contains domctls related to HVM domains
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-07-31 15:00 ` [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later Julien Grall
@ 2014-08-06 15:35 ` Stefano Stabellini
2014-09-09 13:35 ` Ian Campbell
2014-09-11 23:01 ` Julien Grall
2014-08-06 17:06 ` Daniel De Graaf
` (3 subsequent siblings)
4 siblings, 2 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 15:35 UTC (permalink / raw)
To: Julien Grall
Cc: ian.campbell, tim, Ian Jackson, stefano.stabellini, Jan Beulich,
xen-devel, Daniel De Graaf
On Thu, 31 Jul 2014, Julien Grall wrote:
> The virtual GIC may differ between each guest (number of SPIs, emulate GIC
> version...). Those informations may not be know when the domain is created
> (for instance in case of migration). Therefore, move the VGIC initialization
> in a separate function.
>
> Introduce a new DOMCTL for ARM to configure the domain. This has to be called
> before setting the maximum number of VCPUs for the guest.
>
> For the moment, only configure the number SPIs. New informations could be
> added later.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> Cc: Jan Beulich <jbeulich@suse.com>
>
> ---
> Changes in v2:
> - Patch added
> ---
> tools/libxc/xc_domain.c | 12 ++++++++++++
> tools/libxc/xenctrl.h | 4 ++++
> tools/libxl/libxl_arch.h | 3 +++
> tools/libxl/libxl_arm.c | 19 +++++++++++++++++++
> tools/libxl/libxl_dom.c | 4 ++++
> tools/libxl/libxl_x86.c | 7 +++++++
> xen/arch/arm/domain.c | 28 ++++++++++++++++++++++------
> xen/arch/arm/domctl.c | 11 +++++++++++
> xen/arch/arm/setup.c | 10 ++++++++--
> xen/arch/arm/vgic.c | 10 +++++-----
> xen/include/asm-arm/domain.h | 6 ++++++
> xen/include/asm-arm/vgic.h | 4 +++-
> xen/include/public/domctl.h | 14 ++++++++++++++
> xen/xsm/flask/hooks.c | 3 +++
> xen/xsm/flask/policy/access_vectors | 2 ++
> 15 files changed, 123 insertions(+), 14 deletions(-)
>
> diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
> index 27fe3b6..1348905 100644
> --- a/tools/libxc/xc_domain.c
> +++ b/tools/libxc/xc_domain.c
> @@ -48,6 +48,18 @@ int xc_domain_create(xc_interface *xch,
> return 0;
> }
>
> +#if defined(__arm__) || defined(__arch64__)
> +int xc_domain_configure(xc_interface *xch, uint32_t domid,
> + uint32_t nr_spis)
Given that we'll likely add new fields to xen_domctl_configuredomain, I
think it is best if we pass a struct domain_configure to
xc_domain_configure instead of nr_spis. The struct we pass to
xc_domain_configure doesn't have to be identical to struct
xen_domctl_configuredomain, but at the moment it would be.
> +{
> + DECLARE_DOMCTL;
> + domctl.cmd = XEN_DOMCTL_configure_domain;
> + domctl.domain = (domid_t)domid;
> + domctl.u.configuredomain.nr_spis = nr_spis;
> + return do_domctl(xch, &domctl);
> +}
> +#endif
> +
> int xc_domain_cacheflush(xc_interface *xch, uint32_t domid,
> xen_pfn_t start_pfn, xen_pfn_t nr_pfns)
> {
> diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
> index 5beb846..cdda4e5 100644
> --- a/tools/libxc/xenctrl.h
> +++ b/tools/libxc/xenctrl.h
> @@ -482,6 +482,10 @@ int xc_domain_create(xc_interface *xch,
> uint32_t flags,
> uint32_t *pdomid);
>
> +#if defined(__arm__) || defined(__aarch64__)
> +int xc_domain_configure(xc_interface *xch, uint32_t domid,
> + uint32_t nr_spis);
> +#endif
>
> /* Functions to produce a dump of a given domain
> * xc_domain_dumpcore - produces a dump to a specified file
> diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
> index d3bc136..454f8db 100644
> --- a/tools/libxl/libxl_arch.h
> +++ b/tools/libxl/libxl_arch.h
> @@ -16,6 +16,9 @@
> #define LIBXL_ARCH_H
>
> /* arch specific internal domain creation function */
> +int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> + libxl__domain_build_state *state,
> + uint32_t domid);
> int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> uint32_t domid);
>
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> index e19e2f4..b0491c3 100644
> --- a/tools/libxl/libxl_arm.c
> +++ b/tools/libxl/libxl_arm.c
> @@ -23,6 +23,25 @@
> #define DT_IRQ_TYPE_LEVEL_HIGH 0x00000004
> #define DT_IRQ_TYPE_LEVEL_LOW 0x00000008
>
> +int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> + libxl__domain_build_state *state,
> + uint32_t domid)
> +{
> + uint32_t nr_spis = 0;
> +
> + nr_spis += d_config->b_info.num_irqs;
> +
> + LOG(DEBUG, "Allocate %u SPIs\n", nr_spis);
> +
> + if (xc_domain_configure(CTX->xch, domid, nr_spis) != 0) {
> + LOG(ERROR, "Couldn't configure the domain");
> + return ERROR_FAIL;
> + }
> +
> + return 0;
> +}
> +
> +
> int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> uint32_t domid)
> {
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index c944804..a0e4e34 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -235,6 +235,10 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
> char *xs_domid, *con_domid;
> int rc;
>
> + rc = libxl__arch_domain_create_pre(gc, d_config, state, domid);
> + if (rc != 0)
> + return rc;
> +
> if (xc_domain_max_vcpus(ctx->xch, domid, info->max_vcpus) != 0) {
> LOG(ERROR, "Couldn't set max vcpu count");
> return ERROR_FAIL;
> diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
> index 7589060..0abc1aa 100644
> --- a/tools/libxl/libxl_x86.c
> +++ b/tools/libxl/libxl_x86.c
> @@ -244,6 +244,13 @@ static int libxl__e820_alloc(libxl__gc *gc, uint32_t domid,
> return 0;
> }
>
> +int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> + libxl__domain_build_state *state,
> + uint32_t domid)
> +{
> + return 0;
> +}
> +
> int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> uint32_t domid)
> {
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index aa4a8d7..fc97f8c 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -472,6 +472,13 @@ int vcpu_initialise(struct vcpu *v)
> if ( is_idle_vcpu(v) )
> return rc;
>
> + /* We can't initialize the VCPU if the VGIC has not been correctly
> + * initialized.
> + */
> + rc = -ENXIO;
> + if ( !domain_vgic_is_initialized(v->domain) )
> + goto fail;
> +
> v->arch.sctlr = SCTLR_GUEST_INIT;
>
> /*
> @@ -534,12 +541,6 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags)
> if ( (rc = p2m_alloc_table(d)) != 0 )
> goto fail;
>
> - if ( (rc = gicv_setup(d)) != 0 )
> - goto fail;
> -
> - if ( (rc = domain_vgic_init(d)) != 0 )
> - goto fail;
> -
> if ( (rc = domain_vtimer_init(d)) != 0 )
> goto fail;
>
> @@ -580,6 +581,21 @@ void arch_domain_destroy(struct domain *d)
> free_xenheap_page(d->shared_info);
> }
>
> +int domain_configure_vgic(struct domain *d, unsigned int nr_spis)
> +{
> + int rc;
> +
> + if ( (rc = gicv_setup(d)) != 0 )
> + return rc;
> +
> + if ( (rc = domain_vgic_init(d, nr_spis)) != 0 )
> + return rc;
> +
> + d->arch.vgic.initialized = 1;
> +
> + return 0;
> +}
> +
> static int is_guest_pv32_psr(uint32_t psr)
> {
> switch (psr & PSR_MODE_MASK)
> diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
> index 45974e7..bab92b2 100644
> --- a/xen/arch/arm/domctl.c
> +++ b/xen/arch/arm/domctl.c
> @@ -30,6 +30,17 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
>
> return p2m_cache_flush(d, s, e);
> }
> + case XEN_DOMCTL_configure_domain:
> + {
> + if ( domain_vgic_is_initialized(d) )
> + return -EBUSY;
Given that XEN_DOMCTL_configure_domain should be called exactly once at
domain creation, instead of introducing domain_vgic_is_initialized, I
would make sure that XEN_DOMCTL_configure_domain hasn't been called for
this domain before. In other words, I would make this check more
generic, rather than vgic specific.
> + /* Sanity check on the number of SPIs */
> + if ( domctl->u.configuredomain.nr_spis > (gic_number_lines() - 32) )
> + return -EINVAL;
> +
> + return domain_configure_vgic(d, domctl->u.configuredomain.nr_spis);
> + }
>
> default:
> return subarch_do_domctl(domctl, d, u_domctl);
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 446b4dc..2be7dd1 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -840,8 +840,14 @@ void __init start_xen(unsigned long boot_phys_offset,
>
> /* Create initial domain 0. */
> dom0 = domain_create(0, 0, 0);
> - if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
> - panic("Error creating domain 0");
> + if ( IS_ERR(dom0) )
> + panic("Error creating domain 0");
> +
> + if ( domain_configure_vgic(dom0, gic_number_lines() - 32) )
> + panic("Error configure the vgic for domain 0");
> +
> + if ( alloc_dom0_vcpu0(dom0) == NULL )
> + panic("Error create vcpu0 for domain 0");
>
> dom0->is_privileged = 1;
> dom0->target = NULL;
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 17cde7a..a037ecc 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -60,16 +60,16 @@ static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
> p->irq = virq;
> }
>
> -int domain_vgic_init(struct domain *d)
> +int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> {
> int i;
>
> d->arch.vgic.ctlr = 0;
>
> - if ( is_hardware_domain(d) )
> - d->arch.vgic.nr_spis = gic_number_lines() - 32;
> - else
> - d->arch.vgic.nr_spis = 0; /* We don't need SPIs for the guest */
> + /* The number of SPIs as to be aligned to 32 see
> + * GICD_TYPER.ITLinesNumber definition
> + */
> + d->arch.vgic.nr_spis = ROUNDUP(nr_spis, 32);
>
> switch ( gic_hw_version() )
> {
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 5719fe5..44727b2 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -76,6 +76,10 @@ struct arch_domain
> } virt_timer_base;
>
> struct {
> + /* The VGIC initialization is defered to let the toolstack
> + * configure the emulated GIC (such as number of SPIs, version...)
> + */
> + bool_t initialized;
> /* GIC HW version specific vGIC driver handler */
> const struct vgic_ops *handler;
> /*
> @@ -241,6 +245,8 @@ struct arch_vcpu
> void vcpu_show_execution_state(struct vcpu *);
> void vcpu_show_registers(const struct vcpu *);
>
> +int domain_configure_vgic(struct domain *d, unsigned int spis_number);
> +
> #endif /* __ASM_DOMAIN_H__ */
>
> /*
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 5ddc681..84ae441 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -148,6 +148,8 @@ static inline void vgic_byte_write(uint32_t *reg, uint32_t var, int offset)
> *reg |= var;
> }
>
> +#define domain_vgic_is_initialized(d) ((d)->arch.vgic.initialized)
> +
> enum gic_sgi_mode;
>
> /*
> @@ -158,7 +160,7 @@ enum gic_sgi_mode;
>
> #define vgic_num_irqs(d) ((d)->arch.vgic.nr_spis + 32)
>
> -extern int domain_vgic_init(struct domain *d);
> +extern int domain_vgic_init(struct domain *d, unsigned int nr_spis);
> extern void domain_vgic_free(struct domain *d);
> extern int vcpu_vgic_init(struct vcpu *v);
> extern struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq);
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 5b11bbf..b5f2ed7 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -67,6 +67,16 @@ struct xen_domctl_createdomain {
> typedef struct xen_domctl_createdomain xen_domctl_createdomain_t;
> DEFINE_XEN_GUEST_HANDLE(xen_domctl_createdomain_t);
>
> +#if defined(__arm__) || defined(__aarch64__)
> +/* XEN_DOMCTL_configure_domain */
> +struct xen_domctl_configuredomain {
> + /* IN parameters */
> + uint32_t nr_spis;
> +};
> +typedef struct xen_domctl_configuredomain xen_domctl_configuredomain_t;
> +DEFINE_XEN_GUEST_HANDLE(xen_domctl_configuredomain_t);
> +#endif
> +
> /* XEN_DOMCTL_getdomaininfo */
> struct xen_domctl_getdomaininfo {
> /* OUT variables. */
> @@ -1008,6 +1018,7 @@ struct xen_domctl {
> #define XEN_DOMCTL_cacheflush 71
> #define XEN_DOMCTL_get_vcpu_msrs 72
> #define XEN_DOMCTL_set_vcpu_msrs 73
> +#define XEN_DOMCTL_configure_domain 74
> #define XEN_DOMCTL_gdbsx_guestmemio 1000
> #define XEN_DOMCTL_gdbsx_pausevcpu 1001
> #define XEN_DOMCTL_gdbsx_unpausevcpu 1002
> @@ -1016,6 +1027,9 @@ struct xen_domctl {
> domid_t domain;
> union {
> struct xen_domctl_createdomain createdomain;
> +#if defined(__arm__) || defined(__aarch64__)
> + struct xen_domctl_configuredomain configuredomain;
> +#endif
> struct xen_domctl_getdomaininfo getdomaininfo;
> struct xen_domctl_getmemlist getmemlist;
> struct xen_domctl_getpageframeinfo getpageframeinfo;
> diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
> index f2f59ea..4759461 100644
> --- a/xen/xsm/flask/hooks.c
> +++ b/xen/xsm/flask/hooks.c
> @@ -715,6 +715,9 @@ static int flask_domctl(struct domain *d, int cmd)
> case XEN_DOMCTL_cacheflush:
> return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__CACHEFLUSH);
>
> + case XEN_DOMCTL_configure_domain:
> + return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__CONFIGURE_DOMAIN);
> +
> default:
> printk("flask_domctl: Unknown op %d\n", cmd);
> return -EPERM;
> diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
> index 32371a9..33eec66 100644
> --- a/xen/xsm/flask/policy/access_vectors
> +++ b/xen/xsm/flask/policy/access_vectors
> @@ -200,6 +200,8 @@ class domain2
> cacheflush
> # Creation of the hardware domain when it is not dom0
> create_hardware_domain
> +# XEN_DOMCTL_configure_domain
> + configure_domain
> }
>
> # Similar to class domain, but primarily contains domctls related to HVM domains
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-06 15:35 ` Stefano Stabellini
@ 2014-09-09 13:35 ` Ian Campbell
2014-09-09 18:57 ` Julien Grall
2014-09-11 23:01 ` Julien Grall
1 sibling, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:35 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Ian Jackson, Julien Grall, tim, stefano.stabellini, Jan Beulich,
xen-devel, Daniel De Graaf
On Wed, 2014-08-06 at 16:35 +0100, Stefano Stabellini wrote:
> > +#if defined(__arm__) || defined(__arch64__)
> > +int xc_domain_configure(xc_interface *xch, uint32_t domid,
> > + uint32_t nr_spis)
>
> Given that we'll likely add new fields to xen_domctl_configuredomain, I
> think it is best if we pass a struct domain_configure to
> xc_domain_configure instead of nr_spis. The struct we pass to
> xc_domain_configure doesn't have to be identical to struct
> xen_domctl_configuredomain, but at the moment it would be.
In that case you may as well inline the domctl call into
arch_domain_create_pre, which I don't think is a bad idea.
Ian
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-09-09 13:35 ` Ian Campbell
@ 2014-09-09 18:57 ` Julien Grall
2014-09-10 10:08 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-09 18:57 UTC (permalink / raw)
To: Ian Campbell, Stefano Stabellini
Cc: Ian Jackson, tim, stefano.stabellini, Jan Beulich, xen-devel,
Daniel De Graaf
Hi Ian,
On 09/09/14 06:35, Ian Campbell wrote:
> On Wed, 2014-08-06 at 16:35 +0100, Stefano Stabellini wrote:
>
>>> +#if defined(__arm__) || defined(__arch64__)
>>> +int xc_domain_configure(xc_interface *xch, uint32_t domid,
>>> + uint32_t nr_spis)
>>
>> Given that we'll likely add new fields to xen_domctl_configuredomain, I
>> think it is best if we pass a struct domain_configure to
>> xc_domain_configure instead of nr_spis. The struct we pass to
>> xc_domain_configure doesn't have to be identical to struct
>> xen_domctl_configuredomain, but at the moment it would be.
>
> In that case you may as well inline the domctl call into
> arch_domain_create_pre, which I don't think is a bad idea.
It will be hard. Libxl doesn't have access to the facility for create
bounce buffer...
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-09-09 18:57 ` Julien Grall
@ 2014-09-10 10:08 ` Ian Campbell
0 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-10 10:08 UTC (permalink / raw)
To: Julien Grall
Cc: Stefano Stabellini, Ian Jackson, tim, stefano.stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
On Tue, 2014-09-09 at 11:57 -0700, Julien Grall wrote:
> Hi Ian,
>
> On 09/09/14 06:35, Ian Campbell wrote:
> > On Wed, 2014-08-06 at 16:35 +0100, Stefano Stabellini wrote:
> >
> >>> +#if defined(__arm__) || defined(__arch64__)
> >>> +int xc_domain_configure(xc_interface *xch, uint32_t domid,
> >>> + uint32_t nr_spis)
> >>
> >> Given that we'll likely add new fields to xen_domctl_configuredomain, I
> >> think it is best if we pass a struct domain_configure to
> >> xc_domain_configure instead of nr_spis. The struct we pass to
> >> xc_domain_configure doesn't have to be identical to struct
> >> xen_domctl_configuredomain, but at the moment it would be.
> >
> > In that case you may as well inline the domctl call into
> > arch_domain_create_pre, which I don't think is a bad idea.
>
> It will be hard. Libxl doesn't have access to the facility for create
> bounce buffer...
Ah, nevermind then. Passing the struct as a ptr as Stefano suggests
would probably be good though.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-06 15:35 ` Stefano Stabellini
2014-09-09 13:35 ` Ian Campbell
@ 2014-09-11 23:01 ` Julien Grall
2014-09-12 10:14 ` Ian Campbell
1 sibling, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-11 23:01 UTC (permalink / raw)
To: Stefano Stabellini
Cc: ian.campbell, tim, Ian Jackson, stefano.stabellini, Jan Beulich,
xen-devel, Daniel De Graaf
Hi Stefano,
Sorry, I forgot to answer to this mail.
On 06/08/14 08:35, Stefano Stabellini wrote:
> On Thu, 31 Jul 2014, Julien Grall wrote:
>> The virtual GIC may differ between each guest (number of SPIs, emulate GIC
>> version...). Those informations may not be know when the domain is created
>> (for instance in case of migration). Therefore, move the VGIC initialization
>> in a separate function.
>>
>> Introduce a new DOMCTL for ARM to configure the domain. This has to be called
>> before setting the maximum number of VCPUs for the guest.
>>
>> For the moment, only configure the number SPIs. New informations could be
>> added later.
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
>> Cc: Jan Beulich <jbeulich@suse.com>
>>
>> ---
>> Changes in v2:
>> - Patch added
>> ---
>> tools/libxc/xc_domain.c | 12 ++++++++++++
>> tools/libxc/xenctrl.h | 4 ++++
>> tools/libxl/libxl_arch.h | 3 +++
>> tools/libxl/libxl_arm.c | 19 +++++++++++++++++++
>> tools/libxl/libxl_dom.c | 4 ++++
>> tools/libxl/libxl_x86.c | 7 +++++++
>> xen/arch/arm/domain.c | 28 ++++++++++++++++++++++------
>> xen/arch/arm/domctl.c | 11 +++++++++++
>> xen/arch/arm/setup.c | 10 ++++++++--
>> xen/arch/arm/vgic.c | 10 +++++-----
>> xen/include/asm-arm/domain.h | 6 ++++++
>> xen/include/asm-arm/vgic.h | 4 +++-
>> xen/include/public/domctl.h | 14 ++++++++++++++
>> xen/xsm/flask/hooks.c | 3 +++
>> xen/xsm/flask/policy/access_vectors | 2 ++
>> 15 files changed, 123 insertions(+), 14 deletions(-)
>>
>> diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
>> index 27fe3b6..1348905 100644
>> --- a/tools/libxc/xc_domain.c
>> +++ b/tools/libxc/xc_domain.c
>> @@ -48,6 +48,18 @@ int xc_domain_create(xc_interface *xch,
>> return 0;
>> }
>>
>> +#if defined(__arm__) || defined(__arch64__)
>> +int xc_domain_configure(xc_interface *xch, uint32_t domid,
>> + uint32_t nr_spis)
>
> Given that we'll likely add new fields to xen_domctl_configuredomain, I
> think it is best if we pass a struct domain_configure to
> xc_domain_configure instead of nr_spis. The struct we pass to
> xc_domain_configure doesn't have to be identical to struct
> xen_domctl_configuredomain, but at the moment it would be.
Ok. I will do.
[..]
>> diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
>> index 45974e7..bab92b2 100644
>> --- a/xen/arch/arm/domctl.c
>> +++ b/xen/arch/arm/domctl.c
>> @@ -30,6 +30,17 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
>>
>> return p2m_cache_flush(d, s, e);
>> }
>> + case XEN_DOMCTL_configure_domain:
>> + {
>> + if ( domain_vgic_is_initialized(d) )
>> + return -EBUSY;
>
> Given that XEN_DOMCTL_configure_domain should be called exactly once at
> domain creation, instead of introducing domain_vgic_is_initialized, I
> would make sure that XEN_DOMCTL_configure_domain hasn't been called for
> this domain before. In other words, I would make this check more
> generic, rather than vgic specific.
The VGIC initialization may have fail because there is not enough
memory. So it would be valid, even if it's stupid, to call this DOMCTL
twice.
domain_vgic_is_initialized is also used in vcpu_initialise to check that
the VGIC has effectively been initialized
I would keep this check for now.
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-09-11 23:01 ` Julien Grall
@ 2014-09-12 10:14 ` Ian Campbell
0 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-12 10:14 UTC (permalink / raw)
To: Julien Grall
Cc: Stefano Stabellini, Ian Jackson, tim, stefano.stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
On Thu, 2014-09-11 at 16:01 -0700, Julien Grall wrote:
> >> diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
> >> index 45974e7..bab92b2 100644
> >> --- a/xen/arch/arm/domctl.c
> >> +++ b/xen/arch/arm/domctl.c
> >> @@ -30,6 +30,17 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
> >>
> >> return p2m_cache_flush(d, s, e);
> >> }
> >> + case XEN_DOMCTL_configure_domain:
> >> + {
> >> + if ( domain_vgic_is_initialized(d) )
> >> + return -EBUSY;
> >
> > Given that XEN_DOMCTL_configure_domain should be called exactly once at
> > domain creation, instead of introducing domain_vgic_is_initialized, I
> > would make sure that XEN_DOMCTL_configure_domain hasn't been called for
> > this domain before. In other words, I would make this check more
> > generic, rather than vgic specific.
>
> The VGIC initialization may have fail because there is not enough
> memory. So it would be valid, even if it's stupid, to call this DOMCTL
> twice.
I think there was an implied "successfully" in what Stefano said.
That said it's hard to imagine that a toolstack would retry on failure
of such a hypercall, rather than aborting the build entirely (and
perhaps trying again at that level)
> domain_vgic_is_initialized is also used in vcpu_initialise to check that
> the VGIC has effectively been initialized
>
> I would keep this check for now.
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-07-31 15:00 ` [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later Julien Grall
2014-08-06 15:35 ` Stefano Stabellini
@ 2014-08-06 17:06 ` Daniel De Graaf
2014-08-29 13:09 ` Andrii Tseglytskyi
` (2 subsequent siblings)
4 siblings, 0 replies; 119+ messages in thread
From: Daniel De Graaf @ 2014-08-06 17:06 UTC (permalink / raw)
To: Julien Grall, xen-devel
Cc: Ian Jackson, stefano.stabellini, tim, ian.campbell, Jan Beulich
On 07/31/2014 11:00 AM, Julien Grall wrote:
> The virtual GIC may differ between each guest (number of SPIs, emulate GIC
> version...). Those informations may not be know when the domain is created
> (for instance in case of migration). Therefore, move the VGIC initialization
> in a separate function.
>
> Introduce a new DOMCTL for ARM to configure the domain. This has to be called
> before setting the maximum number of VCPUs for the guest.
>
> For the moment, only configure the number SPIs. New informations could be
> added later.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> Cc: Jan Beulich <jbeulich@suse.com>
For the XSM parts:
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-07-31 15:00 ` [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later Julien Grall
2014-08-06 15:35 ` Stefano Stabellini
2014-08-06 17:06 ` Daniel De Graaf
@ 2014-08-29 13:09 ` Andrii Tseglytskyi
2014-08-29 18:57 ` Julien Grall
2014-09-09 13:37 ` Ian Campbell
[not found] ` <CAAHg+HhhsZonrEDdHET93dy=dR1+YF-VPGJ=VwB20RRxWqdSYA@mail.gmail.com>
4 siblings, 1 reply; 119+ messages in thread
From: Andrii Tseglytskyi @ 2014-08-29 13:09 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Tim Deegan, Ian Jackson, Stefano Stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
Hi Julien,
If I need to use 1 to 1 IRQ mapping in domU, I got an error in the
following code:
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> index e19e2f4..b0491c3 100644
> --- a/tools/libxl/libxl_arm.c
> +++ b/tools/libxl/libxl_arm.c
> @@ -23,6 +23,25 @@
> #define DT_IRQ_TYPE_LEVEL_HIGH 0x00000004
> #define DT_IRQ_TYPE_LEVEL_LOW 0x00000008
>
> +int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> + libxl__domain_build_state *state,
> + uint32_t domid)
> +{
> + uint32_t nr_spis = 0;
> +
> + nr_spis += d_config->b_info.num_irqs;
I have 4 IRQS for domU, they numbers are 53, 173, 174. With this
change there will be 3 SPIs for domU, and below code will configure
vgic for 32 lines.
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 17cde7a..a037ecc 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -60,16 +60,16 @@ static void vgic_init_pending_irq(struct pending_irq *p, unsigned virq)
> p->irq = virq;
> }
>
> -int domain_vgic_init(struct domain *d)
> +int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> {
> int i;
>
> d->arch.vgic.ctlr = 0;
>
> - if ( is_hardware_domain(d) )
> - d->arch.vgic.nr_spis = gic_number_lines() - 32;
> - else
> - d->arch.vgic.nr_spis = 0; /* We don't need SPIs for the guest */
> + /* The number of SPIs as to be aligned to 32 see
> + * GICD_TYPER.ITLinesNumber definition
> + */
> + d->arch.vgic.nr_spis = ROUNDUP(nr_spis, 32);
Here I got 32 lines for 3 SPIs
And this will break checking for IRQ validity in
xen/arch/arm/irq.c:route_irq_to_guest() call. I will not be able to
use IRQ with number greater than 63.
Can this be handled somehow? Or maybe it is already handled and I
missed something ?
Regards,
Andrii
--
Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-29 13:09 ` Andrii Tseglytskyi
@ 2014-08-29 18:57 ` Julien Grall
2014-08-29 19:49 ` Andrii Tseglytskyi
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-29 18:57 UTC (permalink / raw)
To: Andrii Tseglytskyi
Cc: Ian Campbell, Tim Deegan, Ian Jackson, Stefano Stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
On 29/08/14 09:09, Andrii Tseglytskyi wrote:
> Hi Julien,
Hi Andrii,
> Here I got 32 lines for 3 SPIs
>
> And this will break checking for IRQ validity in
> xen/arch/arm/irq.c:route_irq_to_guest() call. I will not be able to
> use IRQ with number greater than 63.
>
> Can this be handled somehow? Or maybe it is already handled and I
> missed something ?
You will have to modify the function for your purpose. If you plan to
PIRQ == VIRQ every time, you could do smth like:
d->arch.vgic.nr_spis = gic_number_lines() - 32;
I assume, you already modified vgic_allocate_irq to return directly the
pirq, and do nothing in vgic_free_irq.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-29 18:57 ` Julien Grall
@ 2014-08-29 19:49 ` Andrii Tseglytskyi
2014-08-29 20:04 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Andrii Tseglytskyi @ 2014-08-29 19:49 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Tim Deegan, Ian Jackson, Stefano Stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
On Fri, Aug 29, 2014 at 9:57 PM, Julien Grall <julien.grall@linaro.org> wrote:
> On 29/08/14 09:09, Andrii Tseglytskyi wrote:
>>
>> Hi Julien,
>
>
> Hi Andrii,
>
>
>> Here I got 32 lines for 3 SPIs
>>
>> And this will break checking for IRQ validity in
>> xen/arch/arm/irq.c:route_irq_to_guest() call. I will not be able to
>> use IRQ with number greater than 63.
>>
>> Can this be handled somehow? Or maybe it is already handled and I
>> missed something ?
>
>
> You will have to modify the function for your purpose. If you plan to PIRQ
> == VIRQ every time, you could do smth like:
>
>
> d->arch.vgic.nr_spis = gic_number_lines() - 32;
>
> I assume, you already modified vgic_allocate_irq to return directly the
> pirq, and do nothing in vgic_free_irq.
Yes, locally I did the same as you mentioned, but I'm thinking about
solution which will allow me to have 1 to 1 irq mapping and which will
be acceptable for Xen mainline.
My local diff is:
diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
index 9333aa0..50ff100 100644
--- a/xen/arch/arm/physdev.c
+++ b/xen/arch/arm/physdev.c
@@ -46,16 +46,13 @@ static int physdev_map_pirq(domid_t domid, int
type, int index, int *pirq_p)
virq = vgic_allocate_virq(d, irq);
ret = -EMFILE;
if ( virq == -1 )
goto free_domain;
- ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
+ ret = route_irq_to_guest(d, irq, irq, "routed IRQ");
if ( !ret )
*pirq_p = virq;
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 644742e..33f190b 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -69,7 +69,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
/* The number of SPIs as to be aligned to 32 see
* GICD_TYPER.ITLinesNumber definition
*/
- d->arch.vgic.nr_spis = ROUNDUP(nr_spis, 32);
+ d->arch.vgic.nr_spis = gic_number_lines() - 32;
switch ( gic_hw_version() )
Regards,
Andrii
--
Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-29 19:49 ` Andrii Tseglytskyi
@ 2014-08-29 20:04 ` Julien Grall
2014-08-29 20:14 ` Andrii Tseglytskyi
2014-09-09 13:33 ` Ian Campbell
0 siblings, 2 replies; 119+ messages in thread
From: Julien Grall @ 2014-08-29 20:04 UTC (permalink / raw)
To: Andrii Tseglytskyi
Cc: Ian Campbell, Tim Deegan, Ian Jackson, Stefano Stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
On 29/08/14 15:49, Andrii Tseglytskyi wrote:
> On Fri, Aug 29, 2014 at 9:57 PM, Julien Grall <julien.grall@linaro.org> wrote:
>> You will have to modify the function for your purpose. If you plan to PIRQ
>> == VIRQ every time, you could do smth like:
>>
>>
>> d->arch.vgic.nr_spis = gic_number_lines() - 32;
>>
>> I assume, you already modified vgic_allocate_irq to return directly the
>> pirq, and do nothing in vgic_free_irq.
>
> Yes, locally I did the same as you mentioned, but I'm thinking about
> solution which will allow me to have 1 to 1 irq mapping and which will
> be acceptable for Xen mainline.
IHMO, the best solution would adding a boolean in the new DOMCTL to
specify if we want to use PIRQ == VIRQ or not.
Then you will have to modify vgic_allocate_irq to return the physical
IRQ if the domain as this new option enabled.
> My local diff is:
>
> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> index 9333aa0..50ff100 100644
> --- a/xen/arch/arm/physdev.c
> +++ b/xen/arch/arm/physdev.c
> @@ -46,16 +46,13 @@ static int physdev_map_pirq(domid_t domid, int
> type, int index, int *pirq_p)
>
> virq = vgic_allocate_virq(d, irq);
> ret = -EMFILE;
> if ( virq == -1 )
> goto free_domain;
>
> - ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
> + ret = route_irq_to_guest(d, irq, irq, "routed IRQ");
With this change you may have some issue when unmap_pirq is used. I
would modify vgic_allocate_irq to always return the physical IRQ number.
The code will be also more clean.
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-29 20:04 ` Julien Grall
@ 2014-08-29 20:14 ` Andrii Tseglytskyi
2014-09-09 13:33 ` Ian Campbell
1 sibling, 0 replies; 119+ messages in thread
From: Andrii Tseglytskyi @ 2014-08-29 20:14 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Tim Deegan, Ian Jackson, Stefano Stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
Hi Julien,
On Fri, Aug 29, 2014 at 11:04 PM, Julien Grall <julien.grall@linaro.org> wrote:
>
>
> On 29/08/14 15:49, Andrii Tseglytskyi wrote:
>>
>> On Fri, Aug 29, 2014 at 9:57 PM, Julien Grall <julien.grall@linaro.org>
>> wrote:
>>>
>>> You will have to modify the function for your purpose. If you plan to
>>> PIRQ
>>> == VIRQ every time, you could do smth like:
>>>
>>>
>>> d->arch.vgic.nr_spis = gic_number_lines() - 32;
>>>
>>> I assume, you already modified vgic_allocate_irq to return directly the
>>> pirq, and do nothing in vgic_free_irq.
>>
>>
>> Yes, locally I did the same as you mentioned, but I'm thinking about
>> solution which will allow me to have 1 to 1 irq mapping and which will
>> be acceptable for Xen mainline.
>
>
> IHMO, the best solution would adding a boolean in the new DOMCTL to specify
> if we want to use PIRQ == VIRQ or not.
>
> Then you will have to modify vgic_allocate_irq to return the physical IRQ if
> the domain as this new option enabled.
This sounds good.
>
>
>> My local diff is:
>>
>> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
>> index 9333aa0..50ff100 100644
>> --- a/xen/arch/arm/physdev.c
>> +++ b/xen/arch/arm/physdev.c
>> @@ -46,16 +46,13 @@ static int physdev_map_pirq(domid_t domid, int
>> type, int index, int *pirq_p)
>>
>> virq = vgic_allocate_virq(d, irq);
>> ret = -EMFILE;
>> if ( virq == -1 )
>> goto free_domain;
>>
>> - ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
>> + ret = route_irq_to_guest(d, irq, irq, "routed IRQ");
>
>
> With this change you may have some issue when unmap_pirq is used. I would
> modify vgic_allocate_irq to always return the physical IRQ number.
>
> The code will be also more clean.
Sure, anyway this won't be needed in case of new bool option for domain
Regards,
Andrii
--
Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-08-29 20:04 ` Julien Grall
2014-08-29 20:14 ` Andrii Tseglytskyi
@ 2014-09-09 13:33 ` Ian Campbell
2014-09-09 19:11 ` Julien Grall
1 sibling, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:33 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Jackson, Tim Deegan, Stefano Stabellini, Jan Beulich,
xen-devel, Daniel De Graaf, Andrii Tseglytskyi
On Fri, 2014-08-29 at 16:04 -0400, Julien Grall wrote:
>
> On 29/08/14 15:49, Andrii Tseglytskyi wrote:
> > On Fri, Aug 29, 2014 at 9:57 PM, Julien Grall <julien.grall@linaro.org> wrote:
> >> You will have to modify the function for your purpose. If you plan to PIRQ
> >> == VIRQ every time, you could do smth like:
> >>
> >>
> >> d->arch.vgic.nr_spis = gic_number_lines() - 32;
> >>
> >> I assume, you already modified vgic_allocate_irq to return directly the
> >> pirq, and do nothing in vgic_free_irq.
> >
> > Yes, locally I did the same as you mentioned, but I'm thinking about
> > solution which will allow me to have 1 to 1 irq mapping and which will
> > be acceptable for Xen mainline.
>
> IHMO, the best solution would adding a boolean in the new DOMCTL to
> specify if we want to use PIRQ == VIRQ or not.
Shouldn't it be up to the toolstack to pass in a suitably large nr_spis,
i.e. at least as large as the maximum irq it wasn't to map 1:1?
So with Andrii's set of 53, 173, 174 the toolstack would have to pass
174.
Ian.
> Then you will have to modify vgic_allocate_irq to return the physical
> IRQ if the domain as this new option enabled.
>
> > My local diff is:
> >
> > diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> > index 9333aa0..50ff100 100644
> > --- a/xen/arch/arm/physdev.c
> > +++ b/xen/arch/arm/physdev.c
> > @@ -46,16 +46,13 @@ static int physdev_map_pirq(domid_t domid, int
> > type, int index, int *pirq_p)
> >
> > virq = vgic_allocate_virq(d, irq);
> > ret = -EMFILE;
> > if ( virq == -1 )
> > goto free_domain;
> >
> > - ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
> > + ret = route_irq_to_guest(d, irq, irq, "routed IRQ");
>
> With this change you may have some issue when unmap_pirq is used. I
> would modify vgic_allocate_irq to always return the physical IRQ number.
>
> The code will be also more clean.
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-09-09 13:33 ` Ian Campbell
@ 2014-09-09 19:11 ` Julien Grall
2014-09-10 9:45 ` Andrii Tseglytskyi
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-09 19:11 UTC (permalink / raw)
To: Ian Campbell
Cc: Ian Jackson, Tim Deegan, Stefano Stabellini, Jan Beulich,
xen-devel, Daniel De Graaf, Andrii Tseglytskyi
Hi Ian,
On 09/09/14 06:33, Ian Campbell wrote:
> On Fri, 2014-08-29 at 16:04 -0400, Julien Grall wrote:
>>
>> On 29/08/14 15:49, Andrii Tseglytskyi wrote:
>>> On Fri, Aug 29, 2014 at 9:57 PM, Julien Grall <julien.grall@linaro.org> wrote:
>>>> You will have to modify the function for your purpose. If you plan to PIRQ
>>>> == VIRQ every time, you could do smth like:
>>>>
>>>>
>>>> d->arch.vgic.nr_spis = gic_number_lines() - 32;
>>>>
>>>> I assume, you already modified vgic_allocate_irq to return directly the
>>>> pirq, and do nothing in vgic_free_irq.
>>>
>>> Yes, locally I did the same as you mentioned, but I'm thinking about
>>> solution which will allow me to have 1 to 1 irq mapping and which will
>>> be acceptable for Xen mainline.
>>
>> IHMO, the best solution would adding a boolean in the new DOMCTL to
>> specify if we want to use PIRQ == VIRQ or not.
>
> Shouldn't it be up to the toolstack to pass in a suitably large nr_spis,
> i.e. at least as large as the maximum irq it wasn't to map 1:1?
For the moment, Xen always choose the VIRQ. Looking again to the
hypercall, I think we can let the toolstack asking for a specific VIRQ.
For now, I don't handle this case. I will rework my series to do it.
Hence, it will make patch #20 less uglier.
Also it would help Andrii use case and avoid him to carry a bigger patch :).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-09-09 19:11 ` Julien Grall
@ 2014-09-10 9:45 ` Andrii Tseglytskyi
0 siblings, 0 replies; 119+ messages in thread
From: Andrii Tseglytskyi @ 2014-09-10 9:45 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Campbell, Tim Deegan, Ian Jackson, Stefano Stabellini,
Jan Beulich, xen-devel, Daniel De Graaf
[-- Attachment #1.1: Type: text/plain, Size: 1746 bytes --]
On Tue, Sep 9, 2014 at 10:11 PM, Julien Grall <julien.grall@linaro.org>
wrote:
> Hi Ian,
>
> On 09/09/14 06:33, Ian Campbell wrote:
>
>> On Fri, 2014-08-29 at 16:04 -0400, Julien Grall wrote:
>>
>>>
>>> On 29/08/14 15:49, Andrii Tseglytskyi wrote:
>>>
>>>> On Fri, Aug 29, 2014 at 9:57 PM, Julien Grall <julien.grall@linaro.org>
>>>> wrote:
>>>>
>>>>> You will have to modify the function for your purpose. If you plan to
>>>>> PIRQ
>>>>> == VIRQ every time, you could do smth like:
>>>>>
>>>>>
>>>>> d->arch.vgic.nr_spis = gic_number_lines() - 32;
>>>>>
>>>>> I assume, you already modified vgic_allocate_irq to return directly the
>>>>> pirq, and do nothing in vgic_free_irq.
>>>>>
>>>>
>>>> Yes, locally I did the same as you mentioned, but I'm thinking about
>>>> solution which will allow me to have 1 to 1 irq mapping and which will
>>>> be acceptable for Xen mainline.
>>>>
>>>
>>> IHMO, the best solution would adding a boolean in the new DOMCTL to
>>> specify if we want to use PIRQ == VIRQ or not.
>>>
>>
>> Shouldn't it be up to the toolstack to pass in a suitably large nr_spis,
>> i.e. at least as large as the maximum irq it wasn't to map 1:1?
>>
>
> For the moment, Xen always choose the VIRQ. Looking again to the
> hypercall, I think we can let the toolstack asking for a specific VIRQ.
>
> For now, I don't handle this case. I will rework my series to do it.
> Hence, it will make patch #20 less uglier.
>
> Also it would help Andrii use case and avoid him to carry a bigger patch
> :).
>
>
This sounds really great. I think I will not need any additional local
patch if you handle this.
Thank you
Regards,
Andrii
> Regards,
>
> --
> Julien Grall
>
--
Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com
[-- Attachment #1.2: Type: text/html, Size: 3711 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
2014-07-31 15:00 ` [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later Julien Grall
` (2 preceding siblings ...)
2014-08-29 13:09 ` Andrii Tseglytskyi
@ 2014-09-09 13:37 ` Ian Campbell
[not found] ` <CAAHg+HhhsZonrEDdHET93dy=dR1+YF-VPGJ=VwB20RRxWqdSYA@mail.gmail.com>
4 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:37 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Jackson, tim, stefano.stabellini, Jan Beulich, xen-devel,
Daniel De Graaf
On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
> + /* The number of SPIs as to be aligned to 32 see
"has to be".
Ian
^ permalink raw reply [flat|nested] 119+ messages in thread[parent not found: <CAAHg+HhhsZonrEDdHET93dy=dR1+YF-VPGJ=VwB20RRxWqdSYA@mail.gmail.com>]
* Re: [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later
[not found] ` <CAAHg+HhhsZonrEDdHET93dy=dR1+YF-VPGJ=VwB20RRxWqdSYA@mail.gmail.com>
@ 2014-10-06 16:04 ` Julien Grall
0 siblings, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-10-06 16:04 UTC (permalink / raw)
To: Pranavkumar Sawargaonkar
Cc: Ian Campbell, tim, Ian Jackson, Stefano Stabellini, Jan Beulich,
xen-devel, Daniel De Graaf
On 09/04/2014 10:02 AM, Pranavkumar Sawargaonkar wrote:
> Hi Julien,
Hi Pranav,
> On 31 July 2014 20:30, Julien Grall <julien.grall@linaro.org
> diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
> index 27fe3b6..1348905 100644
> --- a/tools/libxc/xc_domain.c
> +++ b/tools/libxc/xc_domain.c
> @@ -48,6 +48,18 @@ int xc_domain_create(xc_interface *xch,
> return 0;
> }
>
> +#if defined(__arm__) || defined(__arch64__)
>
>
> Please change __arch64__ to __aarch64__ , otherwise it will break on arm64.
Thanks for spotting this error. I will update the patch for the next
version.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (7 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 08/21] xen/arm: Initialize the virtual GIC later Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 15:49 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq Julien Grall
` (12 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
Xen has to release IRQ routed to a domain in order to reuse later. Currently
only SPIs can be routed to the guest so we only need to browse SPIs for a
specific domain.
Futhermore, a guest can crash and let the IRQ in an incorrect state (i.e has
not being EOIed). Xen will have to reset the IRQ in order to be able to reuse
the IRQ later.
Introduce 2 new functions for release an IRQ routed to a domain:
- release_guest_irq: upper level to retrieve the IRQ, call the GIC
code and release the action
- gic_remove_guest_irq: Check if we can remove the IRQ, and reset
it if necessary
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Drop the desc->handler = &no_irq_type in release_irq as it's
buggy the IRQ is routed to Xen
- Add release_guest_irq and gic_remove_guest_irq
---
xen/arch/arm/gic.c | 36 ++++++++++++++++++++++++++++++++++
xen/arch/arm/irq.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/vgic.c | 16 +++++++++++++++
xen/include/asm-arm/gic.h | 4 ++++
xen/include/asm-arm/irq.h | 2 ++
5 files changed, 106 insertions(+)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 8ef8764..22f331a 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -144,6 +144,42 @@ void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
p->desc = desc;
}
+/* This function only works with SPIs for now */
+int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc)
+{
+ struct pending_irq *p = irq_to_pending(d->vcpu[0], virq);
+
+ ASSERT(spin_is_locked(&desc->lock));
+ ASSERT(test_bit(_IRQ_GUEST, &desc->status));
+ ASSERT(p->desc == desc);
+
+ /* If the IRQ is removed when the domain is dying, we only need to
+ * EOI the IRQ if it has not been done by the guest
+ */
+ if ( d->is_dying )
+ {
+ desc->handler->shutdown(desc);
+ if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
+ gic_hw_ops->deactivate_irq(desc);
+ clear_bit(_IRQ_INPROGRESS, &desc->status);
+ goto end;
+ }
+
+ /* TODO: Handle eviction from LRs. For now, deny remove if the IRQ
+ * is inflight and not disabled.
+ */
+ if ( test_bit(_IRQ_INPROGRESS, &desc->status) ||
+ test_bit(_IRQ_DISABLED, &desc->status) )
+ return -EBUSY;
+
+end:
+ desc->handler = &no_irq_type;
+ p->desc = NULL;
+
+ return 0;
+}
+
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
unsigned int *out_hwirq,
unsigned int *out_type)
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 7eeb8dd..ba33571 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -292,6 +292,7 @@ void release_irq(unsigned int irq, const void *dev_id)
if ( !desc->action )
{
desc->handler->shutdown(desc);
+
clear_bit(_IRQ_GUEST, &desc->status);
}
@@ -479,6 +480,53 @@ out:
return retval;
}
+int release_guest_irq(struct domain *d, unsigned int virq)
+{
+ struct irq_desc *desc;
+ struct irq_guest *info;
+ unsigned long flags;
+ struct pending_irq *p;
+ int ret;
+
+ if ( virq >= vgic_num_irqs(d) )
+ return -EINVAL;
+
+ p = irq_to_pending(d->vcpu[0], virq);
+ if ( !p->desc )
+ return -EINVAL;
+
+ desc = p->desc;
+
+ spin_lock_irqsave(&desc->lock, flags);
+
+ ret = -EINVAL;
+ if ( !test_bit(_IRQ_GUEST, &desc->status) )
+ goto unlock;
+
+ ret = -EINVAL;
+
+ info = irq_get_guest_info(desc);
+ if ( d != info->d )
+ goto unlock;
+
+ ret = gic_remove_irq_from_guest(d, virq, desc);
+
+ spin_unlock_irqrestore(&desc->lock, flags);
+
+ if ( !ret )
+ {
+ release_irq(desc->irq, info);
+ xfree(info);
+ }
+
+ return ret;
+
+unlock:
+ spin_unlock_irqrestore(&desc->lock, flags);
+
+ return ret;
+}
+
/*
* pirq event channels. We don't use these on ARM, instead we use the
* features of the GIC to inject virtualised normal interrupts.
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index a037ecc..2a5fc18 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -118,6 +118,22 @@ void register_vgic_ops(struct domain *d, const struct vgic_ops *ops)
void domain_vgic_free(struct domain *d)
{
+ int i;
+ int ret;
+
+ for ( i = 0; i < (d->arch.vgic.nr_spis); i++ )
+ {
+ struct pending_irq *p = &d->arch.vgic.pending_irqs[i];
+
+ if ( p->desc )
+ {
+ ret = release_guest_irq(d, p->irq);
+ if ( ret )
+ dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
+ d->domain_id, p->irq, ret);
+ }
+ }
+
xfree(d->arch.vgic.shared_irqs);
xfree(d->arch.vgic.pending_irqs);
}
diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
index 89816cd..46d7fd6 100644
--- a/xen/include/asm-arm/gic.h
+++ b/xen/include/asm-arm/gic.h
@@ -201,6 +201,10 @@ extern void gic_route_irq_to_guest(struct domain *, unsigned int virq,
const cpumask_t *cpu_mask,
unsigned int priority);
+/* Remove an IRQ passthrough to a guest */
+int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
+ struct irq_desc *desc);
+
extern void gic_inject(void);
extern void gic_clear_pending_irqs(struct vcpu *v);
extern int gic_events_need_delivery(void);
diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
index a7174f3..d25e1a1 100644
--- a/xen/include/asm-arm/irq.h
+++ b/xen/include/asm-arm/irq.h
@@ -44,6 +44,8 @@ void init_secondary_IRQ(void);
int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned int irq, const char *devname);
+int release_guest_irq(struct domain *d, unsigned int irq);
+
void arch_move_irqs(struct vcpu *v);
/* Set IRQ type for an SPI */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-07-31 15:00 ` [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying Julien Grall
@ 2014-08-06 15:49 ` Stefano Stabellini
2014-08-06 16:01 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 15:49 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> Xen has to release IRQ routed to a domain in order to reuse later. Currently
> only SPIs can be routed to the guest so we only need to browse SPIs for a
> specific domain.
>
> Futhermore, a guest can crash and let the IRQ in an incorrect state (i.e has
> not being EOIed). Xen will have to reset the IRQ in order to be able to reuse
> the IRQ later.
>
> Introduce 2 new functions for release an IRQ routed to a domain:
> - release_guest_irq: upper level to retrieve the IRQ, call the GIC
> code and release the action
> - gic_remove_guest_irq: Check if we can remove the IRQ, and reset
> it if necessary
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> ---
> Changes in v2:
> - Drop the desc->handler = &no_irq_type in release_irq as it's
> buggy the IRQ is routed to Xen
> - Add release_guest_irq and gic_remove_guest_irq
> ---
> xen/arch/arm/gic.c | 36 ++++++++++++++++++++++++++++++++++
> xen/arch/arm/irq.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/vgic.c | 16 +++++++++++++++
> xen/include/asm-arm/gic.h | 4 ++++
> xen/include/asm-arm/irq.h | 2 ++
> 5 files changed, 106 insertions(+)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 8ef8764..22f331a 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -144,6 +144,42 @@ void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> p->desc = desc;
> }
>
> +/* This function only works with SPIs for now */
> +int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc)
> +{
> + struct pending_irq *p = irq_to_pending(d->vcpu[0], virq);
Use vgic_get_target_vcpu to get the target vcpu of virq. You can pass
d->vcpu[0] as first argument to vgic_get_target_vcpu.
I am sorry that doing that is going to add a dependency on my gic series.
> + ASSERT(spin_is_locked(&desc->lock));
> + ASSERT(test_bit(_IRQ_GUEST, &desc->status));
> + ASSERT(p->desc == desc);
> +
> + /* If the IRQ is removed when the domain is dying, we only need to
> + * EOI the IRQ if it has not been done by the guest
> + */
> + if ( d->is_dying )
> + {
> + desc->handler->shutdown(desc);
> + if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
> + gic_hw_ops->deactivate_irq(desc);
> + clear_bit(_IRQ_INPROGRESS, &desc->status);
> + goto end;
> + }
> +
> + /* TODO: Handle eviction from LRs. For now, deny remove if the IRQ
> + * is inflight and not disabled.
> + */
> + if ( test_bit(_IRQ_INPROGRESS, &desc->status) ||
> + test_bit(_IRQ_DISABLED, &desc->status) )
> + return -EBUSY;
> +
> +end:
> + desc->handler = &no_irq_type;
> + p->desc = NULL;
> +
> + return 0;
> +}
> +
> int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
> unsigned int *out_hwirq,
> unsigned int *out_type)
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 7eeb8dd..ba33571 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -292,6 +292,7 @@ void release_irq(unsigned int irq, const void *dev_id)
> if ( !desc->action )
> {
> desc->handler->shutdown(desc);
> +
> clear_bit(_IRQ_GUEST, &desc->status);
> }
spurious change
> @@ -479,6 +480,53 @@ out:
> return retval;
> }
>
> +int release_guest_irq(struct domain *d, unsigned int virq)
> +{
> + struct irq_desc *desc;
> + struct irq_guest *info;
> + unsigned long flags;
> + struct pending_irq *p;
> + int ret;
> +
> + if ( virq >= vgic_num_irqs(d) )
> + return -EINVAL;
> +
> + p = irq_to_pending(d->vcpu[0], virq);
> + if ( !p->desc )
> + return -EINVAL;
Same here: call vgic_get_target_vcpu.
Also if this function is supposed to work only with SPIs, you should add
a comment or explicitly check for it.
> + desc = p->desc;
> +
> + spin_lock_irqsave(&desc->lock, flags);
> +
> + ret = -EINVAL;
> + if ( !test_bit(_IRQ_GUEST, &desc->status) )
> + goto unlock;
> +
> + ret = -EINVAL;
> +
> + info = irq_get_guest_info(desc);
> + if ( d != info->d )
> + goto unlock;
> +
> + ret = gic_remove_irq_from_guest(d, virq, desc);
> +
> + spin_unlock_irqrestore(&desc->lock, flags);
> +
> + if ( !ret )
> + {
> + release_irq(desc->irq, info);
> + xfree(info);
> + }
> +
> + return ret;
> +
> +unlock:
> + spin_unlock_irqrestore(&desc->lock, flags);
> +
> + return ret;
> +}
> +
> /*
> * pirq event channels. We don't use these on ARM, instead we use the
> * features of the GIC to inject virtualised normal interrupts.
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index a037ecc..2a5fc18 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -118,6 +118,22 @@ void register_vgic_ops(struct domain *d, const struct vgic_ops *ops)
>
> void domain_vgic_free(struct domain *d)
> {
> + int i;
> + int ret;
> +
> + for ( i = 0; i < (d->arch.vgic.nr_spis); i++ )
> + {
> + struct pending_irq *p = &d->arch.vgic.pending_irqs[i];
> +
> + if ( p->desc )
> + {
> + ret = release_guest_irq(d, p->irq);
> + if ( ret )
> + dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
> + d->domain_id, p->irq, ret);
> + }
> + }
> +
> xfree(d->arch.vgic.shared_irqs);
> xfree(d->arch.vgic.pending_irqs);
> }
> diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h
> index 89816cd..46d7fd6 100644
> --- a/xen/include/asm-arm/gic.h
> +++ b/xen/include/asm-arm/gic.h
> @@ -201,6 +201,10 @@ extern void gic_route_irq_to_guest(struct domain *, unsigned int virq,
> const cpumask_t *cpu_mask,
> unsigned int priority);
>
> +/* Remove an IRQ passthrough to a guest */
> +int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
> + struct irq_desc *desc);
> +
> extern void gic_inject(void);
> extern void gic_clear_pending_irqs(struct vcpu *v);
> extern int gic_events_need_delivery(void);
> diff --git a/xen/include/asm-arm/irq.h b/xen/include/asm-arm/irq.h
> index a7174f3..d25e1a1 100644
> --- a/xen/include/asm-arm/irq.h
> +++ b/xen/include/asm-arm/irq.h
> @@ -44,6 +44,8 @@ void init_secondary_IRQ(void);
>
> int route_irq_to_guest(struct domain *d, unsigned int virq,
> unsigned int irq, const char *devname);
> +int release_guest_irq(struct domain *d, unsigned int irq);
> +
> void arch_move_irqs(struct vcpu *v);
>
> /* Set IRQ type for an SPI */
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-06 15:49 ` Stefano Stabellini
@ 2014-08-06 16:01 ` Julien Grall
2014-08-06 16:53 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 16:01 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
On 08/06/2014 04:49 PM, Stefano Stabellini wrote:
> On Thu, 31 Jul 2014, Julien Grall wrote:
>> Xen has to release IRQ routed to a domain in order to reuse later. Currently
>> only SPIs can be routed to the guest so we only need to browse SPIs for a
>> specific domain.
>>
>> Futhermore, a guest can crash and let the IRQ in an incorrect state (i.e has
>> not being EOIed). Xen will have to reset the IRQ in order to be able to reuse
>> the IRQ later.
>>
>> Introduce 2 new functions for release an IRQ routed to a domain:
>> - release_guest_irq: upper level to retrieve the IRQ, call the GIC
>> code and release the action
>> - gic_remove_guest_irq: Check if we can remove the IRQ, and reset
>> it if necessary
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>
>> ---
>> Changes in v2:
>> - Drop the desc->handler = &no_irq_type in release_irq as it's
>> buggy the IRQ is routed to Xen
>> - Add release_guest_irq and gic_remove_guest_irq
>> ---
>> xen/arch/arm/gic.c | 36 ++++++++++++++++++++++++++++++++++
>> xen/arch/arm/irq.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
>> xen/arch/arm/vgic.c | 16 +++++++++++++++
>> xen/include/asm-arm/gic.h | 4 ++++
>> xen/include/asm-arm/irq.h | 2 ++
>> 5 files changed, 106 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index 8ef8764..22f331a 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -144,6 +144,42 @@ void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>> p->desc = desc;
>> }
>>
>> +/* This function only works with SPIs for now */
>> +int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
>> + struct irq_desc *desc)
>> +{
>> + struct pending_irq *p = irq_to_pending(d->vcpu[0], virq);
>
> Use vgic_get_target_vcpu to get the target vcpu of virq. You can pass
> d->vcpu[0] as first argument to vgic_get_target_vcpu.
Why do I need to add vgic_get_target_vcpu? This function is only able to
handle SPIs which is shared between VCPU.
It might be interesting to introduce spi_to_pending function.
> I am sorry that doing that is going to add a dependency on my gic series.
I'm already depends on your series :).
>> + ASSERT(spin_is_locked(&desc->lock));
>> + ASSERT(test_bit(_IRQ_GUEST, &desc->status));
>> + ASSERT(p->desc == desc);
>> +
>> + /* If the IRQ is removed when the domain is dying, we only need to
>> + * EOI the IRQ if it has not been done by the guest
>> + */
>> + if ( d->is_dying )
>> + {
>> + desc->handler->shutdown(desc);
>> + if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
>> + gic_hw_ops->deactivate_irq(desc);
>> + clear_bit(_IRQ_INPROGRESS, &desc->status);
>> + goto end;
>> + }
>> +
>> + /* TODO: Handle eviction from LRs. For now, deny remove if the IRQ
>> + * is inflight and not disabled.
>> + */
>> + if ( test_bit(_IRQ_INPROGRESS, &desc->status) ||
>> + test_bit(_IRQ_DISABLED, &desc->status) )
>> + return -EBUSY;
>> +
>> +end:
>> + desc->handler = &no_irq_type;
>> + p->desc = NULL;
>> +
>> + return 0;
>> +}
>> +
>> int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
>> unsigned int *out_hwirq,
>> unsigned int *out_type)
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 7eeb8dd..ba33571 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -292,6 +292,7 @@ void release_irq(unsigned int irq, const void *dev_id)
>> if ( !desc->action )
>> {
>> desc->handler->shutdown(desc);
>> +
>> clear_bit(_IRQ_GUEST, &desc->status);
>> }
>
>
> spurious change
Will drop the line.
>
>> @@ -479,6 +480,53 @@ out:
>> return retval;
>> }
>>
>> +int release_guest_irq(struct domain *d, unsigned int virq)
>> +{
>> + struct irq_desc *desc;
>> + struct irq_guest *info;
>> + unsigned long flags;
>> + struct pending_irq *p;
>> + int ret;
>> +
>> + if ( virq >= vgic_num_irqs(d) )
>> + return -EINVAL;
>> +
>> + p = irq_to_pending(d->vcpu[0], virq);
>> + if ( !p->desc )
>> + return -EINVAL;
>
> Same here: call vgic_get_target_vcpu.
> Also if this function is supposed to work only with SPIs, you should add
> a comment or explicitly check for it.
route_irq_to_guest already check if we are able to route an IRQ or not.
For non-SPIs the function will bailout.
So, here, it's impossible to have p->desc set to another value than NULL
for non-SPIs.
Or Xen is buggy will likely fail in another place.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-06 16:01 ` Julien Grall
@ 2014-08-06 16:53 ` Stefano Stabellini
2014-08-06 17:09 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:53 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Wed, 6 Aug 2014, Julien Grall wrote:
> Hi Stefano,
>
> On 08/06/2014 04:49 PM, Stefano Stabellini wrote:
> > On Thu, 31 Jul 2014, Julien Grall wrote:
> >> Xen has to release IRQ routed to a domain in order to reuse later. Currently
> >> only SPIs can be routed to the guest so we only need to browse SPIs for a
> >> specific domain.
> >>
> >> Futhermore, a guest can crash and let the IRQ in an incorrect state (i.e has
> >> not being EOIed). Xen will have to reset the IRQ in order to be able to reuse
> >> the IRQ later.
> >>
> >> Introduce 2 new functions for release an IRQ routed to a domain:
> >> - release_guest_irq: upper level to retrieve the IRQ, call the GIC
> >> code and release the action
> >> - gic_remove_guest_irq: Check if we can remove the IRQ, and reset
> >> it if necessary
> >>
> >> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>
> >> ---
> >> Changes in v2:
> >> - Drop the desc->handler = &no_irq_type in release_irq as it's
> >> buggy the IRQ is routed to Xen
> >> - Add release_guest_irq and gic_remove_guest_irq
> >> ---
> >> xen/arch/arm/gic.c | 36 ++++++++++++++++++++++++++++++++++
> >> xen/arch/arm/irq.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
> >> xen/arch/arm/vgic.c | 16 +++++++++++++++
> >> xen/include/asm-arm/gic.h | 4 ++++
> >> xen/include/asm-arm/irq.h | 2 ++
> >> 5 files changed, 106 insertions(+)
> >>
> >> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> >> index 8ef8764..22f331a 100644
> >> --- a/xen/arch/arm/gic.c
> >> +++ b/xen/arch/arm/gic.c
> >> @@ -144,6 +144,42 @@ void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> >> p->desc = desc;
> >> }
> >>
> >> +/* This function only works with SPIs for now */
> >> +int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
> >> + struct irq_desc *desc)
> >> +{
> >> + struct pending_irq *p = irq_to_pending(d->vcpu[0], virq);
> >
> > Use vgic_get_target_vcpu to get the target vcpu of virq. You can pass
> > d->vcpu[0] as first argument to vgic_get_target_vcpu.
>
> Why do I need to add vgic_get_target_vcpu? This function is only able to
> handle SPIs which is shared between VCPU.
OK, in that case ASSERT(virq >= 32 && virq < nr_lines). I am fine either way.
Also see below.
> It might be interesting to introduce spi_to_pending function.
>
> > I am sorry that doing that is going to add a dependency on my gic series.
>
> I'm already depends on your series :).
>
> >> + ASSERT(spin_is_locked(&desc->lock));
> >> + ASSERT(test_bit(_IRQ_GUEST, &desc->status));
> >> + ASSERT(p->desc == desc);
> >> +
> >> + /* If the IRQ is removed when the domain is dying, we only need to
> >> + * EOI the IRQ if it has not been done by the guest
> >> + */
> >> + if ( d->is_dying )
> >> + {
> >> + desc->handler->shutdown(desc);
> >> + if ( test_bit(_IRQ_INPROGRESS, &desc->status) )
> >> + gic_hw_ops->deactivate_irq(desc);
> >> + clear_bit(_IRQ_INPROGRESS, &desc->status);
> >> + goto end;
> >> + }
> >> +
> >> + /* TODO: Handle eviction from LRs. For now, deny remove if the IRQ
> >> + * is inflight and not disabled.
> >> + */
> >> + if ( test_bit(_IRQ_INPROGRESS, &desc->status) ||
> >> + test_bit(_IRQ_DISABLED, &desc->status) )
> >> + return -EBUSY;
> >> +
> >> +end:
> >> + desc->handler = &no_irq_type;
> >> + p->desc = NULL;
> >> +
> >> + return 0;
> >> +}
> >> +
> >> int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
> >> unsigned int *out_hwirq,
> >> unsigned int *out_type)
> >> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> >> index 7eeb8dd..ba33571 100644
> >> --- a/xen/arch/arm/irq.c
> >> +++ b/xen/arch/arm/irq.c
> >> @@ -292,6 +292,7 @@ void release_irq(unsigned int irq, const void *dev_id)
> >> if ( !desc->action )
> >> {
> >> desc->handler->shutdown(desc);
> >> +
> >> clear_bit(_IRQ_GUEST, &desc->status);
> >> }
> >
> >
> > spurious change
>
> Will drop the line.
>
> >
> >> @@ -479,6 +480,53 @@ out:
> >> return retval;
> >> }
> >>
> >> +int release_guest_irq(struct domain *d, unsigned int virq)
> >> +{
> >> + struct irq_desc *desc;
> >> + struct irq_guest *info;
> >> + unsigned long flags;
> >> + struct pending_irq *p;
> >> + int ret;
> >> +
> >> + if ( virq >= vgic_num_irqs(d) )
> >> + return -EINVAL;
> >> +
> >> + p = irq_to_pending(d->vcpu[0], virq);
> >> + if ( !p->desc )
> >> + return -EINVAL;
> >
> > Same here: call vgic_get_target_vcpu.
> > Also if this function is supposed to work only with SPIs, you should add
> > a comment or explicitly check for it.
>
> route_irq_to_guest already check if we are able to route an IRQ or not.
> For non-SPIs the function will bailout.
>
> So, here, it's impossible to have p->desc set to another value than NULL
> for non-SPIs.
>
> Or Xen is buggy will likely fail in another place.
If you do:
p = irq_to_pending(d->vcpu[0], virq);
you are actually introducing more code that cannot cope with non-SGIs.
So you should either:
1) esplicitely check for it (add an ASSERT)
2) replace it with code that can cope with non-SGIs, such us
irq_to_pending(vgic_get_target_vcpu(d->vcpu[0], virq), virq)
I would prefer 2), but 1) is also an option
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-06 16:53 ` Stefano Stabellini
@ 2014-08-06 17:09 ` Julien Grall
2014-08-07 15:36 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 17:09 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On 08/06/2014 05:53 PM, Stefano Stabellini wrote:
> On Wed, 6 Aug 2014, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 08/06/2014 04:49 PM, Stefano Stabellini wrote:
>>> On Thu, 31 Jul 2014, Julien Grall wrote:
>>>> Xen has to release IRQ routed to a domain in order to reuse later. Currently
>>>> only SPIs can be routed to the guest so we only need to browse SPIs for a
>>>> specific domain.
>>>>
>>>> Futhermore, a guest can crash and let the IRQ in an incorrect state (i.e has
>>>> not being EOIed). Xen will have to reset the IRQ in order to be able to reuse
>>>> the IRQ later.
>>>>
>>>> Introduce 2 new functions for release an IRQ routed to a domain:
>>>> - release_guest_irq: upper level to retrieve the IRQ, call the GIC
>>>> code and release the action
>>>> - gic_remove_guest_irq: Check if we can remove the IRQ, and reset
>>>> it if necessary
>>>>
>>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>>>
>>>> ---
>>>> Changes in v2:
>>>> - Drop the desc->handler = &no_irq_type in release_irq as it's
>>>> buggy the IRQ is routed to Xen
>>>> - Add release_guest_irq and gic_remove_guest_irq
>>>> ---
>>>> xen/arch/arm/gic.c | 36 ++++++++++++++++++++++++++++++++++
>>>> xen/arch/arm/irq.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
>>>> xen/arch/arm/vgic.c | 16 +++++++++++++++
>>>> xen/include/asm-arm/gic.h | 4 ++++
>>>> xen/include/asm-arm/irq.h | 2 ++
>>>> 5 files changed, 106 insertions(+)
>>>>
>>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>>> index 8ef8764..22f331a 100644
>>>> --- a/xen/arch/arm/gic.c
>>>> +++ b/xen/arch/arm/gic.c
>>>> @@ -144,6 +144,42 @@ void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>>>> p->desc = desc;
>>>> }
>>>>
>>>> +/* This function only works with SPIs for now */
>>>> +int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
>>>> + struct irq_desc *desc)
>>>> +{
>>>> + struct pending_irq *p = irq_to_pending(d->vcpu[0], virq);
>>>
>>> Use vgic_get_target_vcpu to get the target vcpu of virq. You can pass
>>> d->vcpu[0] as first argument to vgic_get_target_vcpu.
>>
>> Why do I need to add vgic_get_target_vcpu? This function is only able to
>> handle SPIs which is shared between VCPU.
>
> OK, in that case ASSERT(virq >= 32 && virq < nr_lines). I am fine either way.
> Also see below.
It's implicitly done by (p->desc == desc). p->desc is only set for SPIs
assigned to a guest. If desc is NULL, then it will fault a bit later.
If someone doesn't use this API to route an IRQ then it's his fault.
Hence, this as been checked in route_irq_guest. I don't think we should
bother to check again.
>>>
>>>> @@ -479,6 +480,53 @@ out:
>>>> return retval;
>>>> }
>>>>
>>>> +int release_guest_irq(struct domain *d, unsigned int virq)
>>>> +{
>>>> + struct irq_desc *desc;
>>>> + struct irq_guest *info;
>>>> + unsigned long flags;
>>>> + struct pending_irq *p;
>>>> + int ret;
>>>> +
>>>> + if ( virq >= vgic_num_irqs(d) )
>>>> + return -EINVAL;
>>>> +
>>>> + p = irq_to_pending(d->vcpu[0], virq);
>>>> + if ( !p->desc )
>>>> + return -EINVAL;
>>>
>>> Same here: call vgic_get_target_vcpu.
>>> Also if this function is supposed to work only with SPIs, you should add
>>> a comment or explicitly check for it.
>>
>> route_irq_to_guest already check if we are able to route an IRQ or not.
>> For non-SPIs the function will bailout.
>>
>> So, here, it's impossible to have p->desc set to another value than NULL
>> for non-SPIs.
>>
>> Or Xen is buggy will likely fail in another place.
>
> If you do:
>
> p = irq_to_pending(d->vcpu[0], virq);
>
> you are actually introducing more code that cannot cope with non-SGIs.
> So you should either:
>
> 1) esplicitely check for it (add an ASSERT)
Already done in route_irq_to_guest. I don't think we have to add yet
another assert here.
> 2) replace it with code that can cope with non-SGIs, such us
> irq_to_pending(vgic_get_target_vcpu(d->vcpu[0], virq), virq)
This code won't cope with non-SGIs (here PPIs). As PPIs have an irq_desc
per CPU we will have to loop on every VCPU to unmap it.
But I doubt we will have PPIs in future, there is more issues to handle
(such as the number of VCPUs doesn't match the number of physical CPUs).
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-06 17:09 ` Julien Grall
@ 2014-08-07 15:36 ` Stefano Stabellini
2014-08-07 15:40 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-07 15:36 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Wed, 6 Aug 2014, Julien Grall wrote:
> On 08/06/2014 05:53 PM, Stefano Stabellini wrote:
> > On Wed, 6 Aug 2014, Julien Grall wrote:
> >> Hi Stefano,
> >>
> >> On 08/06/2014 04:49 PM, Stefano Stabellini wrote:
> >>> On Thu, 31 Jul 2014, Julien Grall wrote:
> >>>> Xen has to release IRQ routed to a domain in order to reuse later. Currently
> >>>> only SPIs can be routed to the guest so we only need to browse SPIs for a
> >>>> specific domain.
> >>>>
> >>>> Futhermore, a guest can crash and let the IRQ in an incorrect state (i.e has
> >>>> not being EOIed). Xen will have to reset the IRQ in order to be able to reuse
> >>>> the IRQ later.
> >>>>
> >>>> Introduce 2 new functions for release an IRQ routed to a domain:
> >>>> - release_guest_irq: upper level to retrieve the IRQ, call the GIC
> >>>> code and release the action
> >>>> - gic_remove_guest_irq: Check if we can remove the IRQ, and reset
> >>>> it if necessary
> >>>>
> >>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>>>
> >>>> ---
> >>>> Changes in v2:
> >>>> - Drop the desc->handler = &no_irq_type in release_irq as it's
> >>>> buggy the IRQ is routed to Xen
> >>>> - Add release_guest_irq and gic_remove_guest_irq
> >>>> ---
> >>>> xen/arch/arm/gic.c | 36 ++++++++++++++++++++++++++++++++++
> >>>> xen/arch/arm/irq.c | 48 +++++++++++++++++++++++++++++++++++++++++++++
> >>>> xen/arch/arm/vgic.c | 16 +++++++++++++++
> >>>> xen/include/asm-arm/gic.h | 4 ++++
> >>>> xen/include/asm-arm/irq.h | 2 ++
> >>>> 5 files changed, 106 insertions(+)
> >>>>
> >>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> >>>> index 8ef8764..22f331a 100644
> >>>> --- a/xen/arch/arm/gic.c
> >>>> +++ b/xen/arch/arm/gic.c
> >>>> @@ -144,6 +144,42 @@ void gic_route_irq_to_guest(struct domain *d, unsigned int virq,
> >>>> p->desc = desc;
> >>>> }
> >>>>
> >>>> +/* This function only works with SPIs for now */
> >>>> +int gic_remove_irq_from_guest(struct domain *d, unsigned int virq,
> >>>> + struct irq_desc *desc)
> >>>> +{
> >>>> + struct pending_irq *p = irq_to_pending(d->vcpu[0], virq);
> >>>
> >>> Use vgic_get_target_vcpu to get the target vcpu of virq. You can pass
> >>> d->vcpu[0] as first argument to vgic_get_target_vcpu.
> >>
> >> Why do I need to add vgic_get_target_vcpu? This function is only able to
> >> handle SPIs which is shared between VCPU.
> >
> > OK, in that case ASSERT(virq >= 32 && virq < nr_lines). I am fine either way.
> > Also see below.
>
> It's implicitly done by (p->desc == desc). p->desc is only set for SPIs
> assigned to a guest. If desc is NULL, then it will fault a bit later.
> If someone doesn't use this API to route an IRQ then it's his fault.
>
> Hence, this as been checked in route_irq_guest. I don't think we should
> bother to check again.
>
> >>>
> >>>> @@ -479,6 +480,53 @@ out:
> >>>> return retval;
> >>>> }
> >>>>
> >>>> +int release_guest_irq(struct domain *d, unsigned int virq)
> >>>> +{
> >>>> + struct irq_desc *desc;
> >>>> + struct irq_guest *info;
> >>>> + unsigned long flags;
> >>>> + struct pending_irq *p;
> >>>> + int ret;
> >>>> +
> >>>> + if ( virq >= vgic_num_irqs(d) )
> >>>> + return -EINVAL;
> >>>> +
> >>>> + p = irq_to_pending(d->vcpu[0], virq);
> >>>> + if ( !p->desc )
> >>>> + return -EINVAL;
> >>>
> >>> Same here: call vgic_get_target_vcpu.
> >>> Also if this function is supposed to work only with SPIs, you should add
> >>> a comment or explicitly check for it.
> >>
> >> route_irq_to_guest already check if we are able to route an IRQ or not.
> >> For non-SPIs the function will bailout.
> >>
> >> So, here, it's impossible to have p->desc set to another value than NULL
> >> for non-SPIs.
> >>
> >> Or Xen is buggy will likely fail in another place.
> >
> > If you do:
> >
> > p = irq_to_pending(d->vcpu[0], virq);
> >
> > you are actually introducing more code that cannot cope with non-SGIs.
> > So you should either:
> >
> > 1) esplicitely check for it (add an ASSERT)
>
> Already done in route_irq_to_guest. I don't think we have to add yet
> another assert here.
>
> > 2) replace it with code that can cope with non-SGIs, such us
> > irq_to_pending(vgic_get_target_vcpu(d->vcpu[0], virq), virq)
>
> This code won't cope with non-SGIs (here PPIs). As PPIs have an irq_desc
> per CPU we will have to loop on every VCPU to unmap it.
>
> But I doubt we will have PPIs in future, there is more issues to handle
> (such as the number of VCPUs doesn't match the number of physical CPUs).
In that case, why not call the two functions release_guest_spi and
gic_remove_spi_from_guest?
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-07 15:36 ` Stefano Stabellini
@ 2014-08-07 15:40 ` Julien Grall
2014-08-07 16:31 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-07 15:40 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On 08/07/2014 04:36 PM, Stefano Stabellini wrote:
>> But I doubt we will have PPIs in future, there is more issues to handle
>> (such as the number of VCPUs doesn't match the number of physical CPUs).
>
> In that case, why not call the two functions release_guest_spi and
> gic_remove_spi_from_guest?
Because the 2 functions to route the irq are called route_irq_to_guest
and gic_route_irq_to_guest.
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-07 15:40 ` Julien Grall
@ 2014-08-07 16:31 ` Stefano Stabellini
2014-08-07 16:35 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-07 16:31 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Thu, 7 Aug 2014, Julien Grall wrote:
> On 08/07/2014 04:36 PM, Stefano Stabellini wrote:
> >> But I doubt we will have PPIs in future, there is more issues to handle
> >> (such as the number of VCPUs doesn't match the number of physical CPUs).
> >
> > In that case, why not call the two functions release_guest_spi and
> > gic_remove_spi_from_guest?
>
> Because the 2 functions to route the irq are called route_irq_to_guest
> and gic_route_irq_to_guest.
I would prefer to avoid introducing more functions that look like they
can handle any irqs but actually they cannot by design.
I would be OK with the asymmetry in function names. We could also turn
route_irq_to_guest into route_spi_to_guest and gic_route_irq_to_guest
into gic_route_spi_to_guest.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-07 16:31 ` Stefano Stabellini
@ 2014-08-07 16:35 ` Julien Grall
2014-08-07 16:39 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-07 16:35 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On 08/07/2014 05:31 PM, Stefano Stabellini wrote:
> On Thu, 7 Aug 2014, Julien Grall wrote:
>> On 08/07/2014 04:36 PM, Stefano Stabellini wrote:
>>>> But I doubt we will have PPIs in future, there is more issues to handle
>>>> (such as the number of VCPUs doesn't match the number of physical CPUs).
>>>
>>> In that case, why not call the two functions release_guest_spi and
>>> gic_remove_spi_from_guest?
>>
>> Because the 2 functions to route the irq are called route_irq_to_guest
>> and gic_route_irq_to_guest.
>
> I would prefer to avoid introducing more functions that look like they
> can handle any irqs but actually they cannot by design.
That why Xen checks the IRQ number at the beginning of route_irq_to_guest.
> I would be OK with the asymmetry in function names. We could also turn
> route_irq_to_guest into route_spi_to_guest and gic_route_irq_to_guest
> into gic_route_spi_to_guest.
Those functions will be used for MSI sooner or later. I would prefer to
keep the current name and add an ASSERT.
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-07 16:35 ` Julien Grall
@ 2014-08-07 16:39 ` Stefano Stabellini
2014-09-09 13:53 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-07 16:39 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Thu, 7 Aug 2014, Julien Grall wrote:
> On 08/07/2014 05:31 PM, Stefano Stabellini wrote:
> > On Thu, 7 Aug 2014, Julien Grall wrote:
> >> On 08/07/2014 04:36 PM, Stefano Stabellini wrote:
> >>>> But I doubt we will have PPIs in future, there is more issues to handle
> >>>> (such as the number of VCPUs doesn't match the number of physical CPUs).
> >>>
> >>> In that case, why not call the two functions release_guest_spi and
> >>> gic_remove_spi_from_guest?
> >>
> >> Because the 2 functions to route the irq are called route_irq_to_guest
> >> and gic_route_irq_to_guest.
> >
> > I would prefer to avoid introducing more functions that look like they
> > can handle any irqs but actually they cannot by design.
>
> That why Xen checks the IRQ number at the beginning of route_irq_to_guest.
>
> > I would be OK with the asymmetry in function names. We could also turn
> > route_irq_to_guest into route_spi_to_guest and gic_route_irq_to_guest
> > into gic_route_spi_to_guest.
>
> Those functions will be used for MSI sooner or later. I would prefer to
> keep the current name and add an ASSERT.
OK then, I can settle for that
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-08-07 16:39 ` Stefano Stabellini
@ 2014-09-09 13:53 ` Ian Campbell
2014-09-09 22:29 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 13:53 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, Julien Grall, tim, stefano.stabellini
On Thu, 2014-08-07 at 17:39 +0100, Stefano Stabellini wrote:
> On Thu, 7 Aug 2014, Julien Grall wrote:
> > On 08/07/2014 05:31 PM, Stefano Stabellini wrote:
> > > On Thu, 7 Aug 2014, Julien Grall wrote:
> > >> On 08/07/2014 04:36 PM, Stefano Stabellini wrote:
> > >>>> But I doubt we will have PPIs in future, there is more issues to handle
> > >>>> (such as the number of VCPUs doesn't match the number of physical CPUs).
> > >>>
> > >>> In that case, why not call the two functions release_guest_spi and
> > >>> gic_remove_spi_from_guest?
> > >>
> > >> Because the 2 functions to route the irq are called route_irq_to_guest
> > >> and gic_route_irq_to_guest.
> > >
> > > I would prefer to avoid introducing more functions that look like they
> > > can handle any irqs but actually they cannot by design.
> >
> > That why Xen checks the IRQ number at the beginning of route_irq_to_guest.
> >
> > > I would be OK with the asymmetry in function names. We could also turn
> > > route_irq_to_guest into route_spi_to_guest and gic_route_irq_to_guest
> > > into gic_route_spi_to_guest.
> >
> > Those functions will be used for MSI sooner or later. I would prefer to
> > keep the current name and add an ASSERT.
>
> OK then, I can settle for that
Does that mean you are happy with the patch as is or were there other
changes you wanted?
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying
2014-09-09 13:53 ` Ian Campbell
@ 2014-09-09 22:29 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-09-09 22:29 UTC (permalink / raw)
To: Ian Campbell
Cc: xen-devel, Julien Grall, tim, stefano.stabellini, Stefano Stabellini
On Tue, 9 Sep 2014, Ian Campbell wrote:
> On Thu, 2014-08-07 at 17:39 +0100, Stefano Stabellini wrote:
> > On Thu, 7 Aug 2014, Julien Grall wrote:
> > > On 08/07/2014 05:31 PM, Stefano Stabellini wrote:
> > > > On Thu, 7 Aug 2014, Julien Grall wrote:
> > > >> On 08/07/2014 04:36 PM, Stefano Stabellini wrote:
> > > >>>> But I doubt we will have PPIs in future, there is more issues to handle
> > > >>>> (such as the number of VCPUs doesn't match the number of physical CPUs).
> > > >>>
> > > >>> In that case, why not call the two functions release_guest_spi and
> > > >>> gic_remove_spi_from_guest?
> > > >>
> > > >> Because the 2 functions to route the irq are called route_irq_to_guest
> > > >> and gic_route_irq_to_guest.
> > > >
> > > > I would prefer to avoid introducing more functions that look like they
> > > > can handle any irqs but actually they cannot by design.
> > >
> > > That why Xen checks the IRQ number at the beginning of route_irq_to_guest.
> > >
> > > > I would be OK with the asymmetry in function names. We could also turn
> > > > route_irq_to_guest into route_spi_to_guest and gic_route_irq_to_guest
> > > > into gic_route_spi_to_guest.
> > >
> > > Those functions will be used for MSI sooner or later. I would prefer to
> > > keep the current name and add an ASSERT.
> >
> > OK then, I can settle for that
>
> Does that mean you are happy with the patch as is or were there other
> changes you wanted?
The latter. I am waiting for an update on this patch with the ASSERTs.
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (8 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 09/21] xen/arm: Release IRQ routed to a domain when it's destroying Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 16:10 ` Stefano Stabellini
2014-08-29 12:34 ` Andrii Tseglytskyi
2014-07-31 15:00 ` [PATCH v2 11/21] xen/dts: Use unsigned int for MMIO and IRQ index Julien Grall
` (11 subsequent siblings)
21 siblings, 2 replies; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
The physdev sub-hypercalls PHYSDEVOP_{,map}_pirq allow the toolstack to
assign/deassign a physical IRQ to the guest (via the config options "irqs"
for xl).
For now, we allow only SPIs to be mapped to the guest.
The type MAP_PIRQ_TYPE_GSI is used for this purpose.
The virtual IRQ number is allocated by Xen. The toolstack as to specify
the number of SPIs handled by the vGIC via an hypercall.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
I'm wondering if we should introduce an alias of MAP_PIRQ_TYPE_GSI
for ARM. It's will be less confuse for the user.
Changes in v2:
- Add PHYSDEVOP_unmap_pirq
- Rework commit message
- Add functions to allocate/release a VIRQ
- is_routable_irq has been renamed into is_assignable_irq
---
xen/arch/arm/physdev.c | 120 +++++++++++++++++++++++++++++++++++++++++-
xen/arch/arm/vgic.c | 51 ++++++++++++++++++
xen/include/asm-arm/domain.h | 1 +
xen/include/asm-arm/vgic.h | 5 ++
4 files changed, 175 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
index 61b4a18..9333aa0 100644
--- a/xen/arch/arm/physdev.c
+++ b/xen/arch/arm/physdev.c
@@ -8,13 +8,129 @@
#include <xen/types.h>
#include <xen/lib.h>
#include <xen/errno.h>
+#include <xen/iocap.h>
+#include <xen/guest_access.h>
+#include <xsm/xsm.h>
+#include <asm/current.h>
#include <asm/hypercall.h>
+#include <public/physdev.h>
+static int physdev_map_pirq(domid_t domid, int type, int index, int *pirq_p)
+{
+ struct domain *d;
+ int ret;
+ int irq = index;
+ int virq = 0;
+
+ d = rcu_lock_domain_by_any_id(domid);
+ if ( d == NULL )
+ return -ESRCH;
+
+ ret = xsm_map_domain_pirq(XSM_TARGET, d);
+ if ( ret )
+ goto free_domain;
+
+ /* For now we only suport GSI */
+ if ( type != MAP_PIRQ_TYPE_GSI )
+ {
+ ret = -EINVAL;
+ dprintk(XENLOG_G_ERR, "dom%u: wrong map_pirq type 0x%x\n",
+ d->domain_id, type);
+ goto free_domain;
+ }
+
+ if ( !is_assignable_irq(irq) )
+ {
+ ret = -EINVAL;
+ dprintk(XENLOG_G_ERR, "IRQ%u is not routable to a guest\n", irq);
+ goto free_domain;
+ }
+
+ ret = -EPERM;
+ if ( !irq_access_permitted(current->domain, irq) )
+ goto free_domain;
+
+ virq = vgic_allocate_virq(d, irq);
+ ret = -EMFILE;
+ if ( virq == -1 )
+ goto free_domain;
+
+ ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
+
+ if ( !ret )
+ *pirq_p = virq;
+ else
+ vgic_free_virq(d, virq);
+
+free_domain:
+ rcu_unlock_domain(d);
+
+ return ret;
+}
+
+int physdev_unmap_pirq(domid_t domid, int pirq)
+{
+ struct domain *d;
+ int ret;
+
+ d = rcu_lock_domain_by_any_id(domid);
+ if ( d == NULL )
+ return -ESRCH;
+
+ ret = xsm_unmap_domain_pirq(XSM_TARGET, d);
+ if ( ret )
+ goto free_domain;
+
+ ret = release_guest_irq(d, pirq);
+ if ( ret )
+ goto free_domain;
+
+ vgic_free_virq(d, pirq);
+
+free_domain:
+ rcu_unlock_domain(d);
+
+ return ret;
+}
int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
- printk("%s %d cmd=%d: not implemented yet\n", __func__, __LINE__, cmd);
- return -ENOSYS;
+ int ret;
+
+ switch ( cmd )
+ {
+ case PHYSDEVOP_map_pirq:
+ {
+ physdev_map_pirq_t map;
+
+ ret = -EFAULT;
+ if ( copy_from_guest(&map, arg, 1) != 0 )
+ break;
+
+ ret = physdev_map_pirq(map.domid, map.type, map.index, &map.pirq);
+
+ if ( __copy_to_guest(arg, &map, 1) )
+ ret = -EFAULT;
+ }
+ break;
+
+ case PHYSDEVOP_unmap_pirq:
+ {
+ physdev_unmap_pirq_t unmap;
+
+ ret = -EFAULT;
+ if ( copy_from_guest(&unmap, arg, 1) != 0 )
+ break;
+
+ ret = physdev_unmap_pirq(unmap.domid, unmap.pirq);
+ }
+
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ return ret;
}
/*
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 2a5fc18..644742e 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -81,6 +81,8 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
return -ENODEV;
}
+ spin_lock_init(&d->arch.vgic.lock);
+
d->arch.vgic.shared_irqs =
xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
if ( d->arch.vgic.shared_irqs == NULL )
@@ -108,6 +110,11 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
d->arch.vgic.handler->domain_init(d);
+ d->arch.vgic.allocated_spis =
+ xzalloc_array(unsigned long, BITS_TO_LONGS(d->arch.vgic.nr_spis));
+ if ( !d->arch.vgic.allocated_spis )
+ return -ENOMEM;
+
return 0;
}
@@ -444,6 +451,50 @@ void arch_evtchn_inject(struct vcpu *v)
vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
}
+int vgic_allocate_virq(struct domain *d, unsigned int irq)
+{
+ unsigned int spi;
+ int virq = -1;
+
+ /* Hardware domain has IRQ mapped 1:1 */
+ if ( is_hardware_domain(d) )
+ return irq;
+
+ spin_lock(&d->arch.vgic.lock);
+
+ spi = find_first_zero_bit(d->arch.vgic.allocated_spis,
+ d->arch.vgic.nr_spis);
+
+ if ( spi >= d->arch.vgic.nr_spis )
+ goto unlock;
+
+ set_bit(spi, d->arch.vgic.allocated_spis);
+
+ virq = 32 + spi;
+
+unlock:
+ spin_unlock(&d->arch.vgic.lock);
+
+ return virq;
+}
+
+void vgic_free_virq(struct domain *d, unsigned int virq)
+{
+ unsigned int spi;
+
+ if ( is_hardware_domain(d) )
+ return;
+
+ if ( virq < 32 && virq >= vgic_num_irqs(d) )
+ return;
+
+ spi = virq - 32;
+
+ spin_lock(&d->arch.vgic.lock);
+ clear_bit(spi, d->arch.vgic.allocated_spis);
+ spin_unlock(&d->arch.vgic.lock);
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 44727b2..a4039c1 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -94,6 +94,7 @@ struct arch_domain
spinlock_t lock;
int ctlr;
int nr_spis; /* Number of SPIs */
+ unsigned long *allocated_spis; /* bitmap of SPIs allocated */
struct vgic_irq_rank *shared_irqs;
/*
* SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
index 84ae441..c5d8b2e 100644
--- a/xen/include/asm-arm/vgic.h
+++ b/xen/include/asm-arm/vgic.h
@@ -180,6 +180,11 @@ extern int vgic_to_sgi(struct vcpu *v, register_t sgir,
enum gic_sgi_mode irqmode, int virq,
unsigned long vcpu_mask);
extern void vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq);
+
+/* Allocate a VIRQ number of a guest SPI */
+extern int vgic_allocate_virq(struct domain *d, unsigned int irq);
+extern void vgic_free_virq(struct domain *d, unsigned int irq);
+
#endif /* __ASM_ARM_VGIC_H__ */
/*
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq
2014-07-31 15:00 ` [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq Julien Grall
@ 2014-08-06 16:10 ` Stefano Stabellini
2014-08-29 12:34 ` Andrii Tseglytskyi
1 sibling, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:10 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> The physdev sub-hypercalls PHYSDEVOP_{,map}_pirq allow the toolstack to
> assign/deassign a physical IRQ to the guest (via the config options "irqs"
> for xl).
> For now, we allow only SPIs to be mapped to the guest.
> The type MAP_PIRQ_TYPE_GSI is used for this purpose.
>
> The virtual IRQ number is allocated by Xen. The toolstack as to specify
> the number of SPIs handled by the vGIC via an hypercall.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
> I'm wondering if we should introduce an alias of MAP_PIRQ_TYPE_GSI
> for ARM. It's will be less confuse for the user.
Just improve the error message below, maybe:
"wrong map_pirq type 0x%x, only MAP_PIRQ_TYPE_GSI is supported"
> Changes in v2:
> - Add PHYSDEVOP_unmap_pirq
> - Rework commit message
> - Add functions to allocate/release a VIRQ
> - is_routable_irq has been renamed into is_assignable_irq
> ---
> xen/arch/arm/physdev.c | 120 +++++++++++++++++++++++++++++++++++++++++-
> xen/arch/arm/vgic.c | 51 ++++++++++++++++++
> xen/include/asm-arm/domain.h | 1 +
> xen/include/asm-arm/vgic.h | 5 ++
> 4 files changed, 175 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> index 61b4a18..9333aa0 100644
> --- a/xen/arch/arm/physdev.c
> +++ b/xen/arch/arm/physdev.c
> @@ -8,13 +8,129 @@
> #include <xen/types.h>
> #include <xen/lib.h>
> #include <xen/errno.h>
> +#include <xen/iocap.h>
> +#include <xen/guest_access.h>
> +#include <xsm/xsm.h>
> +#include <asm/current.h>
> #include <asm/hypercall.h>
> +#include <public/physdev.h>
>
> +static int physdev_map_pirq(domid_t domid, int type, int index, int *pirq_p)
> +{
> + struct domain *d;
> + int ret;
> + int irq = index;
> + int virq = 0;
> +
> + d = rcu_lock_domain_by_any_id(domid);
> + if ( d == NULL )
> + return -ESRCH;
> +
> + ret = xsm_map_domain_pirq(XSM_TARGET, d);
> + if ( ret )
> + goto free_domain;
> +
> + /* For now we only suport GSI */
> + if ( type != MAP_PIRQ_TYPE_GSI )
> + {
> + ret = -EINVAL;
> + dprintk(XENLOG_G_ERR, "dom%u: wrong map_pirq type 0x%x\n",
> + d->domain_id, type);
> + goto free_domain;
> + }
> +
> + if ( !is_assignable_irq(irq) )
> + {
> + ret = -EINVAL;
> + dprintk(XENLOG_G_ERR, "IRQ%u is not routable to a guest\n", irq);
> + goto free_domain;
> + }
> +
> + ret = -EPERM;
> + if ( !irq_access_permitted(current->domain, irq) )
> + goto free_domain;
> +
> + virq = vgic_allocate_virq(d, irq);
> + ret = -EMFILE;
> + if ( virq == -1 )
> + goto free_domain;
> +
> + ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
> +
> + if ( !ret )
> + *pirq_p = virq;
> + else
> + vgic_free_virq(d, virq);
> +
> +free_domain:
> + rcu_unlock_domain(d);
> +
> + return ret;
> +}
> +
> +int physdev_unmap_pirq(domid_t domid, int pirq)
> +{
> + struct domain *d;
> + int ret;
> +
> + d = rcu_lock_domain_by_any_id(domid);
> + if ( d == NULL )
> + return -ESRCH;
> +
> + ret = xsm_unmap_domain_pirq(XSM_TARGET, d);
> + if ( ret )
> + goto free_domain;
> +
> + ret = release_guest_irq(d, pirq);
> + if ( ret )
> + goto free_domain;
> +
> + vgic_free_virq(d, pirq);
> +
> +free_domain:
> + rcu_unlock_domain(d);
> +
> + return ret;
> +}
>
> int do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> {
> - printk("%s %d cmd=%d: not implemented yet\n", __func__, __LINE__, cmd);
> - return -ENOSYS;
> + int ret;
> +
> + switch ( cmd )
> + {
> + case PHYSDEVOP_map_pirq:
> + {
> + physdev_map_pirq_t map;
> +
> + ret = -EFAULT;
> + if ( copy_from_guest(&map, arg, 1) != 0 )
> + break;
> +
> + ret = physdev_map_pirq(map.domid, map.type, map.index, &map.pirq);
> +
> + if ( __copy_to_guest(arg, &map, 1) )
> + ret = -EFAULT;
> + }
> + break;
> +
> + case PHYSDEVOP_unmap_pirq:
> + {
> + physdev_unmap_pirq_t unmap;
> +
> + ret = -EFAULT;
> + if ( copy_from_guest(&unmap, arg, 1) != 0 )
> + break;
> +
> + ret = physdev_unmap_pirq(unmap.domid, unmap.pirq);
> + }
> +
> + default:
> + ret = -ENOSYS;
> + break;
> + }
> +
> + return ret;
> }
>
> /*
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 2a5fc18..644742e 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -81,6 +81,8 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> return -ENODEV;
> }
>
> + spin_lock_init(&d->arch.vgic.lock);
> +
> d->arch.vgic.shared_irqs =
> xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_RANKS(d));
> if ( d->arch.vgic.shared_irqs == NULL )
> @@ -108,6 +110,11 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>
> d->arch.vgic.handler->domain_init(d);
>
> + d->arch.vgic.allocated_spis =
> + xzalloc_array(unsigned long, BITS_TO_LONGS(d->arch.vgic.nr_spis));
> + if ( !d->arch.vgic.allocated_spis )
> + return -ENOMEM;
> +
> return 0;
> }
>
> @@ -444,6 +451,50 @@ void arch_evtchn_inject(struct vcpu *v)
> vgic_vcpu_inject_irq(v, v->domain->arch.evtchn_irq);
> }
>
> +int vgic_allocate_virq(struct domain *d, unsigned int irq)
> +{
> + unsigned int spi;
> + int virq = -1;
> +
> + /* Hardware domain has IRQ mapped 1:1 */
> + if ( is_hardware_domain(d) )
> + return irq;
> +
> + spin_lock(&d->arch.vgic.lock);
> +
> + spi = find_first_zero_bit(d->arch.vgic.allocated_spis,
> + d->arch.vgic.nr_spis);
> +
> + if ( spi >= d->arch.vgic.nr_spis )
> + goto unlock;
> +
> + set_bit(spi, d->arch.vgic.allocated_spis);
> +
> + virq = 32 + spi;
> +
> +unlock:
> + spin_unlock(&d->arch.vgic.lock);
> +
> + return virq;
> +}
> +
> +void vgic_free_virq(struct domain *d, unsigned int virq)
> +{
> + unsigned int spi;
> +
> + if ( is_hardware_domain(d) )
> + return;
> +
> + if ( virq < 32 && virq >= vgic_num_irqs(d) )
> + return;
> +
> + spi = virq - 32;
> +
> + spin_lock(&d->arch.vgic.lock);
> + clear_bit(spi, d->arch.vgic.allocated_spis);
> + spin_unlock(&d->arch.vgic.lock);
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 44727b2..a4039c1 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -94,6 +94,7 @@ struct arch_domain
> spinlock_t lock;
> int ctlr;
> int nr_spis; /* Number of SPIs */
> + unsigned long *allocated_spis; /* bitmap of SPIs allocated */
> struct vgic_irq_rank *shared_irqs;
> /*
> * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
> diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h
> index 84ae441..c5d8b2e 100644
> --- a/xen/include/asm-arm/vgic.h
> +++ b/xen/include/asm-arm/vgic.h
> @@ -180,6 +180,11 @@ extern int vgic_to_sgi(struct vcpu *v, register_t sgir,
> enum gic_sgi_mode irqmode, int virq,
> unsigned long vcpu_mask);
> extern void vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq);
> +
> +/* Allocate a VIRQ number of a guest SPI */
> +extern int vgic_allocate_virq(struct domain *d, unsigned int irq);
> +extern void vgic_free_virq(struct domain *d, unsigned int irq);
> +
> #endif /* __ASM_ARM_VGIC_H__ */
>
> /*
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq
2014-07-31 15:00 ` [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq Julien Grall
2014-08-06 16:10 ` Stefano Stabellini
@ 2014-08-29 12:34 ` Andrii Tseglytskyi
2014-08-29 19:08 ` Julien Grall
1 sibling, 1 reply; 119+ messages in thread
From: Andrii Tseglytskyi @ 2014-08-29 12:34 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, Stefano Stabellini, Ian Campbell, Tim Deegan
Hi Julien,
> +static int physdev_map_pirq(domid_t domid, int type, int index, int *pirq_p)
> +{
> + struct domain *d;
> + int ret;
> + int irq = index;
> + int virq = 0;
> +
> + d = rcu_lock_domain_by_any_id(domid);
> + if ( d == NULL )
> + return -ESRCH;
> +
> + ret = xsm_map_domain_pirq(XSM_TARGET, d);
> + if ( ret )
> + goto free_domain;
> +
> + /* For now we only suport GSI */
> + if ( type != MAP_PIRQ_TYPE_GSI )
> + {
> + ret = -EINVAL;
> + dprintk(XENLOG_G_ERR, "dom%u: wrong map_pirq type 0x%x\n",
> + d->domain_id, type);
> + goto free_domain;
> + }
> +
> + if ( !is_assignable_irq(irq) )
> + {
> + ret = -EINVAL;
> + dprintk(XENLOG_G_ERR, "IRQ%u is not routable to a guest\n", irq);
> + goto free_domain;
> + }
> +
> + ret = -EPERM;
> + if ( !irq_access_permitted(current->domain, irq) )
> + goto free_domain;
I have a question here. This code is executed when dom0 creates domU.
current->domain points to dom0 here, in other words access to IRQ must
be permitted for dom0. Right ?
But in this case following call of route_irq_to_guest() fails with message:
(XEN) irq.c:459: IRQ 53 is already used by domain 0
I reproduced this in the following steps:
- define IRQ in device tree, which is used by dom0, as result IRQ is
mapped to dom0
- after I defined it in domU configuration file, tried to launch domU
and got this error
What I'm doing wrong here?
> +
> + virq = vgic_allocate_virq(d, irq);
> + ret = -EMFILE;
> + if ( virq == -1 )
> + goto free_domain;
> +
> + ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
> +
This call fails for me with the message:
(XEN) irq.c:440: IRQ 53 has not been configured.
Locally I added the following code, which sets IRQ type:
diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
index 9333aa0..e853e80 100644
--- a/xen/arch/arm/physdev.c
+++ b/xen/arch/arm/physdev.c
@@ -55,6 +55,7 @@ static int physdev_map_pirq(domid_t domid, int type,
int index, int *pirq_p)
if ( virq == -1 )
goto free_domain;
+ irq_set_spi_type(irq, DT_IRQ_TYPE_LEVEL_MASK);
ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
if ( !ret )
Should it be added, or I need to do this in a different way - some
extra options for IRQ in config file, or something else?
Regards,
Andrii
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq
2014-08-29 12:34 ` Andrii Tseglytskyi
@ 2014-08-29 19:08 ` Julien Grall
2014-08-29 19:44 ` Andrii Tseglytskyi
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-29 19:08 UTC (permalink / raw)
To: Andrii Tseglytskyi
Cc: xen-devel, Stefano Stabellini, Ian Campbell, Tim Deegan
On 29/08/14 08:34, Andrii Tseglytskyi wrote:
> Hi Julien,
Hi Andrii,
>> +static int physdev_map_pirq(domid_t domid, int type, int index, int *pirq_p)
>> +{
>> + struct domain *d;
>> + int ret;
>> + int irq = index;
>> + int virq = 0;
>> +
>> + d = rcu_lock_domain_by_any_id(domid);
>> + if ( d == NULL )
>> + return -ESRCH;
>> +
>> + ret = xsm_map_domain_pirq(XSM_TARGET, d);
>> + if ( ret )
>> + goto free_domain;
>> +
>> + /* For now we only suport GSI */
>> + if ( type != MAP_PIRQ_TYPE_GSI )
>> + {
>> + ret = -EINVAL;
>> + dprintk(XENLOG_G_ERR, "dom%u: wrong map_pirq type 0x%x\n",
>> + d->domain_id, type);
>> + goto free_domain;
>> + }
>> +
>> + if ( !is_assignable_irq(irq) )
>> + {
>> + ret = -EINVAL;
>> + dprintk(XENLOG_G_ERR, "IRQ%u is not routable to a guest\n", irq);
>> + goto free_domain;
>> + }
>> +
>> + ret = -EPERM;
>> + if ( !irq_access_permitted(current->domain, irq) )
>> + goto free_domain;
>
> I have a question here. This code is executed when dom0 creates domU.
> current->domain points to dom0 here, in other words access to IRQ must
> be permitted for dom0. Right ?
> But in this case following call of route_irq_to_guest() fails with message:
>
> (XEN) irq.c:459: IRQ 53 is already used by domain 0
>
> I reproduced this in the following steps:
> - define IRQ in device tree, which is used by dom0, as result IRQ is
> mapped to dom0
> - after I defined it in domU configuration file, tried to launch domU
> and got this error
>
> What I'm doing wrong here?
PHYSDEVOP_map_pirq can assign an IRQ to a domain if it's free.
My series is assuming that the device is not used by anyone (status =
"disabled" property in the device tree node).
In general, a device should not be shared between domains. If you need
to use it at boot in DOM0, for now, you will have to disable the drivers
and unmap every IRQ (see PHYSDEVOP_unmap_irq).
>> +
>> + virq = vgic_allocate_virq(d, irq);
>> + ret = -EMFILE;
>> + if ( virq == -1 )
>> + goto free_domain;
>> +
>> + ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
>> +
>
> This call fails for me with the message:
>
> (XEN) irq.c:440: IRQ 53 has not been configured.
>
> Locally I added the following code, which sets IRQ type:
>
> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
> index 9333aa0..e853e80 100644
> --- a/xen/arch/arm/physdev.c
> +++ b/xen/arch/arm/physdev.c
> @@ -55,6 +55,7 @@ static int physdev_map_pirq(domid_t domid, int type,
> int index, int *pirq_p)
> if ( virq == -1 )
> goto free_domain;
>
> + irq_set_spi_type(irq, DT_IRQ_TYPE_LEVEL_MASK);
> ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
>
> if ( !ret )
>
>
> Should it be added, or I need to do this in a different way - some
> extra options for IRQ in config file, or something else?
At boot Xen is browsing the device tree to configure every IRQ. Does you
IRQ describe in the device tree?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq
2014-08-29 19:08 ` Julien Grall
@ 2014-08-29 19:44 ` Andrii Tseglytskyi
0 siblings, 0 replies; 119+ messages in thread
From: Andrii Tseglytskyi @ 2014-08-29 19:44 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, Stefano Stabellini, Ian Campbell, Tim Deegan
Hi Julien,
On Fri, Aug 29, 2014 at 10:08 PM, Julien Grall <julien.grall@linaro.org> wrote:
>
>
> On 29/08/14 08:34, Andrii Tseglytskyi wrote:
>>
>> Hi Julien,
>
>
> Hi Andrii,
>
>
>>> +static int physdev_map_pirq(domid_t domid, int type, int index, int
>>> *pirq_p)
>>> +{
>>> + struct domain *d;
>>> + int ret;
>>> + int irq = index;
>>> + int virq = 0;
>>> +
>>> + d = rcu_lock_domain_by_any_id(domid);
>>> + if ( d == NULL )
>>> + return -ESRCH;
>>> +
>>> + ret = xsm_map_domain_pirq(XSM_TARGET, d);
>>> + if ( ret )
>>> + goto free_domain;
>>> +
>>> + /* For now we only suport GSI */
>>> + if ( type != MAP_PIRQ_TYPE_GSI )
>>> + {
>>> + ret = -EINVAL;
>>> + dprintk(XENLOG_G_ERR, "dom%u: wrong map_pirq type 0x%x\n",
>>> + d->domain_id, type);
>>> + goto free_domain;
>>> + }
>>> +
>>> + if ( !is_assignable_irq(irq) )
>>> + {
>>> + ret = -EINVAL;
>>> + dprintk(XENLOG_G_ERR, "IRQ%u is not routable to a guest\n",
>>> irq);
>>> + goto free_domain;
>>> + }
>>> +
>>> + ret = -EPERM;
>>> + if ( !irq_access_permitted(current->domain, irq) )
>>> + goto free_domain;
>>
>>
>> I have a question here. This code is executed when dom0 creates domU.
>> current->domain points to dom0 here, in other words access to IRQ must
>> be permitted for dom0. Right ?
>> But in this case following call of route_irq_to_guest() fails with
>> message:
>>
>> (XEN) irq.c:459: IRQ 53 is already used by domain 0
>>
>> I reproduced this in the following steps:
>> - define IRQ in device tree, which is used by dom0, as result IRQ is
>> mapped to dom0
>> - after I defined it in domU configuration file, tried to launch domU
>> and got this error
>>
>> What I'm doing wrong here?
>
>
> PHYSDEVOP_map_pirq can assign an IRQ to a domain if it's free.
>
> My series is assuming that the device is not used by anyone (status =
> "disabled" property in the device tree node).
>
> In general, a device should not be shared between domains. If you need to
> use it at boot in DOM0, for now, you will have to disable the drivers and
> unmap every IRQ (see PHYSDEVOP_unmap_irq).
>
Do you mean, that status=disabled in device tree will be enough to
pass irq_access_permitted(current->domain, irq) check ?
I don't need to share devices between domains and I don't need to have
these IRQs in dom0. I'm just trying to figure out how
to pass irq_access_permitted() check. Because I took a look inside it
and see that it checks rangeset, which may be added only by
irq_permit_access(). And I suppose that this call provides exclusive
access to IRQ for selected domain.
>
>>> +
>>> + virq = vgic_allocate_virq(d, irq);
>>> + ret = -EMFILE;
>>> + if ( virq == -1 )
>>> + goto free_domain;
>>> +
>>> + ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
>>> +
>>
>>
>> This call fails for me with the message:
>>
>> (XEN) irq.c:440: IRQ 53 has not been configured.
>>
>> Locally I added the following code, which sets IRQ type:
>>
>> diff --git a/xen/arch/arm/physdev.c b/xen/arch/arm/physdev.c
>> index 9333aa0..e853e80 100644
>> --- a/xen/arch/arm/physdev.c
>> +++ b/xen/arch/arm/physdev.c
>> @@ -55,6 +55,7 @@ static int physdev_map_pirq(domid_t domid, int type,
>> int index, int *pirq_p)
>> if ( virq == -1 )
>> goto free_domain;
>>
>> + irq_set_spi_type(irq, DT_IRQ_TYPE_LEVEL_MASK);
>> ret = route_irq_to_guest(d, virq, irq, "routed IRQ");
>>
>> if ( !ret )
>>
>>
>> Should it be added, or I need to do this in a different way - some
>> extra options for IRQ in config file, or something else?
>
>
> At boot Xen is browsing the device tree to configure every IRQ. Does you IRQ
> describe in the device tree?
>
This is related to previous comment. If I describe IRQ in device tree
- it is assigned exclusively to dom0.
If I don't describe it - I need to set it type here.
> Regards,
>
> --
> Julien Grall
--
Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 11/21] xen/dts: Use unsigned int for MMIO and IRQ index
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (9 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 10/21] xen/arm: Implement hypercall PHYSDEVOP_{, un}map_pirq Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 16:12 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 12/21] xen/dts: Provide an helper to get a DT node from a path provided by a guest Julien Grall
` (10 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
There is no reason to use signed integer for an index. Futhermore, this will
avoid possible issue when theses functions will be exposed to the guest
via new hypercalls.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Use unsigned int instead fancy one like unsigned or uint32_t
---
xen/common/device_tree.c | 11 ++++++-----
xen/include/xen/device_tree.h | 7 ++++---
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index f72b2e9..315b62a 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -496,7 +496,7 @@ static const struct dt_bus *dt_match_bus(const struct dt_device_node *np)
}
static const __be32 *dt_get_address(const struct dt_device_node *dev,
- int index, u64 *size,
+ unsigned int index, u64 *size,
unsigned int *flags)
{
const __be32 *prop;
@@ -683,7 +683,7 @@ bail:
}
/* dt_device_address - Translate device tree address and return it */
-int dt_device_get_address(const struct dt_device_node *dev, int index,
+int dt_device_get_address(const struct dt_device_node *dev, unsigned int index,
u64 *addr, u64 *size)
{
const __be32 *addrp;
@@ -1006,7 +1006,8 @@ fail:
return -EINVAL;
}
-int dt_device_get_raw_irq(const struct dt_device_node *device, int index,
+int dt_device_get_raw_irq(const struct dt_device_node *device,
+ unsigned int index,
struct dt_raw_irq *out_irq)
{
const struct dt_device_node *p;
@@ -1014,7 +1015,7 @@ int dt_device_get_raw_irq(const struct dt_device_node *device, int index,
u32 intsize, intlen;
int res = -EINVAL;
- dt_dprintk("dt_device_get_raw_irq: dev=%s, index=%d\n",
+ dt_dprintk("dt_device_get_raw_irq: dev=%s, index=%u\n",
device->full_name, index);
/* Get the interrupts property */
@@ -1065,7 +1066,7 @@ int dt_irq_translate(const struct dt_raw_irq *raw,
&out_irq->irq, &out_irq->type);
}
-int dt_device_get_irq(const struct dt_device_node *device, int index,
+int dt_device_get_irq(const struct dt_device_node *device, unsigned int index,
struct dt_irq *out_irq)
{
struct dt_raw_irq raw;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 08db8bc..2115a0c 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -460,7 +460,7 @@ const struct dt_device_node *dt_get_parent(const struct dt_device_node *node);
* This function resolves an address, walking the tree, for a give
* device-tree node. It returns 0 on success.
*/
-int dt_device_get_address(const struct dt_device_node *dev, int index,
+int dt_device_get_address(const struct dt_device_node *dev, unsigned int index,
u64 *addr, u64 *size);
/**
@@ -490,7 +490,7 @@ unsigned int dt_number_of_address(const struct dt_device_node *device);
* This function resolves an interrupt, walking the tree, for a given
* device-tree node. It's the high level pendant to dt_device_get_raw_irq().
*/
-int dt_device_get_irq(const struct dt_device_node *device, int index,
+int dt_device_get_irq(const struct dt_device_node *device, unsigned int index,
struct dt_irq *irq);
/**
@@ -502,7 +502,8 @@ int dt_device_get_irq(const struct dt_device_node *device, int index,
* This function resolves an interrupt for a device, no translation is
* made. dt_irq_translate can be called after.
*/
-int dt_device_get_raw_irq(const struct dt_device_node *device, int index,
+int dt_device_get_raw_irq(const struct dt_device_node *device,
+ unsigned int index,
struct dt_raw_irq *irq);
/**
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 11/21] xen/dts: Use unsigned int for MMIO and IRQ index
2014-07-31 15:00 ` [PATCH v2 11/21] xen/dts: Use unsigned int for MMIO and IRQ index Julien Grall
@ 2014-08-06 16:12 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:12 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> There is no reason to use signed integer for an index. Futhermore, this will
> avoid possible issue when theses functions will be exposed to the guest
> via new hypercalls.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
> Changes in v2:
> - Use unsigned int instead fancy one like unsigned or uint32_t
> ---
> xen/common/device_tree.c | 11 ++++++-----
> xen/include/xen/device_tree.h | 7 ++++---
> 2 files changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index f72b2e9..315b62a 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -496,7 +496,7 @@ static const struct dt_bus *dt_match_bus(const struct dt_device_node *np)
> }
>
> static const __be32 *dt_get_address(const struct dt_device_node *dev,
> - int index, u64 *size,
> + unsigned int index, u64 *size,
> unsigned int *flags)
> {
> const __be32 *prop;
> @@ -683,7 +683,7 @@ bail:
> }
>
> /* dt_device_address - Translate device tree address and return it */
> -int dt_device_get_address(const struct dt_device_node *dev, int index,
> +int dt_device_get_address(const struct dt_device_node *dev, unsigned int index,
> u64 *addr, u64 *size)
> {
> const __be32 *addrp;
> @@ -1006,7 +1006,8 @@ fail:
> return -EINVAL;
> }
>
> -int dt_device_get_raw_irq(const struct dt_device_node *device, int index,
> +int dt_device_get_raw_irq(const struct dt_device_node *device,
> + unsigned int index,
> struct dt_raw_irq *out_irq)
> {
> const struct dt_device_node *p;
> @@ -1014,7 +1015,7 @@ int dt_device_get_raw_irq(const struct dt_device_node *device, int index,
> u32 intsize, intlen;
> int res = -EINVAL;
>
> - dt_dprintk("dt_device_get_raw_irq: dev=%s, index=%d\n",
> + dt_dprintk("dt_device_get_raw_irq: dev=%s, index=%u\n",
> device->full_name, index);
>
> /* Get the interrupts property */
> @@ -1065,7 +1066,7 @@ int dt_irq_translate(const struct dt_raw_irq *raw,
> &out_irq->irq, &out_irq->type);
> }
>
> -int dt_device_get_irq(const struct dt_device_node *device, int index,
> +int dt_device_get_irq(const struct dt_device_node *device, unsigned int index,
> struct dt_irq *out_irq)
> {
> struct dt_raw_irq raw;
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 08db8bc..2115a0c 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -460,7 +460,7 @@ const struct dt_device_node *dt_get_parent(const struct dt_device_node *node);
> * This function resolves an address, walking the tree, for a give
> * device-tree node. It returns 0 on success.
> */
> -int dt_device_get_address(const struct dt_device_node *dev, int index,
> +int dt_device_get_address(const struct dt_device_node *dev, unsigned int index,
> u64 *addr, u64 *size);
>
> /**
> @@ -490,7 +490,7 @@ unsigned int dt_number_of_address(const struct dt_device_node *device);
> * This function resolves an interrupt, walking the tree, for a given
> * device-tree node. It's the high level pendant to dt_device_get_raw_irq().
> */
> -int dt_device_get_irq(const struct dt_device_node *device, int index,
> +int dt_device_get_irq(const struct dt_device_node *device, unsigned int index,
> struct dt_irq *irq);
>
> /**
> @@ -502,7 +502,8 @@ int dt_device_get_irq(const struct dt_device_node *device, int index,
> * This function resolves an interrupt for a device, no translation is
> * made. dt_irq_translate can be called after.
> */
> -int dt_device_get_raw_irq(const struct dt_device_node *device, int index,
> +int dt_device_get_raw_irq(const struct dt_device_node *device,
> + unsigned int index,
> struct dt_raw_irq *irq);
>
> /**
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 12/21] xen/dts: Provide an helper to get a DT node from a path provided by a guest
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (10 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 11/21] xen/dts: Use unsigned int for MMIO and IRQ index Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-09-09 13:55 ` Ian Campbell
2014-07-31 15:00 ` [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information Julien Grall
` (9 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- guest_copy_string_from_guest has been renamed into
safe_copy_string_from_guest
---
xen/common/device_tree.c | 19 +++++++++++++++++++
xen/include/xen/device_tree.h | 17 +++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 315b62a..071109e 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -13,6 +13,7 @@
#include <xen/config.h>
#include <xen/types.h>
#include <xen/init.h>
+#include <xen/guest_access.h>
#include <xen/device_tree.h>
#include <xen/kernel.h>
#include <xen/lib.h>
@@ -277,6 +278,24 @@ struct dt_device_node *dt_find_node_by_path(const char *path)
return np;
}
+int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
+ struct dt_device_node **node)
+{
+ char *path;
+ int ret;
+
+ ret = safe_copy_string_from_guest(u_path, &path, u_plen,
+ DEVICE_TREE_MAX_PATHLEN);
+ if ( ret )
+ return ret;
+
+ *node = dt_find_node_by_path(path);
+
+ xfree(path);
+
+ return (*node == NULL) ? -ESRCH : 0;
+}
+
struct dt_device_node *dt_find_node_by_alias(const char *alias)
{
const struct dt_alias_prop *app;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 2115a0c..6dc34df 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -20,6 +20,9 @@
#define DEVICE_TREE_MAX_DEPTH 16
+/* This limit is used by the hypercalls to restrict the size of the path */
+#define DEVICE_TREE_MAX_PATHLEN 1024
+
/*
* Struct used for matching a device
*/
@@ -442,6 +445,20 @@ struct dt_device_node *dt_find_node_by_alias(const char *alias);
*/
struct dt_device_node *dt_find_node_by_path(const char *path);
+
+/**
+ * dt_find_node_by_gpath - Same as dt_find_node_by_path but retrieve the
+ * path from the guest
+ *
+ * @u_path: Xen Guest handle to the buffer containing the path
+ * @u_plen: Length of the buffer
+ * @node: TODO
+ *
+ * Return 0 if succeed otherwise -errno
+ */
+int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
+ struct dt_device_node **node);
+
/**
* dt_get_parent - Get a node's parent if any
* @node: Node to get parent
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (11 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 12/21] xen/dts: Provide an helper to get a DT node from a path provided by a guest Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-01 8:50 ` Jan Beulich
2014-07-31 15:00 ` [PATCH v2 14/21] xen/passthrough: Introduce iommu_construct Julien Grall
` (8 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, tim, Julien Grall, Ian Jackson, stefano.stabellini,
Jan Beulich
DOM0 doesn't provide a generic way to get information about a device tree
node. If we want to do it in userspace, we will have to duplicate the
MMIO/IRQ translation from Xen. Therefore, we can let the hypervisor
doing the job for us and get nearly all the informations.
This new domctl operation will let the toolstack get the IRQ/MMIO regions
and the compatible string. Most the device node can be described with only
theses 3 items. If we need to add a specific properties, then we will have
to implement it in userspace (some idea was to use a configuration file
describing the additional properties).
The hypercall is divided in 4 parts:
- GET_INFO: get the numbers of IRQ/MMIO and the size of the
compatible string;
- GET_IRQ: get the IRQ by index. If the IRQ is not routable (i.e not
an SPIs), the errno will be set to -EINVAL;
- GET_MMIO: get the MMIO range by index. If the base and the size of
is not page-aligned, the errno will be set to -EINVAL;
- GET_COMPAT: get the compatible string
All the information will be accessible if the device is not used by Xen
and protected by an IOMMU.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
---
I'm wondering if we can let the toolstack retrieve device information for
every device not used by Xen. This would allow embedded guys using passthrough
"easily" when their devices are not under an IOMMU.
TODO: Missing XSM stubs for this new DOMCTL. I'm not sure how to deal with it
Changes in v2:
- is_routable_irq has been renamed to is_assignable_irq
- rename clen field of the compatible structure into len
- use xen_pfn_t for both mfn and nr_mfn fields
- move as a DOMCTL op rather than PHYSDEV one. So it will be
easy to modify the interface in the future. Hence, this is aim
to be used only by the toolstack
---
tools/libxc/xc_domain.c | 136 +++++++++++++++++++++++++++++++++++++++++
tools/libxc/xenctrl.h | 38 ++++++++++++
xen/common/device_tree.c | 113 ++++++++++++++++++++++++++++++++++
xen/common/domctl.c | 10 +++
xen/include/public/domctl.h | 40 ++++++++++++
xen/include/xen/device_tree.h | 3 +
6 files changed, 340 insertions(+)
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index 1348905..4cc0474 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -2145,6 +2145,142 @@ int xc_domain_set_max_evtchn(xc_interface *xch, uint32_t domid,
return do_domctl(xch, &domctl);
}
+int xc_dtdev_getinfo(xc_interface *xch,
+ char *path,
+ xc_dtdev_info_t *info)
+{
+ int rc;
+ size_t size = strlen(path);
+ DECLARE_DOMCTL;
+ struct xen_domctl_dtdev_op *op = &domctl.u.dtdev_op;
+ DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+ if ( xc_hypercall_bounce_pre(xch, path) )
+ return -1;
+
+ domctl.cmd = XEN_DOMCTL_dtdev_op;
+ op->op = DOMCTL_DTDEV_GET_INFO;
+ op->plen = size;
+ set_xen_guest_handle(op->path, path);
+
+ rc = do_domctl(xch, &domctl);
+
+ xc_hypercall_bounce_post(xch, path);
+
+ if ( !rc )
+ {
+ info->num_irqs = op->u.info.num_irqs;
+ info->num_mmios = op->u.info.num_mmios;
+ info->compat_len = op->u.info.compat_len;
+ }
+
+ return rc;
+}
+
+int xc_dtdev_getirq(xc_interface *xch,
+ char *path,
+ uint32_t index,
+ xc_dtdev_irq_t *irq)
+{
+ int rc;
+ size_t size = strlen(path);
+ DECLARE_DOMCTL;
+ struct xen_domctl_dtdev_op *op = &domctl.u.dtdev_op;
+ DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+ if ( xc_hypercall_bounce_pre(xch, path) )
+ return -1;
+
+ domctl.cmd = XEN_DOMCTL_dtdev_op;
+ op->op = DOMCTL_DTDEV_GET_IRQ;
+ op->plen = size;
+ op->index = index;
+ set_xen_guest_handle(op->path, path);
+
+ rc = do_domctl(xch, &domctl);
+
+ xc_hypercall_bounce_post(xch, path);
+
+ if ( !rc )
+ {
+ irq->irq = op->u.irq.irq;
+ irq->type = op->u.irq.type;
+ }
+
+ return rc;
+}
+
+int xc_dtdev_getmmio(xc_interface *xch,
+ char *path,
+ uint32_t index,
+ xc_dtdev_mmio_t *mmio)
+{
+ int rc;
+ size_t size = strlen(path);
+ DECLARE_DOMCTL;
+ struct xen_domctl_dtdev_op *op = &domctl.u.dtdev_op;
+ DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+ if ( xc_hypercall_bounce_pre(xch, path) )
+ return -1;
+
+ op->op = DOMCTL_DTDEV_GET_MMIO;
+ op->plen = size;
+ op->index = index;
+ set_xen_guest_handle(op->path, path);
+
+ rc = do_domctl(xch, &domctl);
+
+ xc_hypercall_bounce_post(xch, path);
+
+ if ( !rc )
+ {
+ mmio->mfn = op->u.mmio.mfn;
+ mmio->nr_mfn = op->u.mmio.nr_mfn;
+ }
+
+ return rc;
+}
+
+int xc_dtdev_getcompat(xc_interface *xch,
+ char *path,
+ char *compat,
+ uint32_t *clen)
+{
+ int rc;
+ size_t size = strlen(path);
+ DECLARE_DOMCTL;
+ struct xen_domctl_dtdev_op *op = &domctl.u.dtdev_op;
+ DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+ DECLARE_HYPERCALL_BOUNCE(compat, *clen, XC_HYPERCALL_BUFFER_BOUNCE_OUT);
+
+ if ( xc_hypercall_bounce_pre(xch, path) )
+ return -1;
+
+ rc = -1;
+ if ( xc_hypercall_bounce_pre(xch, compat) )
+ goto out;
+
+ op->op = DOMCTL_DTDEV_GET_COMPAT;
+ op->plen = size;
+ set_xen_guest_handle(op->path, path);
+
+ op->u.compat.len = *clen;
+ set_xen_guest_handle(op->u.compat.compat, compat);
+
+ rc = do_domctl(xch, &domctl);
+
+ if ( !rc )
+ *clen = op->u.compat.len;
+
+ xc_hypercall_bounce_post(xch, compat);
+
+out:
+ xc_hypercall_bounce_post(xch, path);
+
+ return rc;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index cdda4e5..c549753 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -1208,6 +1208,44 @@ int xc_physdev_pci_access_modify(xc_interface *xch,
int func,
int enable);
+typedef struct xc_dtdev_info {
+ uint32_t num_irqs;
+ uint32_t num_mmios;
+ uint32_t compat_len;
+} xc_dtdev_info_t;
+
+/* Retrieve device tree information */
+
+int xc_dtdev_getinfo(xc_interface *xch,
+ char *path,
+ xc_dtdev_info_t *info);
+
+typedef struct xc_dtdev_irq {
+ uint32_t irq;
+ /* TODO: Maybe an enum here? */
+ uint32_t type;
+} xc_dtdev_irq_t;
+
+int xc_dtdev_getirq(xc_interface *xch,
+ char *path,
+ uint32_t index,
+ xc_dtdev_irq_t *irq);
+
+typedef struct xc_dtdev_mmio {
+ uint64_t mfn;
+ uint64_t nr_mfn;
+} xc_dtdev_mmio_t;
+
+int xc_dtdev_getmmio(xc_interface *xch,
+ char *path,
+ uint32_t index,
+ xc_dtdev_mmio_t *mmio);
+
+int xc_dtdev_getcompat(xc_interface *xch,
+ char *path,
+ char *compat,
+ uint32_t *clen);
+
int xc_readconsolering(xc_interface *xch,
char *buffer,
unsigned int *pnr_chars,
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 071109e..2f6c154 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -24,6 +24,8 @@
#include <xen/cpumask.h>
#include <xen/ctype.h>
#include <asm/setup.h>
+#include <xen/lib.h>
+#include <xen/irq.h>
const void *device_tree_flattened;
dt_irq_xlate_func dt_irq_xlate;
@@ -1642,6 +1644,117 @@ void __init dt_unflatten_host_device_tree(void)
dt_alias_scan();
}
+/* TODO: I think Xen needs a bit of caching in each device node to get the
+ * information in constant time.
+ * For now, Xen has to translate IRQs/MMIOs every time
+ */
+int dt_do_domctl(struct xen_domctl *domctl)
+{
+ struct xen_domctl_dtdev_op *info = &domctl->u.dtdev_op;
+ struct dt_device_node *dev;
+ int ret;
+
+ ret = dt_find_node_by_gpath(info->path, info->plen, &dev);
+ if ( ret )
+ return ret;
+
+ /* Only allow access to protected device and not used by Xen */
+ if ( !dt_device_is_protected(dev) || dt_device_used_by(dev) == DOMID_XEN )
+ return -EACCES;
+
+ switch ( info->op )
+ {
+ case DOMCTL_DTDEV_GET_INFO:
+ {
+ const struct dt_property *compat;
+
+ compat = dt_find_property(dev, "compatible", NULL);
+ /* Hopefully, this case should never happen, print error
+ * if it occurs
+ */
+ if ( !compat )
+ {
+ dprintk(XENLOG_G_ERR, "Unable to find compatible node for %s\n",
+ dt_node_full_name(dev));
+ return -EBADFD;
+ }
+
+ info->u.info.num_irqs = dt_number_of_irq(dev);
+ info->u.info.num_mmios = dt_number_of_address(dev);
+ info->u.info.compat_len = compat->length;
+ }
+ break;
+
+ case DOMCTL_DTDEV_GET_IRQ:
+ {
+ struct dt_irq irq;
+
+ ret = dt_device_get_irq(dev, info->index, &irq);
+ if ( ret )
+ return ret;
+
+ /* Check if Xen is able to assign the IRQ to the guest */
+ if ( !is_assignable_irq(irq.irq) )
+ return -EINVAL;
+
+ info->u.irq.irq = irq.irq;
+ /* TODO: Translate the type into an exportable value */
+ info->u.irq.type = irq.type;
+ }
+ break;
+
+ case DOMCTL_DTDEV_GET_MMIO:
+ {
+ uint64_t addr, size;
+
+ ret = dt_device_get_address(dev, info->index, &addr, &size);
+ if ( ret )
+ return ret;
+
+ /* Make sure the address and the size are page aligned.
+ * If not, we may passthrough MMIO regions which may belong
+ * to another device. Deny it!
+ */
+ if ( (addr & (PAGE_SIZE - 1)) || (size & (PAGE_SIZE - 1)) )
+ {
+ dprintk(XENLOG_ERR, "%s: contain non-page aligned range:"
+ " addr = 0x%"PRIx64" size = 0x%"PRIx64"\n",
+ dt_node_full_name(dev), addr, size);
+ return -EINVAL;
+ }
+
+ info->u.mmio.mfn = paddr_to_pfn(addr);
+ info->u.mmio.nr_mfn = paddr_to_pfn(size);
+ }
+ break;
+
+ case DOMCTL_DTDEV_GET_COMPAT:
+ {
+ const struct dt_property *compat;
+
+ compat = dt_find_property(dev, "compatible", NULL);
+ if ( !compat || !compat->length )
+ return -ENOENT;
+
+ if ( info->u.compat.len < compat->length )
+ return -ENOSPC;
+
+ if ( copy_to_guest(info->u.compat.compat, compat->value,
+ compat->length) != 0 )
+ return -EFAULT;
+
+ info->u.compat.len = compat->length;
+ }
+ break;
+
+ default:
+ return -ENOSYS;
+ }
+
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index 04ecd53..0c9926a 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -315,6 +315,7 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
case XEN_DOMCTL_createdomain:
case XEN_DOMCTL_getdomaininfo:
case XEN_DOMCTL_test_assign_device:
+ case XEN_DOMCTL_dtdev_op:
d = NULL;
break;
default:
@@ -1017,6 +1018,15 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
}
break;
+#ifdef HAS_DEVICE_TREE
+ case XEN_DOMCTL_dtdev_op:
+ {
+ ret = dt_do_domctl(op);
+ copyback = 1;
+ }
+ break;
+#endif
+
default:
ret = arch_do_domctl(op, d, u_domctl);
break;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index b5f2ed7..f7bad9b 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -946,6 +946,44 @@ typedef struct xen_domctl_vcpu_msrs xen_domctl_vcpu_msrs_t;
DEFINE_XEN_GUEST_HANDLE(xen_domctl_vcpu_msrs_t);
#endif
+/* Device Tree: Retrieve informations about a device node */
+struct xen_domctl_dtdev_op {
+ /* IN */
+ uint32_t plen; /* Length of the path */
+ XEN_GUEST_HANDLE(char) path; /* Path to the device tree node */
+#define DOMCTL_DTDEV_GET_INFO 0
+#define DOMCTL_DTDEV_GET_IRQ 1
+#define DOMCTL_DTDEV_GET_MMIO 2
+#define DOMCTL_DTDEV_GET_COMPAT 3
+ uint8_t op;
+ uint32_t pad0:24;
+ uint32_t index; /* Index for the IRQ/MMIO to retrieve */
+ /* OUT */
+ union {
+ struct {
+ uint32_t num_irqs; /* Number of IRQs */
+ uint32_t num_mmios; /* Number of MMIOs */
+ uint32_t compat_len; /* Length of the compatible string */
+ } info;
+ struct {
+ /* TODO: Do we need to handle MSI-X? */
+ uint32_t irq; /* IRQ number */
+ /* TODO: Describe with defines the IRQ type */
+ uint32_t type; /* IRQ type (i.e edge, level...) */
+ } irq;
+ struct {
+ xen_pfn_t mfn;
+ xen_pfn_t nr_mfn;
+ } mmio;
+ struct {
+ uint32_t len; /* IN: Size of buffer. OUT: Size copied */
+ XEN_GUEST_HANDLE_64(char) compat;
+ } compat;
+ } u;
+};
+typedef struct xen_domctl_dtdev_op xen_domctl_dtdev_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_dtdev_op_t);
+
struct xen_domctl {
uint32_t cmd;
#define XEN_DOMCTL_createdomain 1
@@ -1019,6 +1057,7 @@ struct xen_domctl {
#define XEN_DOMCTL_get_vcpu_msrs 72
#define XEN_DOMCTL_set_vcpu_msrs 73
#define XEN_DOMCTL_configure_domain 74
+#define XEN_DOMCTL_dtdev_op 75
#define XEN_DOMCTL_gdbsx_guestmemio 1000
#define XEN_DOMCTL_gdbsx_pausevcpu 1001
#define XEN_DOMCTL_gdbsx_unpausevcpu 1002
@@ -1080,6 +1119,7 @@ struct xen_domctl {
struct xen_domctl_gdbsx_memio gdbsx_guest_memio;
struct xen_domctl_set_broken_page_p2m set_broken_page_p2m;
struct xen_domctl_cacheflush cacheflush;
+ struct xen_domctl_dtdev_op dtdev_op;
struct xen_domctl_gdbsx_pauseunp_vcpu gdbsx_pauseunp_vcpu;
struct xen_domctl_gdbsx_domstatus gdbsx_domstatus;
uint8_t pad[128];
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 6dc34df..8481939 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -12,6 +12,7 @@
#include <asm/byteorder.h>
#include <public/xen.h>
+#include <public/domctl.h>
#include <xen/init.h>
#include <xen/string.h>
#include <xen/types.h>
@@ -669,6 +670,8 @@ int dt_parse_phandle_with_args(const struct dt_device_node *np,
const char *cells_name, int index,
struct dt_phandle_args *out_args);
+int dt_do_domctl(struct xen_domctl *domctl);
+
#endif /* __XEN_DEVICE_TREE_H */
/*
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information
2014-07-31 15:00 ` [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information Julien Grall
@ 2014-08-01 8:50 ` Jan Beulich
2014-08-06 15:17 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Jan Beulich @ 2014-08-01 8:50 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Jackson, tim, stefano.stabellini, ian.campbell, xen-devel
>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -315,6 +315,7 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> case XEN_DOMCTL_createdomain:
> case XEN_DOMCTL_getdomaininfo:
> case XEN_DOMCTL_test_assign_device:
> + case XEN_DOMCTL_dtdev_op:
> d = NULL;
> break;
> default:
> @@ -1017,6 +1018,15 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> }
> break;
>
> +#ifdef HAS_DEVICE_TREE
> + case XEN_DOMCTL_dtdev_op:
> + {
> + ret = dt_do_domctl(op);
> + copyback = 1;
> + }
> + break;
> +#endif
No pointless braces please.
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -946,6 +946,44 @@ typedef struct xen_domctl_vcpu_msrs xen_domctl_vcpu_msrs_t;
> DEFINE_XEN_GUEST_HANDLE(xen_domctl_vcpu_msrs_t);
> #endif
>
> +/* Device Tree: Retrieve informations about a device node */
> +struct xen_domctl_dtdev_op {
> + /* IN */
> + uint32_t plen; /* Length of the path */
> + XEN_GUEST_HANDLE(char) path; /* Path to the device tree node */
XEN_GUEST_HANDLE_64? And padding between the two above
members, or fields re-ordered?
> +#define DOMCTL_DTDEV_GET_INFO 0
> +#define DOMCTL_DTDEV_GET_IRQ 1
> +#define DOMCTL_DTDEV_GET_MMIO 2
> +#define DOMCTL_DTDEV_GET_COMPAT 3
> + uint8_t op;
> + uint32_t pad0:24;
uint8_t pad0[3].
> + uint32_t index; /* Index for the IRQ/MMIO to retrieve */
> + /* OUT */
> + union {
> + struct {
> + uint32_t num_irqs; /* Number of IRQs */
> + uint32_t num_mmios; /* Number of MMIOs */
> + uint32_t compat_len; /* Length of the compatible string */
> + } info;
> + struct {
> + /* TODO: Do we need to handle MSI-X? */
> + uint32_t irq; /* IRQ number */
> + /* TODO: Describe with defines the IRQ type */
???
Also, are you planning to address these two TODOs before this gets
ready to be committed?
> + uint32_t type; /* IRQ type (i.e edge, level...) */
#define-s to specify what values here mean?
> + } irq;
> + struct {
> + xen_pfn_t mfn;
> + xen_pfn_t nr_mfn;
> + } mmio;
> + struct {
> + uint32_t len; /* IN: Size of buffer. OUT: Size copied */
> + XEN_GUEST_HANDLE_64(char) compat;
Padding again?
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information
2014-08-01 8:50 ` Jan Beulich
@ 2014-08-06 15:17 ` Julien Grall
2014-08-06 15:47 ` Jan Beulich
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 15:17 UTC (permalink / raw)
To: Jan Beulich; +Cc: Ian Jackson, tim, stefano.stabellini, ian.campbell, xen-devel
Hi Jan,
On 08/01/2014 09:50 AM, Jan Beulich wrote:
>>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
>> --- a/xen/common/domctl.c
>> +++ b/xen/common/domctl.c
>> @@ -315,6 +315,7 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>> case XEN_DOMCTL_createdomain:
>> case XEN_DOMCTL_getdomaininfo:
>> case XEN_DOMCTL_test_assign_device:
>> + case XEN_DOMCTL_dtdev_op:
>> d = NULL;
>> break;
>> default:
>> @@ -1017,6 +1018,15 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>> }
>> break;
>>
>> +#ifdef HAS_DEVICE_TREE
>> + case XEN_DOMCTL_dtdev_op:
>> + {
>> + ret = dt_do_domctl(op);
>> + copyback = 1;
>> + }
>> + break;
>> +#endif
>
> No pointless braces please.
Will drop it.
>> --- a/xen/include/public/domctl.h
>> +++ b/xen/include/public/domctl.h
>> @@ -946,6 +946,44 @@ typedef struct xen_domctl_vcpu_msrs xen_domctl_vcpu_msrs_t;
>> DEFINE_XEN_GUEST_HANDLE(xen_domctl_vcpu_msrs_t);
>> #endif
>>
>> +/* Device Tree: Retrieve informations about a device node */
>> +struct xen_domctl_dtdev_op {
>> + /* IN */
>> + uint32_t plen; /* Length of the path */
>> + XEN_GUEST_HANDLE(char) path; /* Path to the device tree node */
>
> XEN_GUEST_HANDLE_64?
Right. I never know when I should use XEN_GUEST_HANDLE_64 or
XEN_GUEST_HANDLE.
FYI, it's exactly the same on ARM.
> And padding between the two above
> members, or fields re-ordered?
I can reorder the field.
>> +#define DOMCTL_DTDEV_GET_INFO 0
>> +#define DOMCTL_DTDEV_GET_IRQ 1
>> +#define DOMCTL_DTDEV_GET_MMIO 2
>> +#define DOMCTL_DTDEV_GET_COMPAT 3
>> + uint8_t op;
>> + uint32_t pad0:24;
>
> uint8_t pad0[3].
Ok.
>> + uint32_t index; /* Index for the IRQ/MMIO to retrieve */
>> + /* OUT */
>> + union {
>> + struct {
>> + uint32_t num_irqs; /* Number of IRQs */
>> + uint32_t num_mmios; /* Number of MMIOs */
>> + uint32_t compat_len; /* Length of the compatible string */
>> + } info;
>> + struct {
>> + /* TODO: Do we need to handle MSI-X? */
>> + uint32_t irq; /* IRQ number */
>> + /* TODO: Describe with defines the IRQ type */
>
> ???
For now we are using the DT_IRQ_TYPE_* provided by the device tree (see
include/xen/device_tree.h). I haven't yet introduced specific IRQ type here.
> Also, are you planning to address these two TODOs before this gets
> ready to be committed?
I plan to handle the latter TODO (i.e IRQ type), for the MSI I don't
know how they will be described in the device tree. So I will drop the
TODO. It will be fine because it's a DOMCTL so it's possible to change
the interface easily.
>> + uint32_t type; /* IRQ type (i.e edge, level...) */
>
> #define-s to specify what values here mean?
See my answer a bit above.
>> + } irq;
>> + struct {
>> + xen_pfn_t mfn;
>> + xen_pfn_t nr_mfn;
>> + } mmio;
>> + struct {
>> + uint32_t len; /* IN: Size of buffer. OUT: Size copied */
>> + XEN_GUEST_HANDLE_64(char) compat;
>
> Padding again?
I will invert the 2 fields.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information
2014-08-06 15:17 ` Julien Grall
@ 2014-08-06 15:47 ` Jan Beulich
0 siblings, 0 replies; 119+ messages in thread
From: Jan Beulich @ 2014-08-06 15:47 UTC (permalink / raw)
To: Julien Grall
Cc: Ian Jackson, tim, stefano.stabellini, ian.campbell, xen-devel
>>> On 06.08.14 at 17:17, <julien.grall@linaro.org> wrote:
> On 08/01/2014 09:50 AM, Jan Beulich wrote:
>>>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
>>> --- a/xen/include/public/domctl.h
>>> +++ b/xen/include/public/domctl.h
>>> @@ -946,6 +946,44 @@ typedef struct xen_domctl_vcpu_msrs xen_domctl_vcpu_msrs_t;
>>> DEFINE_XEN_GUEST_HANDLE(xen_domctl_vcpu_msrs_t);
>>> #endif
>>>
>>> +/* Device Tree: Retrieve informations about a device node */
>>> +struct xen_domctl_dtdev_op {
>>> + /* IN */
>>> + uint32_t plen; /* Length of the path */
>>> + XEN_GUEST_HANDLE(char) path; /* Path to the device tree node */
>>
>> XEN_GUEST_HANDLE_64?
>
> Right. I never know when I should use XEN_GUEST_HANDLE_64 or
> XEN_GUEST_HANDLE.
The former is for interfaces without compat mode translation, i.e.
mainly domctl and sysctl (other toolstack only ones may use them
too).
> FYI, it's exactly the same on ARM.
>
>> And padding between the two above
>> members, or fields re-ordered?
>
> I can reorder the field.
> [...]
>>> + } irq;
>>> + struct {
>>> + xen_pfn_t mfn;
>>> + xen_pfn_t nr_mfn;
>>> + } mmio;
>>> + struct {
>>> + uint32_t len; /* IN: Size of buffer. OUT: Size copied
> */
>>> + XEN_GUEST_HANDLE_64(char) compat;
>>
>> Padding again?
>
> I will invert the 2 fields.
The re-ordering won't help unless there's another field following which
fills the then trailing padding space.
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 14/21] xen/passthrough: Introduce iommu_construct
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (12 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 13/21] xen/dts: Add hypercalls to retrieve device node information Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-01 8:55 ` Jan Beulich
2014-07-31 15:00 ` [PATCH v2 15/21] xen/passthrough: Call arch_iommu_domain_destroy before calling iommu_teardown Julien Grall
` (7 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, Julien Grall, tim, ian.campbell, Jan Beulich
This new function will correctly initialize the IOMMU page table for the
current domain.
Also use it in iommu_assign_dt_device even though the current IOMMU
implementation on ARM shares P2M with the processor.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Jan Beulich <jbeulich@suse.com>
---
Changes in v2:
- Add missing Signed-off-by
- Rename iommu_buildup to iommu_construct
---
xen/drivers/passthrough/arm/iommu.c | 6 ++++++
xen/drivers/passthrough/device_tree.c | 7 +++++++
xen/drivers/passthrough/iommu.c | 25 +++++++++++++++++++++++++
xen/drivers/passthrough/pci.c | 12 ++++--------
xen/include/xen/iommu.h | 2 ++
5 files changed, 44 insertions(+), 8 deletions(-)
diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c
index 3007b99..9234657 100644
--- a/xen/drivers/passthrough/arm/iommu.c
+++ b/xen/drivers/passthrough/arm/iommu.c
@@ -68,3 +68,9 @@ void arch_iommu_domain_destroy(struct domain *d)
{
iommu_dt_domain_destroy(d);
}
+
+int arch_iommu_populate_page_table(struct domain *d)
+{
+ /* The IOMMU shares the p2m with the CPU */
+ return -ENOSYS;
+}
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 3e47df5..45d4a59 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -41,6 +41,13 @@ int iommu_assign_dt_device(struct domain *d, struct dt_device_node *dev)
if ( !list_empty(&dev->domain_list) )
goto fail;
+ if ( need_iommu(d) <= 0 )
+ {
+ rc = iommu_construct(d);
+ if ( rc )
+ goto fail;
+ }
+
rc = hd->platform_ops->assign_dt_device(d, dev);
if ( rc )
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index cc12735..65479d6 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -187,6 +187,31 @@ void iommu_teardown(struct domain *d)
tasklet_schedule(&iommu_pt_cleanup_tasklet);
}
+int iommu_construct(struct domain *d)
+{
+ int rc = 0;
+
+ /*
+ * The caller should check we effectively need to set up the IOMMMU
+ * for this domain.
+ */
+ ASSERT(need_iommu(d) <= 0);
+
+ if ( need_iommu(d) > 0 )
+ return 0;
+
+ if ( !iommu_use_hap_pt(d) )
+ {
+ rc = arch_iommu_populate_page_table(d);
+ if ( rc )
+ return rc;
+ }
+
+ d->need_iommu = 1;
+
+ return rc;
+}
+
void iommu_domain_destroy(struct domain *d)
{
struct hvm_iommu *hd = domain_hvm_iommu(d);
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 1eba833..e711a2e 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1342,14 +1342,10 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn)
if ( need_iommu(d) <= 0 )
{
- if ( !iommu_use_hap_pt(d) )
- {
- rc = arch_iommu_populate_page_table(d);
- if ( rc )
- {
- spin_unlock(&pcidevs_lock);
- return rc;
- }
+ rc = iommu_construct(d);
+ if ( rc ) {
+ spin_unlock(&pcidevs_lock);
+ return rc;
}
d->need_iommu = 1;
}
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8eb764a..8dc7bd2 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -64,6 +64,8 @@ int arch_iommu_domain_init(struct domain *d);
int arch_iommu_populate_page_table(struct domain *d);
void arch_iommu_check_autotranslated_hwdom(struct domain *d);
+int iommu_construct(struct domain *d);
+
/* Function used internally, use iommu_domain_destroy */
void iommu_teardown(struct domain *d);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 14/21] xen/passthrough: Introduce iommu_construct
2014-07-31 15:00 ` [PATCH v2 14/21] xen/passthrough: Introduce iommu_construct Julien Grall
@ 2014-08-01 8:55 ` Jan Beulich
0 siblings, 0 replies; 119+ messages in thread
From: Jan Beulich @ 2014-08-01 8:55 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, stefano.stabellini, ian.campbell, tim
>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> +int iommu_construct(struct domain *d)
> +{
> + int rc = 0;
> +
> + /*
> + * The caller should check we effectively need to set up the IOMMMU
> + * for this domain.
> + */
> + ASSERT(need_iommu(d) <= 0);
> +
> + if ( need_iommu(d) > 0 )
> + return 0;
Either ASSERT() or if() (and in the latter case there's be no need for
the callers to check unless they need the check for other purposes).
> +
> + if ( !iommu_use_hap_pt(d) )
> + {
> + rc = arch_iommu_populate_page_table(d);
> + if ( rc )
> + return rc;
> + }
> +
> + d->need_iommu = 1;
This wasn't removed from the caller - an oversight?
> --- a/xen/drivers/passthrough/pci.c
> +++ b/xen/drivers/passthrough/pci.c
> @@ -1342,14 +1342,10 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn)
>
> if ( need_iommu(d) <= 0 )
> {
> - if ( !iommu_use_hap_pt(d) )
> - {
> - rc = arch_iommu_populate_page_table(d);
> - if ( rc )
> - {
> - spin_unlock(&pcidevs_lock);
> - return rc;
> - }
> + rc = iommu_construct(d);
> + if ( rc ) {
Coding style.
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 15/21] xen/passthrough: Call arch_iommu_domain_destroy before calling iommu_teardown
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (13 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 14/21] xen/passthrough: Introduce iommu_construct Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-01 9:00 ` Jan Beulich
2014-07-31 15:00 ` [PATCH v2 16/21] xen/passthrough: iommu_deassign_device_dt: By default reassign device to nobody Julien Grall
` (6 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, Julien Grall, tim, ian.campbell, Jan Beulich
arch_iommu_domain_destroy contains specific architecture code.
On x86, this code will clean up the ioport_list which is not used in
both iommu (i.e AMD & x86) drivers.
On ARM, the toolstack may not have deassign every device to the guest.
Therefore, we have to go through the device list and removing them before
asking the IOMMU drivers to release memory for this domain. This is done
by iommu_dt_domain_destroy which is called by arch_iommu_domain_destroy.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Jan Beulich <jbeulich@suse.com>
---
I didn't change this patch as I wasn't sure what was the conclusion of
the discussion on the RFC
---
xen/drivers/passthrough/iommu.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 65479d6..2953ede 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -219,10 +219,10 @@ void iommu_domain_destroy(struct domain *d)
if ( !iommu_enabled || !hd->platform_ops )
return;
+ arch_iommu_domain_destroy(d);
+
if ( need_iommu(d) )
iommu_teardown(d);
-
- arch_iommu_domain_destroy(d);
}
int iommu_map_page(struct domain *d, unsigned long gfn, unsigned long mfn,
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 15/21] xen/passthrough: Call arch_iommu_domain_destroy before calling iommu_teardown
2014-07-31 15:00 ` [PATCH v2 15/21] xen/passthrough: Call arch_iommu_domain_destroy before calling iommu_teardown Julien Grall
@ 2014-08-01 9:00 ` Jan Beulich
0 siblings, 0 replies; 119+ messages in thread
From: Jan Beulich @ 2014-08-01 9:00 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, stefano.stabellini, ian.campbell, tim
>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> arch_iommu_domain_destroy contains specific architecture code.
>
> On x86, this code will clean up the ioport_list which is not used in
> both iommu (i.e AMD & x86) drivers.
>
> On ARM, the toolstack may not have deassign every device to the guest.
> Therefore, we have to go through the device list and removing them before
> asking the IOMMU drivers to release memory for this domain. This is done
> by iommu_dt_domain_destroy which is called by arch_iommu_domain_destroy.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Jan Beulich <jbeulich@suse.com>
>
> ---
> I didn't change this patch as I wasn't sure what was the conclusion of
> the discussion on the RFC
My perspective hasn't changed. And rather than leaving hunting down
that previous discussion in the list archives to every one of those who
read this, you could have included a reference here.
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 16/21] xen/passthrough: iommu_deassign_device_dt: By default reassign device to nobody
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (14 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 15/21] xen/passthrough: Call arch_iommu_domain_destroy before calling iommu_teardown Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 16:23 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 17/21] xen/iommu: arm: Wire iommu DOMCTL for ARM Julien Grall
` (5 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
Currently, when the device is deassigned from a domain, we directly reassign
to DOM0.
As the device may not have been correctly reset, this may lead to corruption or
expose some part of DOM0 memory. Also, we may have no way to reset some
platform devices.
If Xen reassigns the device to "nobody", it may receive some global/context
fault because the transaction has failed (indeed the context has been
marked invalid). Unfortunately there is no simple way to quiesce a buggy
hardware. I think we could live with that for a first version of platform
device passthrough.
DOM0 will have to issue an hypercall to assign the device to itself if it
wants to use it.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
Changes in v2:
- Fix typoes in the commit message
- Update commit message
---
xen/drivers/passthrough/arm/smmu.c | 7 ++++---
xen/drivers/passthrough/device_tree.c | 8 +++-----
2 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index f4eb2a2..b25034e 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -1245,8 +1245,8 @@ static int arm_smmu_reassign_dt_dev(struct domain *s, struct domain *t,
{
int ret = 0;
- /* Don't allow remapping on other domain than hwdom */
- if ( t != hardware_domain )
+ /* Allow remapping either on the hardware domain or to nothing */
+ if ( t && t != hardware_domain )
return -EPERM;
if ( t == s )
@@ -1256,7 +1256,8 @@ static int arm_smmu_reassign_dt_dev(struct domain *s, struct domain *t,
if ( ret )
return ret;
- ret = arm_smmu_attach_dev(t, dev);
+ if ( t )
+ ret = arm_smmu_attach_dev(t, dev);
return ret;
}
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index 45d4a59..f8affa0 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -75,14 +75,12 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev)
spin_lock(&dtdevs_lock);
- rc = hd->platform_ops->reassign_dt_device(d, hardware_domain, dev);
+ rc = hd->platform_ops->reassign_dt_device(d, NULL, dev);
if ( rc )
goto fail;
- list_del(&dev->domain_list);
-
- dt_device_set_used_by(dev, hardware_domain->domain_id);
- list_add(&dev->domain_list, &domain_hvm_iommu(hardware_domain)->dt_devices);
+ list_del_init(&dev->domain_list);
+ dt_device_set_used_by(dev, DOMID_IO);
fail:
spin_unlock(&dtdevs_lock);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 16/21] xen/passthrough: iommu_deassign_device_dt: By default reassign device to nobody
2014-07-31 15:00 ` [PATCH v2 16/21] xen/passthrough: iommu_deassign_device_dt: By default reassign device to nobody Julien Grall
@ 2014-08-06 16:23 ` Stefano Stabellini
2015-01-12 16:33 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:23 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> Currently, when the device is deassigned from a domain, we directly reassign
> to DOM0.
>
> As the device may not have been correctly reset, this may lead to corruption or
> expose some part of DOM0 memory. Also, we may have no way to reset some
> platform devices.
>
> If Xen reassigns the device to "nobody", it may receive some global/context
> fault because the transaction has failed (indeed the context has been
> marked invalid). Unfortunately there is no simple way to quiesce a buggy
> hardware. I think we could live with that for a first version of platform
> device passthrough.
>
> DOM0 will have to issue an hypercall to assign the device to itself if it
> wants to use it.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> ---
> Changes in v2:
> - Fix typoes in the commit message
> - Update commit message
> ---
> xen/drivers/passthrough/arm/smmu.c | 7 ++++---
> xen/drivers/passthrough/device_tree.c | 8 +++-----
> 2 files changed, 7 insertions(+), 8 deletions(-)
>
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index f4eb2a2..b25034e 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -1245,8 +1245,8 @@ static int arm_smmu_reassign_dt_dev(struct domain *s, struct domain *t,
> {
> int ret = 0;
>
> - /* Don't allow remapping on other domain than hwdom */
> - if ( t != hardware_domain )
> + /* Allow remapping either on the hardware domain or to nothing */
> + if ( t && t != hardware_domain )
> return -EPERM;
>
> if ( t == s )
> @@ -1256,7 +1256,8 @@ static int arm_smmu_reassign_dt_dev(struct domain *s, struct domain *t,
> if ( ret )
> return ret;
>
> - ret = arm_smmu_attach_dev(t, dev);
> + if ( t )
> + ret = arm_smmu_attach_dev(t, dev);
>
> return ret;
> }
> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
> index 45d4a59..f8affa0 100644
> --- a/xen/drivers/passthrough/device_tree.c
> +++ b/xen/drivers/passthrough/device_tree.c
> @@ -75,14 +75,12 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev)
>
> spin_lock(&dtdevs_lock);
>
> - rc = hd->platform_ops->reassign_dt_device(d, hardware_domain, dev);
> + rc = hd->platform_ops->reassign_dt_device(d, NULL, dev);
> if ( rc )
> goto fail;
At this point, isn't it better just to call arm_smmu_detach_dev?
> - list_del(&dev->domain_list);
> -
> - dt_device_set_used_by(dev, hardware_domain->domain_id);
> - list_add(&dev->domain_list, &domain_hvm_iommu(hardware_domain)->dt_devices);
> + list_del_init(&dev->domain_list);
> + dt_device_set_used_by(dev, DOMID_IO);
>
> fail:
> spin_unlock(&dtdevs_lock);
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 16/21] xen/passthrough: iommu_deassign_device_dt: By default reassign device to nobody
2014-08-06 16:23 ` Stefano Stabellini
@ 2015-01-12 16:33 ` Julien Grall
0 siblings, 0 replies; 119+ messages in thread
From: Julien Grall @ 2015-01-12 16:33 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
Sorry for the very late answer on this patch.
On 06/08/14 17:23, Stefano Stabellini wrote:
>> diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
>> index 45d4a59..f8affa0 100644
>> --- a/xen/drivers/passthrough/device_tree.c
>> +++ b/xen/drivers/passthrough/device_tree.c
>> @@ -75,14 +75,12 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev)
>>
>> spin_lock(&dtdevs_lock);
>>
>> - rc = hd->platform_ops->reassign_dt_device(d, hardware_domain, dev);
>> + rc = hd->platform_ops->reassign_dt_device(d, NULL, dev);
>> if ( rc )
>> goto fail;
>
> At this point, isn't it better just to call arm_smmu_detach_dev?
That would mean introducing an new callback in structure. I would prefer
modify reassign_dt_device rather than adding a new one.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 17/21] xen/iommu: arm: Wire iommu DOMCTL for ARM
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (15 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 16/21] xen/passthrough: iommu_deassign_device_dt: By default reassign device to nobody Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 16:24 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 18/21] xen/passthrough: dt: Add new domctl XEN_DOMCTL_assign_dt_device Julien Grall
` (4 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: stefano.stabellini, Julien Grall, tim, ian.campbell, Jan Beulich
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Jan Beulich <jbeulich@suse.com>
---
Changes in v2:
- Don't move the call in common code.
---
xen/arch/arm/domctl.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
index bab92b2..370dd99 100644
--- a/xen/arch/arm/domctl.c
+++ b/xen/arch/arm/domctl.c
@@ -43,7 +43,16 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
}
default:
- return subarch_do_domctl(domctl, d, u_domctl);
+ {
+ int rc;
+
+ rc = subarch_do_domctl(domctl, d, u_domctl);
+
+ if ( rc == -ENOSYS )
+ rc = iommu_do_domctl(domctl, d, u_domctl);
+
+ return rc;
+ }
}
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 17/21] xen/iommu: arm: Wire iommu DOMCTL for ARM
2014-07-31 15:00 ` [PATCH v2 17/21] xen/iommu: arm: Wire iommu DOMCTL for ARM Julien Grall
@ 2014-08-06 16:24 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:24 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Jan Beulich, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Jan Beulich <jbeulich@suse.com>
>
> ---
> Changes in v2:
> - Don't move the call in common code.
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> xen/arch/arm/domctl.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c
> index bab92b2..370dd99 100644
> --- a/xen/arch/arm/domctl.c
> +++ b/xen/arch/arm/domctl.c
> @@ -43,7 +43,16 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
> }
>
> default:
> - return subarch_do_domctl(domctl, d, u_domctl);
> + {
> + int rc;
> +
> + rc = subarch_do_domctl(domctl, d, u_domctl);
> +
> + if ( rc == -ENOSYS )
> + rc = iommu_do_domctl(domctl, d, u_domctl);
> +
> + return rc;
> + }
> }
> }
>
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 18/21] xen/passthrough: dt: Add new domctl XEN_DOMCTL_assign_dt_device
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (16 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 17/21] xen/iommu: arm: Wire iommu DOMCTL for ARM Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-01 9:05 ` Jan Beulich
2014-07-31 15:00 ` [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough Julien Grall
` (3 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, tim, Julien Grall, Ian Jackson,
stefano.stabellini
A device node is described by a path. It will be used to retrieved the
node in the device tree and assign the related device to the domain.
Only device protected by an IOMMU can be assigned to a guest.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v2:
- Use a different number for XEN_DOMCTL_assign_dt_device
---
tools/libxc/xc_domain.c | 29 +++++++++++++++++++++
tools/libxc/xenctrl.h | 4 +++
xen/drivers/passthrough/device_tree.c | 45 ++++++++++++++++++++++++++++++---
xen/drivers/passthrough/iommu.c | 7 +++++
xen/include/public/domctl.h | 10 ++++++++
xen/include/xen/iommu.h | 3 +++
6 files changed, 95 insertions(+), 3 deletions(-)
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index 4cc0474..08b6f02 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -1677,6 +1677,35 @@ int xc_deassign_device(
return do_domctl(xch, &domctl);
}
+int xc_assign_dt_device(
+ xc_interface *xch,
+ uint32_t domid,
+ char *path)
+{
+ int rc;
+ size_t size = strlen(path);
+ xen_domctl_assign_dt_device_t *assign_dt_device;
+ DECLARE_DOMCTL;
+ DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN);
+
+ if ( xc_hypercall_bounce_pre(xch, path) )
+ return -1;
+
+ domctl.cmd = XEN_DOMCTL_assign_dt_device;
+ domctl.domain = (domid_t)domid;
+
+ assign_dt_device = &domctl.u.assign_dt_device;
+
+ assign_dt_device->size = size;
+ set_xen_guest_handle(assign_dt_device->path, path);
+
+ rc = do_domctl(xch, &domctl);
+
+ xc_hypercall_bounce_post(xch, path);
+
+ return rc;
+}
+
int xc_domain_update_msi_irq(
xc_interface *xch,
uint32_t domid,
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index c549753..bd24e1d 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -2071,6 +2071,10 @@ int xc_deassign_device(xc_interface *xch,
uint32_t domid,
uint32_t machine_bdf);
+int xc_assign_dt_device(xc_interface *xch,
+ uint32_t domid,
+ char *path);
+
int xc_domain_memory_mapping(xc_interface *xch,
uint32_t domid,
unsigned long first_gfn,
diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
index f8affa0..232159a 100644
--- a/xen/drivers/passthrough/device_tree.c
+++ b/xen/drivers/passthrough/device_tree.c
@@ -1,9 +1,6 @@
/*
* Code to passthrough a device tree node to a guest
*
- * TODO: This contains only the necessary code to protected device passed to
- * dom0. It will need some updates when device passthrough will is added.
- *
* Julien Grall <julien.grall@linaro.org>
* Copyright (c) 2014 Linaro Limited.
*
@@ -20,6 +17,7 @@
#include <xen/lib.h>
#include <xen/sched.h>
+#include <xen/guest_access.h>
#include <xen/iommu.h>
#include <xen/device_tree.h>
@@ -111,3 +109,44 @@ void iommu_dt_domain_destroy(struct domain *d)
dt_node_full_name(dev), d->domain_id);
}
}
+
+int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d,
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
+{
+ int ret;
+
+ /* TODO: How to deal with XSM? */
+
+ switch ( domctl->cmd )
+ {
+ case XEN_DOMCTL_assign_dt_device:
+ {
+ struct dt_device_node *dev;
+
+
+ /* TODO: Do we need to check is_dying? Mostly to protect against
+ * hypercall trying to passthrough a device while we are
+ * dying.
+ */
+
+ ret = dt_find_node_by_gpath(domctl->u.assign_dt_device.path,
+ domctl->u.assign_dt_device.size,
+ &dev);
+ if ( ret )
+ break;
+
+ ret = iommu_assign_dt_device(d, dev);
+
+ if ( ret )
+ printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\""
+ " to dom%u failed (%d)\n",
+ dt_node_full_name(dev), d->domain_id, ret);
+ }
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ return ret;
+}
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 2953ede..5507c93 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -343,6 +343,13 @@ int iommu_do_domctl(
ret = iommu_do_pci_domctl(domctl, d, u_domctl);
#endif
+ if ( ret != -ENOSYS )
+ return ret;
+
+#ifdef HAS_DEVICE_TREE
+ ret = iommu_do_dt_domctl(domctl, d, u_domctl);
+#endif
+
return ret;
}
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index f7bad9b..8c4d4c5 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -984,6 +984,14 @@ struct xen_domctl_dtdev_op {
typedef struct xen_domctl_dtdev_op xen_domctl_dtdev_op_t;
DEFINE_XEN_GUEST_HANDLE(xen_domctl_dtdev_op_t);
+/* Device Tree: Assign a non-PCI device to a guest */
+struct xen_domctl_assign_dt_device {
+ uint32_t size; /* IN: Length of the path */
+ XEN_GUEST_HANDLE_64(char) path; /* IN: path to the device tree node */
+};
+typedef struct xen_domctl_assign_dt_device xen_domctl_assign_dt_device_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_assign_dt_device_t);
+
struct xen_domctl {
uint32_t cmd;
#define XEN_DOMCTL_createdomain 1
@@ -1058,6 +1066,7 @@ struct xen_domctl {
#define XEN_DOMCTL_set_vcpu_msrs 73
#define XEN_DOMCTL_configure_domain 74
#define XEN_DOMCTL_dtdev_op 75
+#define XEN_DOMCTL_assign_dt_device 76
#define XEN_DOMCTL_gdbsx_guestmemio 1000
#define XEN_DOMCTL_gdbsx_pausevcpu 1001
#define XEN_DOMCTL_gdbsx_unpausevcpu 1002
@@ -1097,6 +1106,7 @@ struct xen_domctl {
struct xen_domctl_sendtrigger sendtrigger;
struct xen_domctl_get_device_group get_device_group;
struct xen_domctl_assign_device assign_device;
+ struct xen_domctl_assign_dt_device assign_dt_device;
struct xen_domctl_bind_pt_irq bind_pt_irq;
struct xen_domctl_memory_mapping memory_mapping;
struct xen_domctl_ioport_mapping ioport_mapping;
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8dc7bd2..6dd2670 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -118,6 +118,9 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev);
int iommu_dt_domain_init(struct domain *d);
void iommu_dt_domain_destroy(struct domain *d);
+int iommu_do_dt_domctl(struct xen_domctl *, struct domain *,
+ XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
+
#endif /* HAS_DEVICE_TREE */
struct page_info;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 18/21] xen/passthrough: dt: Add new domctl XEN_DOMCTL_assign_dt_device
2014-07-31 15:00 ` [PATCH v2 18/21] xen/passthrough: dt: Add new domctl XEN_DOMCTL_assign_dt_device Julien Grall
@ 2014-08-01 9:05 ` Jan Beulich
0 siblings, 0 replies; 119+ messages in thread
From: Jan Beulich @ 2014-08-01 9:05 UTC (permalink / raw)
To: Julien Grall
Cc: ian.campbell, Stefano Stabellini, tim, Ian Jackson,
stefano.stabellini, xen-devel
>>> On 31.07.14 at 17:00, <julien.grall@linaro.org> wrote:
> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -343,6 +343,13 @@ int iommu_do_domctl(
> ret = iommu_do_pci_domctl(domctl, d, u_domctl);
> #endif
>
> + if ( ret != -ENOSYS )
> + return ret;
> +
> +#ifdef HAS_DEVICE_TREE
> + ret = iommu_do_dt_domctl(domctl, d, u_domctl);
> +#endif
I think the if() (with the condition inverted) should be moved inside
the #ifdef.
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -984,6 +984,14 @@ struct xen_domctl_dtdev_op {
> typedef struct xen_domctl_dtdev_op xen_domctl_dtdev_op_t;
> DEFINE_XEN_GUEST_HANDLE(xen_domctl_dtdev_op_t);
>
> +/* Device Tree: Assign a non-PCI device to a guest */
> +struct xen_domctl_assign_dt_device {
> + uint32_t size; /* IN: Length of the path */
> + XEN_GUEST_HANDLE_64(char) path; /* IN: path to the device tree node */
Missing padding again?
Also, no XSM adjustment needed this time?
Jan
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (17 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 18/21] xen/passthrough: dt: Add new domctl XEN_DOMCTL_assign_dt_device Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 16:27 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 20/21] libxl: Add support for non-PCI passthrough Julien Grall
` (2 subsequent siblings)
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel; +Cc: stefano.stabellini, Julien Grall, tim, ian.campbell
This region will be split by the toolstack to allocate MMIO range for eac
device.
For now only reserve a 768MB region, this should be enought to passthrough
multiple device at the same time.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
I can't allocate easily a range of 1GB in the current layout without shrink
the first RAM bank. So I decided to use only 768MB. If in the future we need
more, then we will have to choose at runtime the guest layout.
Changes in v2:
- Bump the reserved region from 512MB to 768MB
---
xen/include/public/arch-arm.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index ac54cd6..cebb349 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -369,6 +369,10 @@ typedef uint64_t xen_callback_t;
#define GUEST_GICC_BASE 0x03002000ULL
#define GUEST_GICC_SIZE 0x00000100ULL
+/* Space for mapping MMIO from device passthrough: 768MB @ 256MB*/
+#define GUEST_MMIO_BASE 0x10000000ULL
+#define GUEST_MMIO_SIZE 0x30000000ULL
+
/* 16MB == 4096 pages reserved for guest to use as a region to map its
* grant table in.
*/
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-07-31 15:00 ` [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough Julien Grall
@ 2014-08-06 16:27 ` Stefano Stabellini
2014-08-06 16:33 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:27 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On Thu, 31 Jul 2014, Julien Grall wrote:
> This region will be split by the toolstack to allocate MMIO range for eac
> device.
>
> For now only reserve a 768MB region, this should be enought to passthrough
> multiple device at the same time.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>
> ---
>
> I can't allocate easily a range of 1GB in the current layout without shrink
> the first RAM bank. So I decided to use only 768MB. If in the future we need
> more, then we will have to choose at runtime the guest layout.
>
> Changes in v2:
> - Bump the reserved region from 512MB to 768MB
Better than nothing.
Would it be possible to add a second region > 4GB? Maybe after RAM?
How hard would be to handle that?
> xen/include/public/arch-arm.h | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> index ac54cd6..cebb349 100644
> --- a/xen/include/public/arch-arm.h
> +++ b/xen/include/public/arch-arm.h
> @@ -369,6 +369,10 @@ typedef uint64_t xen_callback_t;
> #define GUEST_GICC_BASE 0x03002000ULL
> #define GUEST_GICC_SIZE 0x00000100ULL
>
> +/* Space for mapping MMIO from device passthrough: 768MB @ 256MB*/
> +#define GUEST_MMIO_BASE 0x10000000ULL
> +#define GUEST_MMIO_SIZE 0x30000000ULL
> +
> /* 16MB == 4096 pages reserved for guest to use as a region to map its
> * grant table in.
> */
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-08-06 16:27 ` Stefano Stabellini
@ 2014-08-06 16:33 ` Julien Grall
2014-08-06 16:44 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 16:33 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
On 08/06/2014 05:27 PM, Stefano Stabellini wrote:
> On Thu, 31 Jul 2014, Julien Grall wrote:
>> This region will be split by the toolstack to allocate MMIO range for eac
>> device.
>>
>> For now only reserve a 768MB region, this should be enought to passthrough
>> multiple device at the same time.
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>
>> ---
>>
>> I can't allocate easily a range of 1GB in the current layout without shrink
>> the first RAM bank. So I decided to use only 768MB. If in the future we need
>> more, then we will have to choose at runtime the guest layout.
>>
>> Changes in v2:
>> - Bump the reserved region from 512MB to 768MB
>
>
> Better than nothing.
> Would it be possible to add a second region > 4GB? Maybe after RAM?
> How hard would be to handle that?
It won't work if the guest is using short-descriptor translation table
format because only 32 bits address will be supported.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-08-06 16:33 ` Julien Grall
@ 2014-08-06 16:44 ` Stefano Stabellini
2014-08-06 16:45 ` Stefano Stabellini
2014-08-06 16:47 ` Julien Grall
0 siblings, 2 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:44 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Wed, 6 Aug 2014, Julien Grall wrote:
> Hi Stefano,
>
> On 08/06/2014 05:27 PM, Stefano Stabellini wrote:
> > On Thu, 31 Jul 2014, Julien Grall wrote:
> >> This region will be split by the toolstack to allocate MMIO range for eac
> >> device.
> >>
> >> For now only reserve a 768MB region, this should be enought to passthrough
> >> multiple device at the same time.
> >>
> >> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>
> >> ---
> >>
> >> I can't allocate easily a range of 1GB in the current layout without shrink
> >> the first RAM bank. So I decided to use only 768MB. If in the future we need
> >> more, then we will have to choose at runtime the guest layout.
> >>
> >> Changes in v2:
> >> - Bump the reserved region from 512MB to 768MB
> >
> >
> > Better than nothing.
> > Would it be possible to add a second region > 4GB? Maybe after RAM?
> > How hard would be to handle that?
>
> It won't work if the guest is using short-descriptor translation table
> format because only 32 bits address will be supported.
I know.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-08-06 16:44 ` Stefano Stabellini
@ 2014-08-06 16:45 ` Stefano Stabellini
2014-08-06 16:55 ` Julien Grall
2014-08-06 16:47 ` Julien Grall
1 sibling, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:45 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel, Julien Grall, tim, ian.campbell, stefano.stabellini
On Wed, 6 Aug 2014, Stefano Stabellini wrote:
> On Wed, 6 Aug 2014, Julien Grall wrote:
> > Hi Stefano,
> >
> > On 08/06/2014 05:27 PM, Stefano Stabellini wrote:
> > > On Thu, 31 Jul 2014, Julien Grall wrote:
> > >> This region will be split by the toolstack to allocate MMIO range for eac
> > >> device.
> > >>
> > >> For now only reserve a 768MB region, this should be enought to passthrough
> > >> multiple device at the same time.
> > >>
> > >> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> > >>
> > >> ---
> > >>
> > >> I can't allocate easily a range of 1GB in the current layout without shrink
> > >> the first RAM bank. So I decided to use only 768MB. If in the future we need
> > >> more, then we will have to choose at runtime the guest layout.
> > >>
> > >> Changes in v2:
> > >> - Bump the reserved region from 512MB to 768MB
> > >
> > >
> > > Better than nothing.
> > > Would it be possible to add a second region > 4GB? Maybe after RAM?
> > > How hard would be to handle that?
> >
> > It won't work if the guest is using short-descriptor translation table
> > format because only 32 bits address will be supported.
>
> I know.
However it would make things easier if you want to assign a device with
a large MMIO region to a guest that can handle it.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-08-06 16:45 ` Stefano Stabellini
@ 2014-08-06 16:55 ` Julien Grall
2014-08-06 16:57 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 16:55 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On 08/06/2014 05:45 PM, Stefano Stabellini wrote:
> On Wed, 6 Aug 2014, Stefano Stabellini wrote:
>> On Wed, 6 Aug 2014, Julien Grall wrote:
>>> Hi Stefano,
>>>
>>> On 08/06/2014 05:27 PM, Stefano Stabellini wrote:
>>>> On Thu, 31 Jul 2014, Julien Grall wrote:
>>>>> This region will be split by the toolstack to allocate MMIO range for eac
>>>>> device.
>>>>>
>>>>> For now only reserve a 768MB region, this should be enought to passthrough
>>>>> multiple device at the same time.
>>>>>
>>>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>>>>
>>>>> ---
>>>>>
>>>>> I can't allocate easily a range of 1GB in the current layout without shrink
>>>>> the first RAM bank. So I decided to use only 768MB. If in the future we need
>>>>> more, then we will have to choose at runtime the guest layout.
>>>>>
>>>>> Changes in v2:
>>>>> - Bump the reserved region from 512MB to 768MB
>>>>
>>>>
>>>> Better than nothing.
>>>> Would it be possible to add a second region > 4GB? Maybe after RAM?
>>>> How hard would be to handle that?
>>>
>>> It won't work if the guest is using short-descriptor translation table
>>> format because only 32 bits address will be supported.
>>
>> I know.
>
> However it would make things easier if you want to assign a device with
> a large MMIO region to a guest that can handle it.
Last time you told me about graphics card. But we don't have any board
with graphic card and IOMMU supported by Xen.
In any case the developer will have to modify Xen to add the new
platform. It won't hurt to add a new region at this time.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-08-06 16:55 ` Julien Grall
@ 2014-08-06 16:57 ` Stefano Stabellini
0 siblings, 0 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:57 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, ian.campbell, stefano.stabellini, Stefano Stabellini
On Wed, 6 Aug 2014, Julien Grall wrote:
> On 08/06/2014 05:45 PM, Stefano Stabellini wrote:
> > On Wed, 6 Aug 2014, Stefano Stabellini wrote:
> >> On Wed, 6 Aug 2014, Julien Grall wrote:
> >>> Hi Stefano,
> >>>
> >>> On 08/06/2014 05:27 PM, Stefano Stabellini wrote:
> >>>> On Thu, 31 Jul 2014, Julien Grall wrote:
> >>>>> This region will be split by the toolstack to allocate MMIO range for eac
> >>>>> device.
> >>>>>
> >>>>> For now only reserve a 768MB region, this should be enought to passthrough
> >>>>> multiple device at the same time.
> >>>>>
> >>>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>>>>
> >>>>> ---
> >>>>>
> >>>>> I can't allocate easily a range of 1GB in the current layout without shrink
> >>>>> the first RAM bank. So I decided to use only 768MB. If in the future we need
> >>>>> more, then we will have to choose at runtime the guest layout.
> >>>>>
> >>>>> Changes in v2:
> >>>>> - Bump the reserved region from 512MB to 768MB
> >>>>
> >>>>
> >>>> Better than nothing.
> >>>> Would it be possible to add a second region > 4GB? Maybe after RAM?
> >>>> How hard would be to handle that?
> >>>
> >>> It won't work if the guest is using short-descriptor translation table
> >>> format because only 32 bits address will be supported.
> >>
> >> I know.
> >
> > However it would make things easier if you want to assign a device with
> > a large MMIO region to a guest that can handle it.
>
> Last time you told me about graphics card. But we don't have any board
> with graphic card and IOMMU supported by Xen.
> In any case the developer will have to modify Xen to add the new
> platform. It won't hurt to add a new region at this time.
I understand your concerns and in theory you are right, but in practice
it would be much easier to add it now than then. The new guy won't know
anything about device passthrough and libxl. While at the moment it is
all still new and fresh in our minds.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough
2014-08-06 16:44 ` Stefano Stabellini
2014-08-06 16:45 ` Stefano Stabellini
@ 2014-08-06 16:47 ` Julien Grall
1 sibling, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-08-06 16:47 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, tim, ian.campbell, stefano.stabellini
On 08/06/2014 05:44 PM, Stefano Stabellini wrote:
> On Wed, 6 Aug 2014, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 08/06/2014 05:27 PM, Stefano Stabellini wrote:
>>> On Thu, 31 Jul 2014, Julien Grall wrote:
>>>> This region will be split by the toolstack to allocate MMIO range for eac
>>>> device.
>>>>
>>>> For now only reserve a 768MB region, this should be enought to passthrough
>>>> multiple device at the same time.
>>>>
>>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>>>
>>>> ---
>>>>
>>>> I can't allocate easily a range of 1GB in the current layout without shrink
>>>> the first RAM bank. So I decided to use only 768MB. If in the future we need
>>>> more, then we will have to choose at runtime the guest layout.
>>>>
>>>> Changes in v2:
>>>> - Bump the reserved region from 512MB to 768MB
>>>
>>>
>>> Better than nothing.
>>> Would it be possible to add a second region > 4GB? Maybe after RAM?
>>> How hard would be to handle that?
>>
>> It won't work if the guest is using short-descriptor translation table
>> format because only 32 bits address will be supported.
>
> I know.
I don't plan to handle another region for this series. If someone come
with a real use case, then I will think about it.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (18 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 19/21] xen/arm: Reserve region in guest memory for device passthrough Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-08-06 16:44 ` Stefano Stabellini
2014-07-31 15:00 ` [PATCH v2 21/21] xl: Add new option dtdev Julien Grall
2014-09-09 14:34 ` [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Ian Campbell
21 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, tim, Julien Grall, Ian Jackson,
stefano.stabellini
On ARM, every non-PCI device are described in the device tree. Each of them
can be found via a path.
This path will be used to retrieved the different informations about the
device (compatible string, interrupts, MMIOs). Libxl will take care of:
- Allocate the MMIOs regions for the device in the guest
- Create the device node in the guest device tree
- Map the IRQs and MMIOs range in the guest P2M
Note, that the device node won't contains specific properties for the node.
Only generic one (compatible, interrupts, regs) will be created by libxl.
In the future, per-device properties will be added. Maybe via a configuration
file listing what is needed.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v2:
- Get DT infos earlier
- Allocate/map IRQ in libxl__arch_domain_create rather than in
libxl__device_dt_add
- Use VIRQ rather than the PIRQ to construct the interrupts
properties of the device tree
- Correct cpumask in make_dtdev_node. We allow the interrupt to
be used on the 8 CPUs
- Fix LOGE when we map the MMIO region in the guest in
libxl__device_dt_add. The domain and the IRQ were inverted
- Calculate the number of SPIs to configure the VGIC
- xc_physdev_dtdev_* helpers has been renamed to xc_dtdev_*
- Rename libxl_device_dt to libxl_device_dtdev
---
tools/libxl/Makefile | 2 +-
tools/libxl/libxl_arch.h | 4 +-
tools/libxl/libxl_arm.c | 129 ++++++++++++++++++++++++++++++++++-
tools/libxl/libxl_create.c | 27 ++++++++
tools/libxl/libxl_dom.c | 5 +-
tools/libxl/libxl_dtdev.c | 153 ++++++++++++++++++++++++++++++++++++++++++
tools/libxl/libxl_internal.h | 32 +++++++++
tools/libxl/libxl_types.idl | 5 ++
tools/libxl/libxl_x86.c | 4 +-
9 files changed, 355 insertions(+), 6 deletions(-)
create mode 100644 tools/libxl/libxl_dtdev.c
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index bd0db3b..dc38216 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -81,7 +81,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
libxl_json.o libxl_aoutils.o libxl_numa.o \
libxl_save_callout.o _libxl_save_msgs_callout.o \
libxl_qmp.o libxl_event.o libxl_fork.o $(LIBXL_OBJS-y)
-LIBXL_OBJS += libxl_genid.o
+LIBXL_OBJS += libxl_genid.o libxl_dtdev.o
LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
LIBXL_TESTS += timedereg
diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
index 454f8db..9532f02 100644
--- a/tools/libxl/libxl_arch.h
+++ b/tools/libxl/libxl_arch.h
@@ -20,11 +20,13 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
libxl__domain_build_state *state,
uint32_t domid);
int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
- uint32_t domid);
+ libxl__domain_build_state *state,
+ uint32_t domid);
/* setup arch specific hardware description, i.e. DTB on ARM */
int libxl__arch_domain_init_hw_description(libxl__gc *gc,
libxl_domain_build_info *info,
+ libxl__domain_build_state *state,
struct xc_dom_image *dom);
/* finalize arch specific hardware description. */
int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
index b0491c3..0609f86 100644
--- a/tools/libxl/libxl_arm.c
+++ b/tools/libxl/libxl_arm.c
@@ -27,8 +27,12 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
libxl__domain_build_state *state,
uint32_t domid)
{
+ int dev_index;
uint32_t nr_spis = 0;
+ for (dev_index = 0; dev_index < d_config->num_dtdevs; dev_index++)
+ nr_spis += state->dtdevs_info[dev_index].num_irqs;
+
nr_spis += d_config->b_info.num_irqs;
LOG(DEBUG, "Allocate %u SPIs\n", nr_spis);
@@ -43,8 +47,60 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
+ libxl__domain_build_state *state,
uint32_t domid)
{
+ int dev_index;
+ uint32_t mmio_index;
+ uint64_t mmiocurr = GUEST_MMIO_BASE >> XC_PAGE_SHIFT;
+ /* Convenient */
+ const uint64_t mmioend = (GUEST_MMIO_BASE + GUEST_MMIO_SIZE) >> XC_PAGE_SHIFT;
+ uint32_t i;
+ int ret;
+
+ for (dev_index = 0; dev_index < d_config->num_dtdevs; dev_index++) {
+ const libxl_device_dtdev *dtdev = &d_config->dtdevs[dev_index];
+ libxl__dtdev_info *info = &state->dtdevs_info[dev_index];
+
+ LOG(DEBUG, "Allocate %d MMIOs region for \"%s\"",
+ info->num_mmios, dtdev->path);
+ for (mmio_index = 0; mmio_index < info->num_mmios; mmio_index++) {
+ libxl_iomem_range *io = &info->mmios[mmio_index];
+
+ /* Check if we have enough space for the MMIO region */
+ /* TODO: Do I need to check overlap? */
+ if ((mmiocurr + io->number) > mmioend) {
+ LOG(ERROR, "Not enough space in the guest layout to allocate the MMIOs regions");
+ return -ENOMEM;
+ }
+ LOG(DEBUG, "\t0x%"PRIx64"-0x%"PRIx64,
+ mmiocurr, mmiocurr + io->number);
+ io->gfn = mmiocurr;
+ mmiocurr += io->number;
+ }
+
+ LOG(DEBUG, "Allocate %d virtual IRQs for \"%s\n",
+ info->num_irqs, dtdev->path);
+ for (i = 0; i < info->num_irqs; i++) {
+ int virq;
+ int irq = info->irqs[i].irq;
+
+ /* xc_physdev_map_pirq will both assign the IRQ to the guest
+ * and allocate a virtual IRQ number, we need it early in
+ * order to generate the device tree
+ */
+ ret = xc_physdev_map_pirq(CTX->xch, domid, irq, &virq);
+ if (ret < 0) {
+ LOGE(ERROR, "%s: failed to map the IRQ %u into dom%u\n",
+ info->conf->path, irq, domid);
+ return ret;
+ }
+
+ info->irqs[i].virq = virq;
+ LOG(DEBUG, "\t irq %u -> virq %u\n", irq, virq);
+ }
+ }
+
return 0;
}
@@ -431,6 +487,72 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
return 0;
}
+static int make_dtdev_node(libxl__gc *gc, void *fdt,
+ int index, const libxl__dtdev_info *dtdev)
+{
+ const char *name;
+ uint32_t i;
+ int res;
+
+ /* The unit-address (after @) is only request when the device has MMIO */
+ if (dtdev->num_mmios > 0) {
+ uint64_t base = dtdev->mmios[0].gfn << XC_PAGE_SHIFT;
+
+ name = GCSPRINTF("dtdev-%u@%"PRIx64, index, base);
+ } else
+ name = GCSPRINTF("dtdev-%u", index);
+
+ res = fdt_begin_node(fdt, name);
+ if (res) return res;
+
+ assert(dtdev->compat_len != 0);
+ fdt_property(fdt, "compatible", dtdev->compat, dtdev->compat_len);
+
+ if (dtdev->num_mmios > 0) {
+ be32 *regs, *cells;
+ /* Convenient */
+ const unsigned addr_cells = ROOT_ADDRESS_CELLS;
+ const unsigned size_cells = ROOT_SIZE_CELLS;
+ const unsigned len = sizeof(*regs) * (addr_cells + size_cells);
+
+ regs = libxl__malloc(gc, len);
+ cells = ®s[0];
+
+ for (i = 0; i < dtdev->num_mmios; i++) {
+ uint64_t base = dtdev->mmios[i].gfn << XC_PAGE_SHIFT;
+ uint64_t size = dtdev->mmios[i].number << XC_PAGE_SHIFT;
+
+ set_range(&cells, addr_cells, size_cells, base, size);
+ }
+
+ res = fdt_property(fdt, "reg", regs, len);
+ if (res) return res;
+ }
+
+ if (dtdev->num_irqs > 0) {
+ gic_interrupt *ints;
+
+ ints = libxl__malloc(gc, sizeof(*ints) * dtdev->num_irqs);
+ for (i =0; i < dtdev->num_irqs; i++) {
+ /* TODO: Translate the IRQ type into DT type. We should
+ * not assume a 1:1 mapping */
+ /* For now, Xen is only handling SPIs passthrough and
+ * forward to VCPU0
+ */
+ assert(dtdev->irqs[i].irq >= 32);
+ set_interrupt(ints[i], dtdev->irqs[i].virq,
+ 0xf, dtdev->irqs[i].type);
+ }
+
+ res = fdt_property_interrupts(gc, fdt, ints, dtdev->num_irqs);
+ }
+
+ res = fdt_end_node(fdt);
+ if (res) return res;
+
+ return 0;
+}
+
static const struct arch_info *get_arch_info(libxl__gc *gc,
const struct xc_dom_image *dom)
{
@@ -473,10 +595,11 @@ out:
int libxl__arch_domain_init_hw_description(libxl__gc *gc,
libxl_domain_build_info *info,
+ libxl__domain_build_state *state,
struct xc_dom_image *dom)
{
void *fdt = NULL;
- int rc, res;
+ int rc, res, i;
size_t fdt_size = 0;
const libxl_version_info *vers;
@@ -546,6 +669,10 @@ next_resize:
FDT( make_timer_node(gc, fdt, ainfo) );
FDT( make_hypervisor_node(gc, fdt, vers) );
+ for (i = 0; i < state->num_dtdevs; i++) {
+ FDT( make_dtdev_node(gc, fdt, i, &state->dtdevs_info[i]) );
+ }
+
FDT( fdt_end_node(fdt) );
FDT( fdt_finish(fdt) );
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 1bcee8b..6cb7c69 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -731,6 +731,7 @@ static void initiate_domain_create(libxl__egc *egc,
/* convenience aliases */
libxl_domain_config *const d_config = dcs->guest_config;
const int restore_fd = dcs->restore_fd;
+ libxl__domain_build_state *state = &dcs->build_state;
memset(&dcs->build_state, 0, sizeof(dcs->build_state));
domid = 0;
@@ -866,6 +867,21 @@ static void initiate_domain_create(libxl__egc *egc,
d_config->nics[i].devid = ++last_devid;
}
+ /* We need to retrieve DT devs information early to be able to
+ * configure the domain correctly (i.e allocate GFN for each MMIO
+ * regions, get the number of SPIs...) */
+ state->num_dtdevs = d_config->num_dtdevs;
+ state->dtdevs_info = libxl__calloc(gc, sizeof(*state->dtdevs_info),
+ d_config->num_dtdevs);
+ for (i = 0; i < d_config->num_dtdevs; i++) {
+ ret = libxl__device_dt_get_info(gc, &d_config->dtdevs[i],
+ &state->dtdevs_info[i]);
+ if (ret) {
+ LOG(ERROR, "libxl__device_dt_get_info failed: %d", ret);
+ goto error_out;
+ }
+ }
+
if (restore_fd >= 0) {
LOG(DEBUG, "restoring, not running bootloader\n");
domcreate_bootloader_done(egc, &dcs->bl, 0);
@@ -1391,6 +1407,7 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
/* convenience aliases */
libxl_domain_config *const d_config = dcs->guest_config;
+ libxl__domain_build_state *const state = &dcs->build_state;
if (ret) {
LOG(ERROR, "unable to add vtpm devices");
@@ -1416,6 +1433,16 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
}
}
+ for (i = 0; i < d_config->num_dtdevs; i++) {
+
+ ret = libxl__device_dt_add(gc, domid, &state->dtdevs_info[i]);
+ if (ret < 0) {
+ LIBXL__LOG(ctx, LIBXL__LOG_ERROR,
+ "libxl__device_dt_add failed: %d\n", ret);
+ goto error_out;
+ }
+ }
+
domcreate_console_available(egc, dcs);
domcreate_complete(egc, dcs, 0);
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index a0e4e34..d605a62 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -357,7 +357,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
if (info->type == LIBXL_DOMAIN_TYPE_HVM)
hvm_set_conf_params(ctx->xch, domid, info);
- rc = libxl__arch_domain_create(gc, d_config, domid);
+ rc = libxl__arch_domain_create(gc, d_config, state, domid);
return rc;
}
@@ -509,7 +509,8 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
LOGE(ERROR, "xc_dom_parse_image failed");
goto out;
}
- if ( (ret = libxl__arch_domain_init_hw_description(gc, info, dom)) != 0 ) {
+ ret = libxl__arch_domain_init_hw_description(gc, info, state, dom);
+ if ( ret != 0 ) {
LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
goto out;
}
diff --git a/tools/libxl/libxl_dtdev.c b/tools/libxl/libxl_dtdev.c
new file mode 100644
index 0000000..6eb3f07
--- /dev/null
+++ b/tools/libxl/libxl_dtdev.c
@@ -0,0 +1,153 @@
+/*
+ * Copyright (C) 2014 Linaro Limited.
+ * Author Julien Grall <julien.grall@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* Must come before other headers */
+
+#include "libxl_internal.h"
+
+int libxl__device_dt_add(libxl__gc *gc, uint32_t domid,
+ const libxl__dtdev_info *dtdev)
+{
+ uint32_t i;
+ int ret;
+
+ LOG(DEBUG, "Assign device \"%s\" to dom%u", dtdev->conf->path, domid);
+
+ for (i = 0; i < dtdev->num_mmios; i++) {
+ const libxl_iomem_range *io = &dtdev->mmios[i];
+
+ ret = xc_domain_iomem_permission(CTX->xch, domid, io->start,
+ io->number, 1);
+ if (ret < 0) {
+ LOGE(ERROR,
+ "%s: failed to give dom%d access to iomem range"
+ "%"PRIx64"-%"PRIx64, dtdev->conf->path,
+ domid, io->start, io->start + io->number + 1);
+ return ret;
+ }
+
+ ret = xc_domain_memory_mapping(CTX->xch, domid, io->gfn,
+ io->start, io->number, 1);
+ if (ret < 0) {
+ LOGE(ERROR,
+ "%s: failed to map to dom%u iomem range %"PRIx64"-%"PRIx64
+ " to guest address %"PRIx64,
+ dtdev->conf->path, domid, io->start,
+ io->start + io->number - 1, io->gfn);
+ return ret;
+ }
+ }
+
+ /* IRQs are already assigned in libxl__arch_domain_create because
+ * it's at the same time used to allocate the virtual IRQ number
+ */
+
+ return xc_assign_dt_device(CTX->xch, domid, dtdev->conf->path);
+}
+
+int libxl__device_dt_get_info(libxl__gc *gc,
+ const libxl_device_dtdev *dtdev,
+ libxl__dtdev_info *dtinfo)
+{
+ xc_dtdev_info_t info;
+ int ret = 0;
+ uint32_t i;
+ char *buff;
+ uint32_t len;
+
+ LOG(DEBUG, "Get information for DT dev \"%s\"", dtdev->path);
+
+ ret = xc_dtdev_getinfo(CTX->xch, dtdev->path, &info);
+ if (ret) {
+ LOGE(ERROR, "Unable to get the informations for \"%s\"",
+ dtdev->path);
+ return ret;
+ }
+
+ LOG(DEBUG, "num_irqs = %u num_mmios = %u compat_len = %u",
+ info.num_irqs, info.num_mmios, info.compat_len);
+
+ dtinfo->conf = dtdev;
+
+ /* Retrieve the IRQs */
+ dtinfo->num_irqs = info.num_irqs;
+ dtinfo->irqs = libxl__calloc(gc, dtinfo->num_irqs,
+ sizeof(*dtinfo->mmios));
+
+ LOG(DEBUG, "List of IRQs");
+ for (i = 0; i < dtinfo->num_irqs; i++) {
+ xc_dtdev_irq_t irq;
+
+ ret = xc_dtdev_getirq(CTX->xch, dtdev->path, i, &irq);
+ if (ret) {
+ LOGE(ERROR, "Unable to get IRQ%u for \"%s\"", i, dtdev->path);
+ return ret;
+ }
+
+ LOG(DEBUG, "\t- irq = %u type = %u", irq.irq, irq.type);
+
+ dtinfo->irqs[i].irq = irq.irq;
+ /* TODO translate the type correctly */
+ dtinfo->irqs[i].type = irq.type;
+ }
+
+ /* Retrieve the MMIOs range */
+ dtinfo->num_mmios = info.num_mmios;
+ dtinfo->mmios = libxl__calloc(gc, dtinfo->num_mmios,
+ sizeof(*dtinfo->mmios));
+
+ LOG(DEBUG, "List of MMIOs");
+ for (i = 0; i < dtinfo->num_mmios; i++) {
+ xc_dtdev_mmio_t mmio;
+
+ ret = xc_dtdev_getmmio(CTX->xch, dtdev->path, i, &mmio);
+ if (ret) {
+ LOGE(ERROR, "Unable to get the MMIO range %u for \"%s\"",
+ i, dtdev->path);
+ return ret;
+ }
+
+ LOG(DEBUG, "\t- mfn = 0x%"PRIx64" nr_mfn = 0x%"PRIx64,
+ mmio.mfn, mmio.nr_mfn);
+
+ dtinfo->mmios[i].start = mmio.mfn;
+ dtinfo->mmios[i].number = mmio.nr_mfn;
+ dtinfo->mmios[i].gfn = LIBXL_INVALID_GFN;
+ }
+
+ /* Retrieve the compatible property */
+ len = info.compat_len;
+ buff = libxl__malloc(gc, len);
+
+ ret = xc_dtdev_getcompat(CTX->xch, dtdev->path,
+ buff, &len);
+ if (ret) {
+ LOGE(ERROR, "Unable to get the compatible string for \"%s\"",
+ dtdev->path);
+ return ret;
+ }
+ dtinfo->compat_len = len;
+ dtinfo->compat = buff;
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 04c9378..2231b76 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -944,6 +944,26 @@ typedef struct {
_hidden int libxl__file_reference_map(libxl__file_reference *f);
_hidden int libxl__file_reference_unmap(libxl__file_reference *f);
+/* DT dev representation */
+typedef struct {
+ uint32_t num_mmios;
+ libxl_iomem_range *mmios;
+
+ uint32_t num_irqs;
+ struct
+ {
+ uint32_t irq;
+ uint32_t virq;
+ uint32_t type;
+ } *irqs;
+
+ const char *compat;
+ ssize_t compat_len; /* The compatible string may contain \0 */
+
+ /* Short-hand to the User configuration for this device */
+ const libxl_device_dtdev *conf;
+} libxl__dtdev_info;
+
/* from xl_dom */
_hidden libxl_domain_type libxl__domain_type(libxl__gc *gc, uint32_t domid);
_hidden int libxl__domain_shutdown_reason(libxl__gc *gc, uint32_t domid);
@@ -967,6 +987,9 @@ typedef struct {
libxl__file_reference pv_ramdisk;
const char * pv_cmdline;
bool pvh_enabled;
+
+ int num_dtdevs;
+ libxl__dtdev_info *dtdevs_info;
} libxl__domain_build_state;
_hidden int libxl__build_pre(libxl__gc *gc, uint32_t domid,
@@ -1150,6 +1173,15 @@ _hidden int libxl__create_pci_backend(libxl__gc *gc, uint32_t domid,
libxl_device_pci *pcidev, int num);
_hidden int libxl__device_pci_destroy_all(libxl__gc *gc, uint32_t domid);
+/* from libxl_dtdev */
+
+_hidden int libxl__device_dt_add(libxl__gc *gc, uint32_t domid,
+ const libxl__dtdev_info *info);
+_hidden int libxl__device_dt_get_info(libxl__gc *gc,
+ const libxl_device_dtdev *dtdev,
+ libxl__dtdev_info *info);
+
+
/*----- xswait: wait for a xenstore node to be suitable -----*/
typedef struct libxl__xswait_state libxl__xswait_state;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 874cc49..03a4367 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -477,6 +477,10 @@ libxl_device_pci = Struct("device_pci", [
("seize", bool),
])
+libxl_device_dtdev = Struct("device_dtdev", [
+ ("path", string),
+ ])
+
libxl_device_vtpm = Struct("device_vtpm", [
("backend_domid", libxl_domid),
("backend_domname", string),
@@ -491,6 +495,7 @@ libxl_domain_config = Struct("domain_config", [
("disks", Array(libxl_device_disk, "num_disks")),
("nics", Array(libxl_device_nic, "num_nics")),
("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
+ ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
("vfbs", Array(libxl_device_vfb, "num_vfbs")),
("vkbs", Array(libxl_device_vkb, "num_vkbs")),
("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
index 0abc1aa..17807c2 100644
--- a/tools/libxl/libxl_x86.c
+++ b/tools/libxl/libxl_x86.c
@@ -252,7 +252,8 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
}
int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
- uint32_t domid)
+ libxl__domain_build_state *state,
+ uint32_t domid)
{
int ret = 0;
int tsc_mode;
@@ -320,6 +321,7 @@ int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
int libxl__arch_domain_init_hw_description(libxl__gc *gc,
libxl_domain_build_info *info,
+ libxl__domain_build_state *state,
struct xc_dom_image *dom)
{
return 0;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-07-31 15:00 ` [PATCH v2 20/21] libxl: Add support for non-PCI passthrough Julien Grall
@ 2014-08-06 16:44 ` Stefano Stabellini
2014-08-06 16:50 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:44 UTC (permalink / raw)
To: Julien Grall
Cc: ian.campbell, Stefano Stabellini, tim, Ian Jackson,
stefano.stabellini, xen-devel
On Thu, 31 Jul 2014, Julien Grall wrote:
> On ARM, every non-PCI device are described in the device tree. Each of them
> can be found via a path.
>
> This path will be used to retrieved the different informations about the
> device (compatible string, interrupts, MMIOs). Libxl will take care of:
> - Allocate the MMIOs regions for the device in the guest
> - Create the device node in the guest device tree
> - Map the IRQs and MMIOs range in the guest P2M
>
> Note, that the device node won't contains specific properties for the node.
> Only generic one (compatible, interrupts, regs) will be created by libxl.
>
> In the future, per-device properties will be added. Maybe via a configuration
> file listing what is needed.
>
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
>
> ---
> Changes in v2:
> - Get DT infos earlier
> - Allocate/map IRQ in libxl__arch_domain_create rather than in
> libxl__device_dt_add
I understand why you moved it earlier but if it is not too hard I would
suggest to try to keep mmio and irq mapping all together, because it
would make the code much easier to read and modify in the future, being
all in one place.
The rest looks OK.
> - Use VIRQ rather than the PIRQ to construct the interrupts
> properties of the device tree
> - Correct cpumask in make_dtdev_node. We allow the interrupt to
> be used on the 8 CPUs
> - Fix LOGE when we map the MMIO region in the guest in
> libxl__device_dt_add. The domain and the IRQ were inverted
> - Calculate the number of SPIs to configure the VGIC
> - xc_physdev_dtdev_* helpers has been renamed to xc_dtdev_*
> - Rename libxl_device_dt to libxl_device_dtdev
> ---
> tools/libxl/Makefile | 2 +-
> tools/libxl/libxl_arch.h | 4 +-
> tools/libxl/libxl_arm.c | 129 ++++++++++++++++++++++++++++++++++-
> tools/libxl/libxl_create.c | 27 ++++++++
> tools/libxl/libxl_dom.c | 5 +-
> tools/libxl/libxl_dtdev.c | 153 ++++++++++++++++++++++++++++++++++++++++++
> tools/libxl/libxl_internal.h | 32 +++++++++
> tools/libxl/libxl_types.idl | 5 ++
> tools/libxl/libxl_x86.c | 4 +-
> 9 files changed, 355 insertions(+), 6 deletions(-)
> create mode 100644 tools/libxl/libxl_dtdev.c
>
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> index bd0db3b..dc38216 100644
> --- a/tools/libxl/Makefile
> +++ b/tools/libxl/Makefile
> @@ -81,7 +81,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
> libxl_json.o libxl_aoutils.o libxl_numa.o \
> libxl_save_callout.o _libxl_save_msgs_callout.o \
> libxl_qmp.o libxl_event.o libxl_fork.o $(LIBXL_OBJS-y)
> -LIBXL_OBJS += libxl_genid.o
> +LIBXL_OBJS += libxl_genid.o libxl_dtdev.o
> LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
>
> LIBXL_TESTS += timedereg
> diff --git a/tools/libxl/libxl_arch.h b/tools/libxl/libxl_arch.h
> index 454f8db..9532f02 100644
> --- a/tools/libxl/libxl_arch.h
> +++ b/tools/libxl/libxl_arch.h
> @@ -20,11 +20,13 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> libxl__domain_build_state *state,
> uint32_t domid);
> int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> - uint32_t domid);
> + libxl__domain_build_state *state,
> + uint32_t domid);
>
> /* setup arch specific hardware description, i.e. DTB on ARM */
> int libxl__arch_domain_init_hw_description(libxl__gc *gc,
> libxl_domain_build_info *info,
> + libxl__domain_build_state *state,
> struct xc_dom_image *dom);
> /* finalize arch specific hardware description. */
> int libxl__arch_domain_finalise_hw_description(libxl__gc *gc,
> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c
> index b0491c3..0609f86 100644
> --- a/tools/libxl/libxl_arm.c
> +++ b/tools/libxl/libxl_arm.c
> @@ -27,8 +27,12 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> libxl__domain_build_state *state,
> uint32_t domid)
> {
> + int dev_index;
> uint32_t nr_spis = 0;
>
> + for (dev_index = 0; dev_index < d_config->num_dtdevs; dev_index++)
> + nr_spis += state->dtdevs_info[dev_index].num_irqs;
> +
> nr_spis += d_config->b_info.num_irqs;
>
> LOG(DEBUG, "Allocate %u SPIs\n", nr_spis);
> @@ -43,8 +47,60 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
>
>
> int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> + libxl__domain_build_state *state,
> uint32_t domid)
> {
> + int dev_index;
> + uint32_t mmio_index;
> + uint64_t mmiocurr = GUEST_MMIO_BASE >> XC_PAGE_SHIFT;
> + /* Convenient */
> + const uint64_t mmioend = (GUEST_MMIO_BASE + GUEST_MMIO_SIZE) >> XC_PAGE_SHIFT;
> + uint32_t i;
> + int ret;
> +
> + for (dev_index = 0; dev_index < d_config->num_dtdevs; dev_index++) {
> + const libxl_device_dtdev *dtdev = &d_config->dtdevs[dev_index];
> + libxl__dtdev_info *info = &state->dtdevs_info[dev_index];
> +
> + LOG(DEBUG, "Allocate %d MMIOs region for \"%s\"",
> + info->num_mmios, dtdev->path);
> + for (mmio_index = 0; mmio_index < info->num_mmios; mmio_index++) {
> + libxl_iomem_range *io = &info->mmios[mmio_index];
> +
> + /* Check if we have enough space for the MMIO region */
> + /* TODO: Do I need to check overlap? */
> + if ((mmiocurr + io->number) > mmioend) {
> + LOG(ERROR, "Not enough space in the guest layout to allocate the MMIOs regions");
> + return -ENOMEM;
> + }
> + LOG(DEBUG, "\t0x%"PRIx64"-0x%"PRIx64,
> + mmiocurr, mmiocurr + io->number);
> + io->gfn = mmiocurr;
> + mmiocurr += io->number;
> + }
> +
> + LOG(DEBUG, "Allocate %d virtual IRQs for \"%s\n",
> + info->num_irqs, dtdev->path);
> + for (i = 0; i < info->num_irqs; i++) {
> + int virq;
> + int irq = info->irqs[i].irq;
> +
> + /* xc_physdev_map_pirq will both assign the IRQ to the guest
> + * and allocate a virtual IRQ number, we need it early in
> + * order to generate the device tree
> + */
> + ret = xc_physdev_map_pirq(CTX->xch, domid, irq, &virq);
> + if (ret < 0) {
> + LOGE(ERROR, "%s: failed to map the IRQ %u into dom%u\n",
> + info->conf->path, irq, domid);
> + return ret;
> + }
> +
> + info->irqs[i].virq = virq;
> + LOG(DEBUG, "\t irq %u -> virq %u\n", irq, virq);
> + }
> + }
> +
> return 0;
> }
>
> @@ -431,6 +487,72 @@ static int make_hypervisor_node(libxl__gc *gc, void *fdt,
> return 0;
> }
>
> +static int make_dtdev_node(libxl__gc *gc, void *fdt,
> + int index, const libxl__dtdev_info *dtdev)
> +{
> + const char *name;
> + uint32_t i;
> + int res;
> +
> + /* The unit-address (after @) is only request when the device has MMIO */
> + if (dtdev->num_mmios > 0) {
> + uint64_t base = dtdev->mmios[0].gfn << XC_PAGE_SHIFT;
> +
> + name = GCSPRINTF("dtdev-%u@%"PRIx64, index, base);
> + } else
> + name = GCSPRINTF("dtdev-%u", index);
> +
> + res = fdt_begin_node(fdt, name);
> + if (res) return res;
> +
> + assert(dtdev->compat_len != 0);
> + fdt_property(fdt, "compatible", dtdev->compat, dtdev->compat_len);
> +
> + if (dtdev->num_mmios > 0) {
> + be32 *regs, *cells;
> + /* Convenient */
> + const unsigned addr_cells = ROOT_ADDRESS_CELLS;
> + const unsigned size_cells = ROOT_SIZE_CELLS;
> + const unsigned len = sizeof(*regs) * (addr_cells + size_cells);
> +
> + regs = libxl__malloc(gc, len);
> + cells = ®s[0];
> +
> + for (i = 0; i < dtdev->num_mmios; i++) {
> + uint64_t base = dtdev->mmios[i].gfn << XC_PAGE_SHIFT;
> + uint64_t size = dtdev->mmios[i].number << XC_PAGE_SHIFT;
> +
> + set_range(&cells, addr_cells, size_cells, base, size);
> + }
> +
> + res = fdt_property(fdt, "reg", regs, len);
> + if (res) return res;
> + }
> +
> + if (dtdev->num_irqs > 0) {
> + gic_interrupt *ints;
> +
> + ints = libxl__malloc(gc, sizeof(*ints) * dtdev->num_irqs);
> + for (i =0; i < dtdev->num_irqs; i++) {
> + /* TODO: Translate the IRQ type into DT type. We should
> + * not assume a 1:1 mapping */
> + /* For now, Xen is only handling SPIs passthrough and
> + * forward to VCPU0
> + */
> + assert(dtdev->irqs[i].irq >= 32);
> + set_interrupt(ints[i], dtdev->irqs[i].virq,
> + 0xf, dtdev->irqs[i].type);
> + }
> +
> + res = fdt_property_interrupts(gc, fdt, ints, dtdev->num_irqs);
> + }
> +
> + res = fdt_end_node(fdt);
> + if (res) return res;
> +
> + return 0;
> +}
> +
> static const struct arch_info *get_arch_info(libxl__gc *gc,
> const struct xc_dom_image *dom)
> {
> @@ -473,10 +595,11 @@ out:
>
> int libxl__arch_domain_init_hw_description(libxl__gc *gc,
> libxl_domain_build_info *info,
> + libxl__domain_build_state *state,
> struct xc_dom_image *dom)
> {
> void *fdt = NULL;
> - int rc, res;
> + int rc, res, i;
> size_t fdt_size = 0;
>
> const libxl_version_info *vers;
> @@ -546,6 +669,10 @@ next_resize:
> FDT( make_timer_node(gc, fdt, ainfo) );
> FDT( make_hypervisor_node(gc, fdt, vers) );
>
> + for (i = 0; i < state->num_dtdevs; i++) {
> + FDT( make_dtdev_node(gc, fdt, i, &state->dtdevs_info[i]) );
> + }
> +
> FDT( fdt_end_node(fdt) );
>
> FDT( fdt_finish(fdt) );
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index 1bcee8b..6cb7c69 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -731,6 +731,7 @@ static void initiate_domain_create(libxl__egc *egc,
> /* convenience aliases */
> libxl_domain_config *const d_config = dcs->guest_config;
> const int restore_fd = dcs->restore_fd;
> + libxl__domain_build_state *state = &dcs->build_state;
> memset(&dcs->build_state, 0, sizeof(dcs->build_state));
>
> domid = 0;
> @@ -866,6 +867,21 @@ static void initiate_domain_create(libxl__egc *egc,
> d_config->nics[i].devid = ++last_devid;
> }
>
> + /* We need to retrieve DT devs information early to be able to
> + * configure the domain correctly (i.e allocate GFN for each MMIO
> + * regions, get the number of SPIs...) */
> + state->num_dtdevs = d_config->num_dtdevs;
> + state->dtdevs_info = libxl__calloc(gc, sizeof(*state->dtdevs_info),
> + d_config->num_dtdevs);
> + for (i = 0; i < d_config->num_dtdevs; i++) {
> + ret = libxl__device_dt_get_info(gc, &d_config->dtdevs[i],
> + &state->dtdevs_info[i]);
> + if (ret) {
> + LOG(ERROR, "libxl__device_dt_get_info failed: %d", ret);
> + goto error_out;
> + }
> + }
> +
> if (restore_fd >= 0) {
> LOG(DEBUG, "restoring, not running bootloader\n");
> domcreate_bootloader_done(egc, &dcs->bl, 0);
> @@ -1391,6 +1407,7 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
>
> /* convenience aliases */
> libxl_domain_config *const d_config = dcs->guest_config;
> + libxl__domain_build_state *const state = &dcs->build_state;
>
> if (ret) {
> LOG(ERROR, "unable to add vtpm devices");
> @@ -1416,6 +1433,16 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
> }
> }
>
> + for (i = 0; i < d_config->num_dtdevs; i++) {
> +
> + ret = libxl__device_dt_add(gc, domid, &state->dtdevs_info[i]);
> + if (ret < 0) {
> + LIBXL__LOG(ctx, LIBXL__LOG_ERROR,
> + "libxl__device_dt_add failed: %d\n", ret);
> + goto error_out;
> + }
> + }
> +
> domcreate_console_available(egc, dcs);
>
> domcreate_complete(egc, dcs, 0);
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index a0e4e34..d605a62 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -357,7 +357,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
> if (info->type == LIBXL_DOMAIN_TYPE_HVM)
> hvm_set_conf_params(ctx->xch, domid, info);
>
> - rc = libxl__arch_domain_create(gc, d_config, domid);
> + rc = libxl__arch_domain_create(gc, d_config, state, domid);
>
> return rc;
> }
> @@ -509,7 +509,8 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
> LOGE(ERROR, "xc_dom_parse_image failed");
> goto out;
> }
> - if ( (ret = libxl__arch_domain_init_hw_description(gc, info, dom)) != 0 ) {
> + ret = libxl__arch_domain_init_hw_description(gc, info, state, dom);
> + if ( ret != 0 ) {
> LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
> goto out;
> }
> diff --git a/tools/libxl/libxl_dtdev.c b/tools/libxl/libxl_dtdev.c
> new file mode 100644
> index 0000000..6eb3f07
> --- /dev/null
> +++ b/tools/libxl/libxl_dtdev.c
> @@ -0,0 +1,153 @@
> +/*
> + * Copyright (C) 2014 Linaro Limited.
> + * Author Julien Grall <julien.grall@linaro.org>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "libxl_osdeps.h" /* Must come before other headers */
> +
> +#include "libxl_internal.h"
> +
> +int libxl__device_dt_add(libxl__gc *gc, uint32_t domid,
> + const libxl__dtdev_info *dtdev)
> +{
> + uint32_t i;
> + int ret;
> +
> + LOG(DEBUG, "Assign device \"%s\" to dom%u", dtdev->conf->path, domid);
> +
> + for (i = 0; i < dtdev->num_mmios; i++) {
> + const libxl_iomem_range *io = &dtdev->mmios[i];
> +
> + ret = xc_domain_iomem_permission(CTX->xch, domid, io->start,
> + io->number, 1);
> + if (ret < 0) {
> + LOGE(ERROR,
> + "%s: failed to give dom%d access to iomem range"
> + "%"PRIx64"-%"PRIx64, dtdev->conf->path,
> + domid, io->start, io->start + io->number + 1);
> + return ret;
> + }
> +
> + ret = xc_domain_memory_mapping(CTX->xch, domid, io->gfn,
> + io->start, io->number, 1);
> + if (ret < 0) {
> + LOGE(ERROR,
> + "%s: failed to map to dom%u iomem range %"PRIx64"-%"PRIx64
> + " to guest address %"PRIx64,
> + dtdev->conf->path, domid, io->start,
> + io->start + io->number - 1, io->gfn);
> + return ret;
> + }
> + }
> +
> + /* IRQs are already assigned in libxl__arch_domain_create because
> + * it's at the same time used to allocate the virtual IRQ number
> + */
> +
> + return xc_assign_dt_device(CTX->xch, domid, dtdev->conf->path);
> +}
> +
> +int libxl__device_dt_get_info(libxl__gc *gc,
> + const libxl_device_dtdev *dtdev,
> + libxl__dtdev_info *dtinfo)
> +{
> + xc_dtdev_info_t info;
> + int ret = 0;
> + uint32_t i;
> + char *buff;
> + uint32_t len;
> +
> + LOG(DEBUG, "Get information for DT dev \"%s\"", dtdev->path);
> +
> + ret = xc_dtdev_getinfo(CTX->xch, dtdev->path, &info);
> + if (ret) {
> + LOGE(ERROR, "Unable to get the informations for \"%s\"",
> + dtdev->path);
> + return ret;
> + }
> +
> + LOG(DEBUG, "num_irqs = %u num_mmios = %u compat_len = %u",
> + info.num_irqs, info.num_mmios, info.compat_len);
> +
> + dtinfo->conf = dtdev;
> +
> + /* Retrieve the IRQs */
> + dtinfo->num_irqs = info.num_irqs;
> + dtinfo->irqs = libxl__calloc(gc, dtinfo->num_irqs,
> + sizeof(*dtinfo->mmios));
> +
> + LOG(DEBUG, "List of IRQs");
> + for (i = 0; i < dtinfo->num_irqs; i++) {
> + xc_dtdev_irq_t irq;
> +
> + ret = xc_dtdev_getirq(CTX->xch, dtdev->path, i, &irq);
> + if (ret) {
> + LOGE(ERROR, "Unable to get IRQ%u for \"%s\"", i, dtdev->path);
> + return ret;
> + }
> +
> + LOG(DEBUG, "\t- irq = %u type = %u", irq.irq, irq.type);
> +
> + dtinfo->irqs[i].irq = irq.irq;
> + /* TODO translate the type correctly */
> + dtinfo->irqs[i].type = irq.type;
> + }
> +
> + /* Retrieve the MMIOs range */
> + dtinfo->num_mmios = info.num_mmios;
> + dtinfo->mmios = libxl__calloc(gc, dtinfo->num_mmios,
> + sizeof(*dtinfo->mmios));
> +
> + LOG(DEBUG, "List of MMIOs");
> + for (i = 0; i < dtinfo->num_mmios; i++) {
> + xc_dtdev_mmio_t mmio;
> +
> + ret = xc_dtdev_getmmio(CTX->xch, dtdev->path, i, &mmio);
> + if (ret) {
> + LOGE(ERROR, "Unable to get the MMIO range %u for \"%s\"",
> + i, dtdev->path);
> + return ret;
> + }
> +
> + LOG(DEBUG, "\t- mfn = 0x%"PRIx64" nr_mfn = 0x%"PRIx64,
> + mmio.mfn, mmio.nr_mfn);
> +
> + dtinfo->mmios[i].start = mmio.mfn;
> + dtinfo->mmios[i].number = mmio.nr_mfn;
> + dtinfo->mmios[i].gfn = LIBXL_INVALID_GFN;
> + }
> +
> + /* Retrieve the compatible property */
> + len = info.compat_len;
> + buff = libxl__malloc(gc, len);
> +
> + ret = xc_dtdev_getcompat(CTX->xch, dtdev->path,
> + buff, &len);
> + if (ret) {
> + LOGE(ERROR, "Unable to get the compatible string for \"%s\"",
> + dtdev->path);
> + return ret;
> + }
> + dtinfo->compat_len = len;
> + dtinfo->compat = buff;
> +
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index 04c9378..2231b76 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -944,6 +944,26 @@ typedef struct {
> _hidden int libxl__file_reference_map(libxl__file_reference *f);
> _hidden int libxl__file_reference_unmap(libxl__file_reference *f);
>
> +/* DT dev representation */
> +typedef struct {
> + uint32_t num_mmios;
> + libxl_iomem_range *mmios;
> +
> + uint32_t num_irqs;
> + struct
> + {
> + uint32_t irq;
> + uint32_t virq;
> + uint32_t type;
> + } *irqs;
> +
> + const char *compat;
> + ssize_t compat_len; /* The compatible string may contain \0 */
> +
> + /* Short-hand to the User configuration for this device */
> + const libxl_device_dtdev *conf;
> +} libxl__dtdev_info;
> +
> /* from xl_dom */
> _hidden libxl_domain_type libxl__domain_type(libxl__gc *gc, uint32_t domid);
> _hidden int libxl__domain_shutdown_reason(libxl__gc *gc, uint32_t domid);
> @@ -967,6 +987,9 @@ typedef struct {
> libxl__file_reference pv_ramdisk;
> const char * pv_cmdline;
> bool pvh_enabled;
> +
> + int num_dtdevs;
> + libxl__dtdev_info *dtdevs_info;
> } libxl__domain_build_state;
>
> _hidden int libxl__build_pre(libxl__gc *gc, uint32_t domid,
> @@ -1150,6 +1173,15 @@ _hidden int libxl__create_pci_backend(libxl__gc *gc, uint32_t domid,
> libxl_device_pci *pcidev, int num);
> _hidden int libxl__device_pci_destroy_all(libxl__gc *gc, uint32_t domid);
>
> +/* from libxl_dtdev */
> +
> +_hidden int libxl__device_dt_add(libxl__gc *gc, uint32_t domid,
> + const libxl__dtdev_info *info);
> +_hidden int libxl__device_dt_get_info(libxl__gc *gc,
> + const libxl_device_dtdev *dtdev,
> + libxl__dtdev_info *info);
> +
> +
> /*----- xswait: wait for a xenstore node to be suitable -----*/
>
> typedef struct libxl__xswait_state libxl__xswait_state;
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 874cc49..03a4367 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -477,6 +477,10 @@ libxl_device_pci = Struct("device_pci", [
> ("seize", bool),
> ])
>
> +libxl_device_dtdev = Struct("device_dtdev", [
> + ("path", string),
> + ])
> +
> libxl_device_vtpm = Struct("device_vtpm", [
> ("backend_domid", libxl_domid),
> ("backend_domname", string),
> @@ -491,6 +495,7 @@ libxl_domain_config = Struct("domain_config", [
> ("disks", Array(libxl_device_disk, "num_disks")),
> ("nics", Array(libxl_device_nic, "num_nics")),
> ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
> + ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
> ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
> ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
> ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
> diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
> index 0abc1aa..17807c2 100644
> --- a/tools/libxl/libxl_x86.c
> +++ b/tools/libxl/libxl_x86.c
> @@ -252,7 +252,8 @@ int libxl__arch_domain_create_pre(libxl__gc *gc, libxl_domain_config *d_config,
> }
>
> int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
> - uint32_t domid)
> + libxl__domain_build_state *state,
> + uint32_t domid)
> {
> int ret = 0;
> int tsc_mode;
> @@ -320,6 +321,7 @@ int libxl__arch_domain_create(libxl__gc *gc, libxl_domain_config *d_config,
>
> int libxl__arch_domain_init_hw_description(libxl__gc *gc,
> libxl_domain_build_info *info,
> + libxl__domain_build_state *state,
> struct xc_dom_image *dom)
> {
> return 0;
> --
> 1.7.10.4
>
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-08-06 16:44 ` Stefano Stabellini
@ 2014-08-06 16:50 ` Julien Grall
2014-08-06 16:58 ` Stefano Stabellini
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-08-06 16:50 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Ian Jackson, xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
On 08/06/2014 05:44 PM, Stefano Stabellini wrote:
> On Thu, 31 Jul 2014, Julien Grall wrote:
>> On ARM, every non-PCI device are described in the device tree. Each of them
>> can be found via a path.
>>
>> This path will be used to retrieved the different informations about the
>> device (compatible string, interrupts, MMIOs). Libxl will take care of:
>> - Allocate the MMIOs regions for the device in the guest
>> - Create the device node in the guest device tree
>> - Map the IRQs and MMIOs range in the guest P2M
>>
>> Note, that the device node won't contains specific properties for the node.
>> Only generic one (compatible, interrupts, regs) will be created by libxl.
>>
>> In the future, per-device properties will be added. Maybe via a configuration
>> file listing what is needed.
>>
>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>>
>> ---
>> Changes in v2:
>> - Get DT infos earlier
>> - Allocate/map IRQ in libxl__arch_domain_create rather than in
>> libxl__device_dt_add
>
> I understand why you moved it earlier but if it is not too hard I would
> suggest to try to keep mmio and irq mapping all together, because it
> would make the code much easier to read and modify in the future, being
> all in one place.
I could fixup the device tree later. So we don't have to "map/retrieve"
the IRQ earlier.
But I don't much like this solution.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-08-06 16:50 ` Julien Grall
@ 2014-08-06 16:58 ` Stefano Stabellini
2014-08-08 14:15 ` Julien Grall
2014-09-09 19:12 ` Julien Grall
0 siblings, 2 replies; 119+ messages in thread
From: Stefano Stabellini @ 2014-08-06 16:58 UTC (permalink / raw)
To: Julien Grall
Cc: ian.campbell, Stefano Stabellini, tim, Ian Jackson,
stefano.stabellini, xen-devel
On Wed, 6 Aug 2014, Julien Grall wrote:
> Hi Stefano,
>
> On 08/06/2014 05:44 PM, Stefano Stabellini wrote:
> > On Thu, 31 Jul 2014, Julien Grall wrote:
> >> On ARM, every non-PCI device are described in the device tree. Each of them
> >> can be found via a path.
> >>
> >> This path will be used to retrieved the different informations about the
> >> device (compatible string, interrupts, MMIOs). Libxl will take care of:
> >> - Allocate the MMIOs regions for the device in the guest
> >> - Create the device node in the guest device tree
> >> - Map the IRQs and MMIOs range in the guest P2M
> >>
> >> Note, that the device node won't contains specific properties for the node.
> >> Only generic one (compatible, interrupts, regs) will be created by libxl.
> >>
> >> In the future, per-device properties will be added. Maybe via a configuration
> >> file listing what is needed.
> >>
> >> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> >> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> >> Cc: Ian Campbell <ian.campbell@citrix.com>
> >>
> >> ---
> >> Changes in v2:
> >> - Get DT infos earlier
> >> - Allocate/map IRQ in libxl__arch_domain_create rather than in
> >> libxl__device_dt_add
> >
> > I understand why you moved it earlier but if it is not too hard I would
> > suggest to try to keep mmio and irq mapping all together, because it
> > would make the code much easier to read and modify in the future, being
> > all in one place.
>
> I could fixup the device tree later. So we don't have to "map/retrieve"
> the IRQ earlier.
>
> But I don't much like this solution.
Urgh, that's horrible. I'll leave it to Ian to decide what's best.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-08-06 16:58 ` Stefano Stabellini
@ 2014-08-08 14:15 ` Julien Grall
2014-09-09 19:12 ` Julien Grall
1 sibling, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-08-08 14:15 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Ian Jackson, xen-devel, tim, ian.campbell, stefano.stabellini
On 08/06/2014 05:58 PM, Stefano Stabellini wrote:
> On Wed, 6 Aug 2014, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 08/06/2014 05:44 PM, Stefano Stabellini wrote:
>>> On Thu, 31 Jul 2014, Julien Grall wrote:
>>>> On ARM, every non-PCI device are described in the device tree. Each of them
>>>> can be found via a path.
>>>>
>>>> This path will be used to retrieved the different informations about the
>>>> device (compatible string, interrupts, MMIOs). Libxl will take care of:
>>>> - Allocate the MMIOs regions for the device in the guest
>>>> - Create the device node in the guest device tree
>>>> - Map the IRQs and MMIOs range in the guest P2M
>>>>
>>>> Note, that the device node won't contains specific properties for the node.
>>>> Only generic one (compatible, interrupts, regs) will be created by libxl.
>>>>
>>>> In the future, per-device properties will be added. Maybe via a configuration
>>>> file listing what is needed.
>>>>
>>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>>>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>>> Cc: Ian Campbell <ian.campbell@citrix.com>
>>>>
>>>> ---
>>>> Changes in v2:
>>>> - Get DT infos earlier
>>>> - Allocate/map IRQ in libxl__arch_domain_create rather than in
>>>> libxl__device_dt_add
>>>
>>> I understand why you moved it earlier but if it is not too hard I would
>>> suggest to try to keep mmio and irq mapping all together, because it
>>> would make the code much easier to read and modify in the future, being
>>> all in one place.
>>
>> I could fixup the device tree later. So we don't have to "map/retrieve"
>> the IRQ earlier.
>>
>> But I don't much like this solution.
>
> Urgh, that's horrible. I'll leave it to Ian to decide what's best.
FWIW, we already fix up the device later for the initrd node and the
memory banks.
Thinking a bit more to this solution and it won't be easily possible.
The device tree is finalized and copy the guest memory in
xc_dom_build_image which is called by libxl__build_pv.
So unless we assign platform device very early (i.e between
libxl_build_pre call and libxl__build_pv call in
libxl__domain_build_pv), we will have to map/get the interrupt very
early in a hackish way.
This is because we have to get the virtual IRQ to create the DT node for
the device.
I'm open to any other ideas.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-08-06 16:58 ` Stefano Stabellini
2014-08-08 14:15 ` Julien Grall
@ 2014-09-09 19:12 ` Julien Grall
2014-09-10 10:08 ` Ian Campbell
1 sibling, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-09 19:12 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Ian Jackson, xen-devel, tim, ian.campbell, stefano.stabellini
Hi Stefano,
On 06/08/14 09:58, Stefano Stabellini wrote:
> On Wed, 6 Aug 2014, Julien Grall wrote:
>> Hi Stefano,
>>
>> On 08/06/2014 05:44 PM, Stefano Stabellini wrote:
>>> On Thu, 31 Jul 2014, Julien Grall wrote:
>>>> On ARM, every non-PCI device are described in the device tree. Each of them
>>>> can be found via a path.
>>>>
>>>> This path will be used to retrieved the different informations about the
>>>> device (compatible string, interrupts, MMIOs). Libxl will take care of:
>>>> - Allocate the MMIOs regions for the device in the guest
>>>> - Create the device node in the guest device tree
>>>> - Map the IRQs and MMIOs range in the guest P2M
>>>>
>>>> Note, that the device node won't contains specific properties for the node.
>>>> Only generic one (compatible, interrupts, regs) will be created by libxl.
>>>>
>>>> In the future, per-device properties will be added. Maybe via a configuration
>>>> file listing what is needed.
>>>>
>>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>>>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>>> Cc: Ian Campbell <ian.campbell@citrix.com>
>>>>
>>>> ---
>>>> Changes in v2:
>>>> - Get DT infos earlier
>>>> - Allocate/map IRQ in libxl__arch_domain_create rather than in
>>>> libxl__device_dt_add
>>>
>>> I understand why you moved it earlier but if it is not too hard I would
>>> suggest to try to keep mmio and irq mapping all together, because it
>>> would make the code much easier to read and modify in the future, being
>>> all in one place.
>>
>> I could fixup the device tree later. So we don't have to "map/retrieve"
>> the IRQ earlier.
>>
>> But I don't much like this solution.
>
> Urgh, that's horrible. I'll leave it to Ian to decide what's best.
I think I found a way to avoid this hackish code. We could let the
toolstack decides of the VIRQ number.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 20/21] libxl: Add support for non-PCI passthrough
2014-09-09 19:12 ` Julien Grall
@ 2014-09-10 10:08 ` Ian Campbell
0 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-10 10:08 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, Ian Jackson, stefano.stabellini, Stefano Stabellini
On Tue, 2014-09-09 at 12:12 -0700, Julien Grall wrote:
> Hi Stefano,
>
> On 06/08/14 09:58, Stefano Stabellini wrote:
> > On Wed, 6 Aug 2014, Julien Grall wrote:
> >> Hi Stefano,
> >>
> >> On 08/06/2014 05:44 PM, Stefano Stabellini wrote:
> >>> On Thu, 31 Jul 2014, Julien Grall wrote:
> >>>> On ARM, every non-PCI device are described in the device tree. Each of them
> >>>> can be found via a path.
> >>>>
> >>>> This path will be used to retrieved the different informations about the
> >>>> device (compatible string, interrupts, MMIOs). Libxl will take care of:
> >>>> - Allocate the MMIOs regions for the device in the guest
> >>>> - Create the device node in the guest device tree
> >>>> - Map the IRQs and MMIOs range in the guest P2M
> >>>>
> >>>> Note, that the device node won't contains specific properties for the node.
> >>>> Only generic one (compatible, interrupts, regs) will be created by libxl.
> >>>>
> >>>> In the future, per-device properties will be added. Maybe via a configuration
> >>>> file listing what is needed.
> >>>>
> >>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> >>>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> >>>> Cc: Ian Campbell <ian.campbell@citrix.com>
> >>>>
> >>>> ---
> >>>> Changes in v2:
> >>>> - Get DT infos earlier
> >>>> - Allocate/map IRQ in libxl__arch_domain_create rather than in
> >>>> libxl__device_dt_add
> >>>
> >>> I understand why you moved it earlier but if it is not too hard I would
> >>> suggest to try to keep mmio and irq mapping all together, because it
> >>> would make the code much easier to read and modify in the future, being
> >>> all in one place.
> >>
> >> I could fixup the device tree later. So we don't have to "map/retrieve"
> >> the IRQ earlier.
> >>
> >> But I don't much like this solution.
> >
> > Urgh, that's horrible. I'll leave it to Ian to decide what's best.
>
> I think I found a way to avoid this hackish code. We could let the
> toolstack decides of the VIRQ number.
That would fit nicely with the toolstack picking nr_spis, and also helps
people who want a 1:1 mapping to achieve that.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* [PATCH v2 21/21] xl: Add new option dtdev
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (19 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 20/21] libxl: Add support for non-PCI passthrough Julien Grall
@ 2014-07-31 15:00 ` Julien Grall
2014-09-09 14:34 ` [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Ian Campbell
21 siblings, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-07-31 15:00 UTC (permalink / raw)
To: xen-devel
Cc: ian.campbell, Stefano Stabellini, tim, Julien Grall, Ian Jackson,
stefano.stabellini
The option "dtdev" will be used to passthrough a non-PCI device described
in the device tree to a guest.
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v2:
- libxl_device_dt has been rename to libxl_device_dtdev
- use xrealloc instead of realloc
---
docs/man/xl.cfg.pod.5 | 5 +++++
tools/libxl/xl_cmdimpl.c | 21 ++++++++++++++++++++-
2 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index 6d184d6..d78f5c4 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -623,6 +623,11 @@ More information about Xen gfx_passthru feature is available
on the XenVGAPassthrough L<http://wiki.xen.org/wiki/XenVGAPassthrough>
wiki page.
+=item B<dtdev=[ "DTDEV_PATH", "DTDEV_PATH", ... ]>
+
+Specifies the host device node to passthrough to this guest. Each DTDEV_PATH
+is the absolute path in the device tree.
+
=item B<ioports=[ "IOPORT_RANGE", "IOPORT_RANGE", ... ]>
Allow guest to access specific legacy I/O ports. Each B<IOPORT_RANGE>
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 6d9c901..545c2ff 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -775,7 +775,7 @@ static void parse_config_data(const char *config_source,
long l;
XLU_Config *config;
XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms;
- XLU_ConfigList *ioports, *irqs, *iomem;
+ XLU_ConfigList *ioports, *irqs, *iomem, *dtdevs;
int num_ioports, num_irqs, num_iomem, num_cpus;
int pci_power_mgmt = 0;
int pci_msitranslate = 0;
@@ -1483,6 +1483,25 @@ skip_vfb:
libxl_defbool_set(&b_info->u.pv.e820_host, true);
}
+ if (!xlu_cfg_get_list (config, "dtdev", &dtdevs, 0, 0)) {
+ d_config->num_dtdevs = 0;
+ d_config->dtdevs = NULL;
+ for (i = 0; (buf = xlu_cfg_get_listitem(dtdevs, i)) != NULL; i++) {
+ libxl_device_dtdev *dtdev;
+
+ d_config->dtdevs = (libxl_device_dtdev *) xrealloc(d_config->dtdevs, sizeof (libxl_device_dtdev) * (d_config->num_dtdevs + 1));
+ dtdev = d_config->dtdevs + d_config->num_dtdevs;
+ libxl_device_dtdev_init(dtdev);
+
+ dtdev->path = strdup(buf);
+ if (dtdev->path == NULL) {
+ fprintf(stderr, "unable to duplicate string for dtdevs\n");
+ exit(-1);
+ }
+ d_config->num_dtdevs++;
+ }
+ }
+
switch (xlu_cfg_get_list(config, "cpuid", &cpuids, 0, 1)) {
case 0:
{
--
1.7.10.4
^ permalink raw reply related [flat|nested] 119+ messages in thread* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-07-31 15:00 [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Julien Grall
` (20 preceding siblings ...)
2014-07-31 15:00 ` [PATCH v2 21/21] xl: Add new option dtdev Julien Grall
@ 2014-09-09 14:34 ` Ian Campbell
2014-09-09 19:34 ` Julien Grall
21 siblings, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-09 14:34 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, stefano.stabellini
On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
> - Only common device properties (interrupts, regs) are written to
> the guest device tree. Device that needs other properties may not work.
So I've glanced through the later (more toolstack oriented) bits from
towards the end but I think there's a question of the target users which
needs thinking about before I can have a sensible opinion on those.
As I see it the main purpose of this series is to get the underlying
plumbing in place (wiring up iommus, routing IRQs etc) to support guests
with passthrough devices, for embedded folks to use and to provide a
basis for eventual PCI passthrough functionality. I really want to see
this stuff in 4.5
What I'm concerned about is the toolstack side. TBH I'm not very keen on
the thing with exposing very DT specific stuff like compatible strings
down from the hypervisor via domctls.
It's not really clear how best to expose this functionality, I have a
feeling that this series either goes too far or not far enough and ends
up not really satisfying anyone.
My suspicion is that regular folks won't really be using passthrough
until it is via PCI and that in the meantime this functionality is only
going to be used by e.g. people building embedded system and superkeen
early adopters both of whom know what they are doing and can tolerate
some hacks etc to get things working (and I think that's fine, it's
still a worthwhile set of things to get into 4.5 and those folks are
worth supporting).
I'm also worried that we may be committing ourselves to a libxl API
already without really working through all the issues (e.g. other
properties).
Given that I wonder if we wouldn't be better off for 4.5 supporting
something much simpler at the toolstack level, namely allowing users to
use iomem= and irq= in their domain config to map platform devices
through (already works with your series today?) and perhaps a back door
to allow the injection a blob of DT into the guest's DT to describe
them. i.e. enough to actually get stuff done but not pretending to be
too finely integrated.
Then we can revisit the "proper" toolstack side for 4.6. Otherwise I
fear that by the time we get the toolstack side sorted out to our
satisfaction the basic functionality (which seems to be largely done)
will have missed 4.5.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-09 14:34 ` [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough Ian Campbell
@ 2014-09-09 19:34 ` Julien Grall
2014-09-10 9:22 ` Christoffer Dall
2014-09-10 10:11 ` Ian Campbell
0 siblings, 2 replies; 119+ messages in thread
From: Julien Grall @ 2014-09-09 19:34 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, tim, Christoffer Dall, stefano.stabellini
(Adding Christoffer)
Hi Ian,
On 09/09/14 07:34, Ian Campbell wrote:
> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
>> - Only common device properties (interrupts, regs) are written to
>> the guest device tree. Device that needs other properties may not work.
>
> So I've glanced through the later (more toolstack oriented) bits from
> towards the end but I think there's a question of the target users which
> needs thinking about before I can have a sensible opinion on those.
>
> As I see it the main purpose of this series is to get the underlying
> plumbing in place (wiring up iommus, routing IRQs etc) to support guests
> with passthrough devices, for embedded folks to use and to provide a
> basis for eventual PCI passthrough functionality. I really want to see
> this stuff in 4.5
>
> What I'm concerned about is the toolstack side. TBH I'm not very keen on
> the thing with exposing very DT specific stuff like compatible strings
> down from the hypervisor via domctls.
> It's not really clear how best to expose this functionality, I have a
> feeling that this series either goes too far or not far enough and ends
> up not really satisfying anyone.
I don't see many other solutions to get the compatible strings. There is
no easy way to get the property from DOM0, unless we introduce a new
driver in Linux.
> My suspicion is that regular folks won't really be using passthrough
> until it is via PCI and that in the meantime this functionality is only
> going to be used by e.g. people building embedded system and superkeen
> early adopters both of whom know what they are doing and can tolerate
> some hacks etc to get things working (and I think that's fine, it's
> still a worthwhile set of things to get into 4.5 and those folks are
> worth supporting).
>
> I'm also worried that we may be committing ourselves to a libxl API
> already without really working through all the issues (e.g. other
> properties).
>
> Given that I wonder if we wouldn't be better off for 4.5 supporting
> something much simpler at the toolstack level, namely allowing users to
> use iomem= and irq= in their domain config to map platform devices
> through (already works with your series today?)
This would need a bit a plumbing for irq part to allow the user choosing
the VIRQ (like Arianna did for MMIO range).
> and perhaps a back door
> to allow the injection a blob of DT into the guest's DT to describe
> them. i.e. enough to actually get stuff done but not pretending to be
> too finely integrated.
I would be fine with this solution for Xen 4.5. I will give a look to
see what could be done.
> Then we can revisit the "proper" toolstack side for 4.6. Otherwise I
> fear that by the time we get the toolstack side sorted out to our
> satisfaction the basic functionality (which seems to be largely done)
> will have missed 4.5.
Depending of the use cases, your solution based on "iomem", "irq" and DT
blob might be enough for device platform passthrough.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-09 19:34 ` Julien Grall
@ 2014-09-10 9:22 ` Christoffer Dall
2014-09-10 10:51 ` Ian Campbell
2014-09-10 10:11 ` Ian Campbell
1 sibling, 1 reply; 119+ messages in thread
From: Christoffer Dall @ 2014-09-10 9:22 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, Ian Campbell, Stefano Stabellini
On Tue, Sep 9, 2014 at 9:34 PM, Julien Grall <julien.grall@linaro.org> wrote:
> (Adding Christoffer)
>
> Hi Ian,
>
> On 09/09/14 07:34, Ian Campbell wrote:
>>
>> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
>>>
>>> - Only common device properties (interrupts, regs) are written to
>>> the guest device tree. Device that needs other properties may not
>>> work.
>>
>>
>> So I've glanced through the later (more toolstack oriented) bits from
>> towards the end but I think there's a question of the target users which
>> needs thinking about before I can have a sensible opinion on those.
>>
>> As I see it the main purpose of this series is to get the underlying
>> plumbing in place (wiring up iommus, routing IRQs etc) to support guests
>> with passthrough devices, for embedded folks to use and to provide a
>> basis for eventual PCI passthrough functionality. I really want to see
>> this stuff in 4.5
>>
>> What I'm concerned about is the toolstack side. TBH I'm not very keen on
>> the thing with exposing very DT specific stuff like compatible strings
>> down from the hypervisor via domctls.
>> It's not really clear how best to expose this functionality, I have a
>> feeling that this series either goes too far or not far enough and ends
>> up not really satisfying anyone.
>
>
> I don't see many other solutions to get the compatible strings. There is no
> easy way to get the property from DOM0, unless we introduce a new driver in
> Linux.
>
The toolstack you are using to create your guest must necessarily know
which guest it is creating, including device properties of a device a
user wishes to assign It can know this because it's hardcoded,
included in some config files, or supplied directly by the user. I
think you really want to decouple the hardware description method for
Dom0 from retrieving resource description about your device. Can't you
simply reference the device to Linux through its sysfs handle and use
your Xen-passthrough-layer-hypercall-magic (which I know nothing
about) to have Dom0 tell Xen to map/route the relevant resources?
What if your guest runs ACPI or something else different from your
hardware? How will this work when using ACPI for the hardware?
>> My suspicion is that regular folks won't really be using passthrough
>> until it is via PCI and that in the meantime this functionality is only
>> going to be used by e.g. people building embedded system and superkeen
>> early adopters both of whom know what they are doing and can tolerate
>> some hacks etc to get things working (and I think that's fine, it's
>> still a worthwhile set of things to get into 4.5 and those folks are
>> worth supporting).
>>
>> I'm also worried that we may be committing ourselves to a libxl API
>> already without really working through all the issues (e.g. other
>> properties).
>>
>> Given that I wonder if we wouldn't be better off for 4.5 supporting
>> something much simpler at the toolstack level, namely allowing users to
>> use iomem= and irq= in their domain config to map platform devices
>> through (already works with your series today?)
>
>
> This would need a bit a plumbing for irq part to allow the user choosing the
> VIRQ (like Arianna did for MMIO range).
>
My Xen knowledge is limited here. Is the iomem= and irq= commands
given to your Dom0 toolstack, Dom0, or Xen itself? How does a user
know which physical address in Xen's physical address space needs to
be remapped based on the hardware description language for Dom0?
It still seems to me that you need to abstract the simple concept of a
device passthrough vector (device handle, associated MMIO regions,
associated IRQs). That would extend more nicely to PCI/ACPI as well.
Am I missing something?
>> and perhaps a back door
>>
>> to allow the injection a blob of DT into the guest's DT to describe
>> them. i.e. enough to actually get stuff done but not pretending to be
>> too finely integrated.
>
can we not treat the problem of how to describe hardware to the guest
independently? For this matter, I still think you need some way to
retrieve the canonical information specific to your running instance
(mmio regions, irqs) and then you need to be able to create a DT/ACPI
tables as you want.
>
> I would be fine with this solution for Xen 4.5. I will give a look to see
> what could be done.
>
>> Then we can revisit the "proper" toolstack side for 4.6. Otherwise I
>> fear that by the time we get the toolstack side sorted out to our
>> satisfaction the basic functionality (which seems to be largely done)
>> will have missed 4.5.
>
>
> Depending of the use cases, your solution based on "iomem", "irq" and DT
> blob might be enough for device platform passthrough.
>
Hmmm, I don't really know what the Xen development policy is, but I am
generally not very much for creating a system solution on an OS level
that addresses a very limited use case without any sort of generic
abstractions...
-Christoffer
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 9:22 ` Christoffer Dall
@ 2014-09-10 10:51 ` Ian Campbell
2014-09-10 11:45 ` Christoffer Dall
2014-09-12 19:16 ` Julien Grall
0 siblings, 2 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-10 10:51 UTC (permalink / raw)
To: Christoffer Dall; +Cc: xen-devel, Julien Grall, tim, Stefano Stabellini
On Wed, 2014-09-10 at 11:22 +0200, Christoffer Dall wrote:
> On Tue, Sep 9, 2014 at 9:34 PM, Julien Grall <julien.grall@linaro.org> wrote:
> > (Adding Christoffer)
> >
> > Hi Ian,
> >
> > On 09/09/14 07:34, Ian Campbell wrote:
> >>
> >> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
> >>>
> >>> - Only common device properties (interrupts, regs) are written to
> >>> the guest device tree. Device that needs other properties may not
> >>> work.
> >>
> >>
> >> So I've glanced through the later (more toolstack oriented) bits from
> >> towards the end but I think there's a question of the target users which
> >> needs thinking about before I can have a sensible opinion on those.
> >>
> >> As I see it the main purpose of this series is to get the underlying
> >> plumbing in place (wiring up iommus, routing IRQs etc) to support guests
> >> with passthrough devices, for embedded folks to use and to provide a
> >> basis for eventual PCI passthrough functionality. I really want to see
> >> this stuff in 4.5
> >>
> >> What I'm concerned about is the toolstack side. TBH I'm not very keen on
> >> the thing with exposing very DT specific stuff like compatible strings
> >> down from the hypervisor via domctls.
> >> It's not really clear how best to expose this functionality, I have a
> >> feeling that this series either goes too far or not far enough and ends
> >> up not really satisfying anyone.
> >
> >
> > I don't see many other solutions to get the compatible strings. There is no
> > easy way to get the property from DOM0, unless we introduce a new driver in
> > Linux.
> >
>
> The toolstack you are using to create your guest must necessarily know
> which guest it is creating, including device properties of a device a
> user wishes to assign It can know this because it's hardcoded,
> included in some config files, or supplied directly by the user. I
> think you really want to decouple the hardware description method for
> Dom0 from retrieving resource description about your device. Can't you
> simply reference the device to Linux through its sysfs handle and use
> your Xen-passthrough-layer-hypercall-magic (which I know nothing
> about) to have Dom0 tell Xen to map/route the relevant resources?
By Xen-passthrough-layer-hypercall-magic do you mean the thing which
lets the userspace toolstack make hypercalls (which is called "privcmd"
FWIW) or are you talking about some specific passtrhough related kernel
driver (like VFIO? which has no Xen equivalent right now)
If you mean the former then I think Julien's code already does this --
it makes hypercalls telling Xen to map certain MMIO regions to guests.
What's in question is where the inputs to those hypercalls came from.
> >> My suspicion is that regular folks won't really be using passthrough
> >> until it is via PCI and that in the meantime this functionality is only
> >> going to be used by e.g. people building embedded system and superkeen
> >> early adopters both of whom know what they are doing and can tolerate
> >> some hacks etc to get things working (and I think that's fine, it's
> >> still a worthwhile set of things to get into 4.5 and those folks are
> >> worth supporting).
> >>
> >> I'm also worried that we may be committing ourselves to a libxl API
> >> already without really working through all the issues (e.g. other
> >> properties).
> >>
> >> Given that I wonder if we wouldn't be better off for 4.5 supporting
> >> something much simpler at the toolstack level, namely allowing users to
> >> use iomem= and irq= in their domain config to map platform devices
> >> through (already works with your series today?)
> >
> >
> > This would need a bit a plumbing for irq part to allow the user choosing the
> > VIRQ (like Arianna did for MMIO range).
> >
>
> My Xen knowledge is limited here. Is the iomem= and irq= commands
> given to your Dom0 toolstack, Dom0, or Xen itself?
They are things that the user can write into the guest cfg file,
containing lists of the relevant resources. e.g. to pass IRQ 42 to the
guest: irqs = [ 42 ]
The toolstack parses that and makes hypercalls while building a guest to
tell Xen to map those regions through.
> How does a user
> know which physical address in Xen's physical address space needs to
> be remapped based on the hardware description language for Dom0?
This is an interesting question. The short answer for platform device
type things is "they just know" (they presumably have datasheets etc).
But I think the underlying question here is whether they are given in PA
space or dom0 IPA space, right?
The way that this works on x86 is that dom0 sees the real underlying
MMIO addresses in e.g. PCI BARs and /proc/iomem etc. So the hypercalls
to map MMIO regions to guests all take real physical addresses and
nothing has to worry about IPA vs PA issues.
On ARM things are potentially a bit more complex because dom0 is running
with second stage paging. However we always map the MMIO regions 1:1 for
dom0, and I think we will always have to do that (the "1:1 workaround"
refers to RAM regions only). So I think we can continue to treat these
things as proper physical addresses and don't need to introduce variants
of the hypercalls which work in terms of IPAs (or you could argue that
they already do and the translation is currently a nop).
> It still seems to me that you need to abstract the simple concept of a
> device passthrough vector (device handle, associated MMIO regions,
> associated IRQs). That would extend more nicely to PCI/ACPI as well.
> Am I missing something?
I don't think so.
For PCI the toolstack would read sysfs to get at the BAR info which
tells us the real physical addresses and it would then program those
using the existing mechanisms.
> >> and perhaps a back door
> >>
> >> to allow the injection a blob of DT into the guest's DT to describe
> >> them. i.e. enough to actually get stuff done but not pretending to be
> >> too finely integrated.
> >
>
> can we not treat the problem of how to describe hardware to the guest
> independently?
Yes, that's essentially what I am proposing, get the basics working via
iomem= and irq= now and leave the tricky problem of describing platform
hardware until later.
> For this matter, I still think you need some way to
> retrieve the canonical information specific to your running instance
> (mmio regions, irqs) and then you need to be able to create a DT/ACPI
> tables as you want.
I think might be a future nice to have but m specifically advocating
that we not do it for 4.5. For 4.5 I think it will be acceptable to give
the user the parts which they need to put something together, which is:
iomem=/irq= (existing Xen interfaces), the dtc/iasl compiler and some
way to add a predefined blob to the guests generated firmware tables.
Trying to invent up an interface like you suggest simply isn't going to
happen in time for 4.5 (the freeze of which was going to be today but
has been slipped by a couple of weeks). And regardless of that at least
iomem= and irq= are things which we should support anyway, since are
existing Xen interfaces.
> > I would be fine with this solution for Xen 4.5. I will give a look to see
> > what could be done.
> >
> >> Then we can revisit the "proper" toolstack side for 4.6. Otherwise I
> >> fear that by the time we get the toolstack side sorted out to our
> >> satisfaction the basic functionality (which seems to be largely done)
> >> will have missed 4.5.
> >
> >
> > Depending of the use cases, your solution based on "iomem", "irq" and DT
> > blob might be enough for device platform passthrough.
> >
>
> Hmmm, I don't really know what the Xen development policy is, but I am
> generally not very much for creating a system solution on an OS level
> that addresses a very limited use case without any sort of generic
> abstractions...
In the case of passthrough we prove both low level ways of configuring
things and higher level abstractions built on top of them. e.g.
iomem=/irq= vs pci= in a guest cfg file. iomem=/irq= let you assign
arbitrary platform resources (subject to permissions etc) to a guest and
in theory you could assign a PCI device manually using them. pci= lets
you just say "assign this device please" and the toolstack sorts out the
rest, in reality this is the one people use for PCI passthrough (for the
obvious reason). On x86 iomme=/irq= get used for things like passing
through serial ports to guests.
The problem we are having here is that we have no existing higher level
abstraction for platform devices (because on x86 everyone just knows the
address of the std serial ports ;-)). We can consider adding one in the
future, but we should still be providing iomem=/irq= anyway and that is
something which can realistically be done for 4.5.
So I think we should leave the question of what a higher level
abstraction for passing through platform devices aside for now since it
is a distraction from getting something useful in for 4.5, which is on a
very tight timescale right now.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 10:51 ` Ian Campbell
@ 2014-09-10 11:45 ` Christoffer Dall
2014-09-10 12:05 ` Ian Campbell
2014-09-10 22:03 ` Stefano Stabellini
2014-09-12 19:16 ` Julien Grall
1 sibling, 2 replies; 119+ messages in thread
From: Christoffer Dall @ 2014-09-10 11:45 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, Julien Grall, tim, Stefano Stabellini
On Wed, Sep 10, 2014 at 12:51 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2014-09-10 at 11:22 +0200, Christoffer Dall wrote:
>> On Tue, Sep 9, 2014 at 9:34 PM, Julien Grall <julien.grall@linaro.org> wrote:
>> > (Adding Christoffer)
>> >
>> > Hi Ian,
>> >
>> > On 09/09/14 07:34, Ian Campbell wrote:
>> >>
>> >> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
>> >>>
>> >>> - Only common device properties (interrupts, regs) are written to
>> >>> the guest device tree. Device that needs other properties may not
>> >>> work.
>> >>
>> >>
>> >> So I've glanced through the later (more toolstack oriented) bits from
>> >> towards the end but I think there's a question of the target users which
>> >> needs thinking about before I can have a sensible opinion on those.
>> >>
>> >> As I see it the main purpose of this series is to get the underlying
>> >> plumbing in place (wiring up iommus, routing IRQs etc) to support guests
>> >> with passthrough devices, for embedded folks to use and to provide a
>> >> basis for eventual PCI passthrough functionality. I really want to see
>> >> this stuff in 4.5
>> >>
>> >> What I'm concerned about is the toolstack side. TBH I'm not very keen on
>> >> the thing with exposing very DT specific stuff like compatible strings
>> >> down from the hypervisor via domctls.
>> >> It's not really clear how best to expose this functionality, I have a
>> >> feeling that this series either goes too far or not far enough and ends
>> >> up not really satisfying anyone.
>> >
>> >
>> > I don't see many other solutions to get the compatible strings. There is no
>> > easy way to get the property from DOM0, unless we introduce a new driver in
>> > Linux.
>> >
>>
>> The toolstack you are using to create your guest must necessarily know
>> which guest it is creating, including device properties of a device a
>> user wishes to assign It can know this because it's hardcoded,
>> included in some config files, or supplied directly by the user. I
>> think you really want to decouple the hardware description method for
>> Dom0 from retrieving resource description about your device. Can't you
>> simply reference the device to Linux through its sysfs handle and use
>> your Xen-passthrough-layer-hypercall-magic (which I know nothing
>> about) to have Dom0 tell Xen to map/route the relevant resources?
>
> By Xen-passthrough-layer-hypercall-magic do you mean the thing which
> lets the userspace toolstack make hypercalls (which is called "privcmd"
> FWIW) or are you talking about some specific passtrhough related kernel
> driver (like VFIO? which has no Xen equivalent right now)
I was talking about the latter since I assumed the Dom0 address space
and irq number space was completely decoupled from the hardware, but I
understand now that it is not, which is good, because otherwise I'm
thinking we would be pretty screwed wrt. addresses described in the
DSDT in ACPI.
>
> If you mean the former then I think Julien's code already does this --
> it makes hypercalls telling Xen to map certain MMIO regions to guests.
I meant the latter
> What's in question is where the inputs to those hypercalls came from.
>
and how the inputs look like, but we address this later in the mail.
>> >> My suspicion is that regular folks won't really be using passthrough
>> >> until it is via PCI and that in the meantime this functionality is only
>> >> going to be used by e.g. people building embedded system and superkeen
>> >> early adopters both of whom know what they are doing and can tolerate
>> >> some hacks etc to get things working (and I think that's fine, it's
>> >> still a worthwhile set of things to get into 4.5 and those folks are
>> >> worth supporting).
>> >>
>> >> I'm also worried that we may be committing ourselves to a libxl API
>> >> already without really working through all the issues (e.g. other
>> >> properties).
>> >>
>> >> Given that I wonder if we wouldn't be better off for 4.5 supporting
>> >> something much simpler at the toolstack level, namely allowing users to
>> >> use iomem= and irq= in their domain config to map platform devices
>> >> through (already works with your series today?)
>> >
>> >
>> > This would need a bit a plumbing for irq part to allow the user choosing the
>> > VIRQ (like Arianna did for MMIO range).
>> >
>>
>> My Xen knowledge is limited here. Is the iomem= and irq= commands
>> given to your Dom0 toolstack, Dom0, or Xen itself?
>
> They are things that the user can write into the guest cfg file,
> containing lists of the relevant resources. e.g. to pass IRQ 42 to the
> guest: irqs = [ 42 ]
>
> The toolstack parses that and makes hypercalls while building a guest to
> tell Xen to map those regions through.
>
ok, so the irq numbers etc. are passed as-is to Xen and better
represent the same hardware resources in Dom0 as in Xen, then.
>> How does a user
>> know which physical address in Xen's physical address space needs to
>> be remapped based on the hardware description language for Dom0?
>
> This is an interesting question. The short answer for platform device
> type things is "they just know" (they presumably have datasheets etc).
> But I think the underlying question here is whether they are given in PA
> space or dom0 IPA space, right?
>
re. the underlying question, yes.
For platform devices, I think the "they just know" doesn't really
work. There must be a way for user space to determine the IO
addresses and IRQ numbers for a platform device, I just think it
should be completely decoupled from ACPI and DT.
> The way that this works on x86 is that dom0 sees the real underlying
> MMIO addresses in e.g. PCI BARs and /proc/iomem etc. So the hypercalls
> to map MMIO regions to guests all take real physical addresses and
> nothing has to worry about IPA vs PA issues.
>
> On ARM things are potentially a bit more complex because dom0 is running
> with second stage paging. However we always map the MMIO regions 1:1 for
> dom0, and I think we will always have to do that (the "1:1 workaround"
> refers to RAM regions only).
yeah, see my ACPI comment above.
> So I think we can continue to treat these
> things as proper physical addresses and don't need to introduce variants
> of the hypercalls which work in terms of IPAs (or you could argue that
> they already do and the translation is currently a nop).
fair enough, I guess for passthrough it's valid to always specify
things in Dom0's address/number space, because Xen already knows about
the translations to do the right thing....
>
>> It still seems to me that you need to abstract the simple concept of a
>> device passthrough vector (device handle, associated MMIO regions,
>> associated IRQs). That would extend more nicely to PCI/ACPI as well.
>> Am I missing something?
>
> I don't think so.
>
> For PCI the toolstack would read sysfs to get at the BAR info which
> tells us the real physical addresses and it would then program those
> using the existing mechanisms.
>
>> >> and perhaps a back door
>> >>
>> >> to allow the injection a blob of DT into the guest's DT to describe
>> >> them. i.e. enough to actually get stuff done but not pretending to be
>> >> too finely integrated.
>> >
>>
>> can we not treat the problem of how to describe hardware to the guest
>> independently?
>
> Yes, that's essentially what I am proposing, get the basics working via
> iomem= and irq= now and leave the tricky problem of describing platform
> hardware until later.
>
sounds reasonable.
>> For this matter, I still think you need some way to
>> retrieve the canonical information specific to your running instance
>> (mmio regions, irqs) and then you need to be able to create a DT/ACPI
>> tables as you want.
>
> I think might be a future nice to have but m specifically advocating
> that we not do it for 4.5. For 4.5 I think it will be acceptable to give
> the user the parts which they need to put something together, which is:
> iomem=/irq= (existing Xen interfaces), the dtc/iasl compiler and some
> way to add a predefined blob to the guests generated firmware tables.
>
> Trying to invent up an interface like you suggest simply isn't going to
> happen in time for 4.5 (the freeze of which was going to be today but
> has been slipped by a couple of weeks). And regardless of that at least
> iomem= and irq= are things which we should support anyway, since are
> existing Xen interfaces.
>
I don't keep track of the Xen development cycle (sorry, maybe I
should), but what you're saying sounds completely reasonable to me,
given that iomem= and irq= need to be supported anyhow.
>> > I would be fine with this solution for Xen 4.5. I will give a look to see
>> > what could be done.
>> >
>> >> Then we can revisit the "proper" toolstack side for 4.6. Otherwise I
>> >> fear that by the time we get the toolstack side sorted out to our
>> >> satisfaction the basic functionality (which seems to be largely done)
>> >> will have missed 4.5.
>> >
>> >
>> > Depending of the use cases, your solution based on "iomem", "irq" and DT
>> > blob might be enough for device platform passthrough.
>> >
>>
>> Hmmm, I don't really know what the Xen development policy is, but I am
>> generally not very much for creating a system solution on an OS level
>> that addresses a very limited use case without any sort of generic
>> abstractions...
>
> In the case of passthrough we prove both low level ways of configuring
> things and higher level abstractions built on top of them. e.g.
> iomem=/irq= vs pci= in a guest cfg file. iomem=/irq= let you assign
> arbitrary platform resources (subject to permissions etc) to a guest and
> in theory you could assign a PCI device manually using them. pci= lets
> you just say "assign this device please" and the toolstack sorts out the
> rest, in reality this is the one people use for PCI passthrough (for the
> obvious reason). On x86 iomme=/irq= get used for things like passing
> through serial ports to guests.
>
> The problem we are having here is that we have no existing higher level
> abstraction for platform devices (because on x86 everyone just knows the
> address of the std serial ports ;-)). We can consider adding one in the
> future, but we should still be providing iomem=/irq= anyway and that is
> something which can realistically be done for 4.5.
>
> So I think we should leave the question of what a higher level
> abstraction for passing through platform devices aside for now since it
> is a distraction from getting something useful in for 4.5, which is on a
> very tight timescale right now.
>
Understood, thanks.
-Christoffer
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 11:45 ` Christoffer Dall
@ 2014-09-10 12:05 ` Ian Campbell
2014-09-10 22:03 ` Stefano Stabellini
1 sibling, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-10 12:05 UTC (permalink / raw)
To: Christoffer Dall; +Cc: xen-devel, Julien Grall, tim, Stefano Stabellini
On Wed, 2014-09-10 at 13:45 +0200, Christoffer Dall wrote:
> On Wed, Sep 10, 2014 at 12:51 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Wed, 2014-09-10 at 11:22 +0200, Christoffer Dall wrote:
> >> How does a user
> >> know which physical address in Xen's physical address space needs to
> >> be remapped based on the hardware description language for Dom0?
> >
> > This is an interesting question. The short answer for platform device
> > type things is "they just know" (they presumably have datasheets etc).
> > But I think the underlying question here is whether they are given in PA
> > space or dom0 IPA space, right?
> >
>
> re. the underlying question, yes.
>
> For platform devices, I think the "they just know" doesn't really
> work.
It does "work" in the sense that it is usable to achieve a user's aims.
It may not "work" in the sense that it is not as convenient to use as it
could be.
> There must be a way for user space to determine the IO
> addresses and IRQ numbers for a platform device, I just think it
> should be completely decoupled from ACPI and DT.
You are right that it would be good to have something better, but that
is not going to happen for 4.5.
I think you got my point by the end of the mail so I won't belabour it
again ;-)
> > So I think we can continue to treat these
> > things as proper physical addresses and don't need to introduce variants
> > of the hypercalls which work in terms of IPAs (or you could argue that
> > they already do and the translation is currently a nop).
>
> fair enough, I guess for passthrough it's valid to always specify
> things in Dom0's address/number space, because Xen already knows about
> the translations to do the right thing....
Yes, I think if we ever took away 1:1 MMIO mappings from dom0 we would
probably end up retconning things to have always been in dom0 IPA
addressing and fixup the fallout from that in the hypervisor as part of
that shift.
That last paragraph does make me question 05/21 "xen/arm: follow-up to
allow DOM0 manage IRQ and MMIO" which removes the dom0 mapping of
devices which have been "marked for passthrough" though, since it would
break this model.
I think it should be sufficient to mark the devices as disabled to dom0
in the DT (or the ACPI equivalent) but still MMIO map them in the normal
way. This is consistent with e.g. pciback.hide= on x86 and avoids
special casing these devices wrt having perms but not mappings. I'm not
sure what the argument was for removing the MMIO mapping from dom0 (it
seemed to make sense in isolation when I read the patch, but I'm now
wondering about it...).
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 11:45 ` Christoffer Dall
2014-09-10 12:05 ` Ian Campbell
@ 2014-09-10 22:03 ` Stefano Stabellini
2014-09-11 4:03 ` Christoffer Dall
1 sibling, 1 reply; 119+ messages in thread
From: Stefano Stabellini @ 2014-09-10 22:03 UTC (permalink / raw)
To: Christoffer Dall
Cc: xen-devel, Julien Grall, tim, Ian Campbell, Stefano Stabellini
On Wed, 10 Sep 2014, Christoffer Dall wrote:
> >> How does a user
> >> know which physical address in Xen's physical address space needs to
> >> be remapped based on the hardware description language for Dom0?
> >
> > This is an interesting question. The short answer for platform device
> > type things is "they just know" (they presumably have datasheets etc).
> > But I think the underlying question here is whether they are given in PA
> > space or dom0 IPA space, right?
> >
>
> re. the underlying question, yes.
>
> For platform devices, I think the "they just know" doesn't really
> work.
"Who" should "just know"? We are talking about users, specifically
embedded engineers trying to put together their systems. Certainly not a
server use case. A server sysadmin cannot be expected to "just know".
If we wanted to export this feature to end users we would have to
provide a simpler way to do it.
> There must be a way for user space to determine the IO
> addresses and IRQ numbers for a platform device, I just think it
> should be completely decoupled from ACPI and DT.
I don't follow you here. How would a user find the info she needs on a
device without using an ACPI/DT identifier to reference it?
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 22:03 ` Stefano Stabellini
@ 2014-09-11 4:03 ` Christoffer Dall
2014-09-11 8:56 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Christoffer Dall @ 2014-09-11 4:03 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel, Julien Grall, tim, Ian Campbell, Stefano Stabellini
On Wed, Sep 10, 2014 at 3:03 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Wed, 10 Sep 2014, Christoffer Dall wrote:
>> >> How does a user
>> >> know which physical address in Xen's physical address space needs to
>> >> be remapped based on the hardware description language for Dom0?
>> >
>> > This is an interesting question. The short answer for platform device
>> > type things is "they just know" (they presumably have datasheets etc).
>> > But I think the underlying question here is whether they are given in PA
>> > space or dom0 IPA space, right?
>> >
>>
>> re. the underlying question, yes.
>>
>> For platform devices, I think the "they just know" doesn't really
>> work.
>
> "Who" should "just know"?
Ian was arguing that that the Xen toolstack should just know. To me
it sounds like a very unflexible way of doing things.
> We are talking about users, specifically
> embedded engineers trying to put together their systems. Certainly not a
> server use case. A server sysadmin cannot be expected to "just know".
> If we wanted to export this feature to end users we would have to
> provide a simpler way to do it.
>
All I am saying is that hardcoding hardware aspects about a specific
platform in the toolstack sounds strange, especially if it's a matter
of being able to probe for an IO region and an IRQ number.
>
>> There must be a way for user space to determine the IO
>> addresses and IRQ numbers for a platform device, I just think it
>> should be completely decoupled from ACPI and DT.
>
> I don't follow you here. How would a user find the info she needs on a
> device without using an ACPI/DT identifier to reference it?
Your kernel would take care of that. But you shouldn't be exposing
the raw hardware description to user space. My believe is that the
notion of being able to copy a portion of a DTB from the host and
directly insert it into the guest is fundamentally flawed.
But we're sort of discussing at two different axis here, Ian already
suggested that we take the simple approach for the upcoming release,
and we can talk more about generic solutions at LCU14, assuming you're
all going to be there.
-Christoffer
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-11 4:03 ` Christoffer Dall
@ 2014-09-11 8:56 ` Ian Campbell
0 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-11 8:56 UTC (permalink / raw)
To: Christoffer Dall
Cc: xen-devel, Julien Grall, tim, Stefano Stabellini, Stefano Stabellini
On Wed, 2014-09-10 at 21:03 -0700, Christoffer Dall wrote:
> On Wed, Sep 10, 2014 at 3:03 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Wed, 10 Sep 2014, Christoffer Dall wrote:
> >> >> How does a user
> >> >> know which physical address in Xen's physical address space needs to
> >> >> be remapped based on the hardware description language for Dom0?
> >> >
> >> > This is an interesting question. The short answer for platform device
> >> > type things is "they just know" (they presumably have datasheets etc).
> >> > But I think the underlying question here is whether they are given in PA
> >> > space or dom0 IPA space, right?
> >> >
> >>
> >> re. the underlying question, yes.
> >>
> >> For platform devices, I think the "they just know" doesn't really
> >> work.
> >
> > "Who" should "just know"?
>
> Ian was arguing that that the Xen toolstack should just know.
No I'm not, I'm arguing that for 4.5 the toolstack should be told by the
user. I'm certainly not arguing that we should never have a better way.
> To me
> it sounds like a very unflexible way of doing things.
>
> > We are talking about users, specifically
> > embedded engineers trying to put together their systems. Certainly not a
> > server use case. A server sysadmin cannot be expected to "just know".
> > If we wanted to export this feature to end users we would have to
> > provide a simpler way to do it.
> >
>
> All I am saying is that hardcoding hardware aspects about a specific
> platform in the toolstack sounds strange, especially if it's a matter
> of being able to probe for an IO region and an IRQ number.
No one is talking about hardcoding anything in the toolstack. When using
the lowlevel approach these things are written in the guest cfg file,
next to the kernel selection, number of vcpus etc etc.
> >> There must be a way for user space to determine the IO
> >> addresses and IRQ numbers for a platform device, I just think it
> >> should be completely decoupled from ACPI and DT.
> >
> > I don't follow you here. How would a user find the info she needs on a
> > device without using an ACPI/DT identifier to reference it?
For the server usecase almost everything of interest for passthrough
will be PCI, for which we already have a usable higher level
abstraction.
>
> Your kernel would take care of that. But you shouldn't be exposing
> the raw hardware description to user space. My believe is that the
> notion of being able to copy a portion of a DTB from the host and
> directly insert it into the guest is fundamentally flawed.
>
> But we're sort of discussing at two different axis here, Ian already
> suggested that we take the simple approach for the upcoming release,
> and we can talk more about generic solutions at LCU14, assuming you're
> all going to be there.
Yes.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 10:51 ` Ian Campbell
2014-09-10 11:45 ` Christoffer Dall
@ 2014-09-12 19:16 ` Julien Grall
1 sibling, 0 replies; 119+ messages in thread
From: Julien Grall @ 2014-09-12 19:16 UTC (permalink / raw)
To: Ian Campbell, Christoffer Dall; +Cc: xen-devel, tim, Stefano Stabellini
Hi Ian,
On 10/09/14 03:51, Ian Campbell wrote:
> On Wed, 2014-09-10 at 11:22 +0200, Christoffer Dall wrote:
>> On Tue, Sep 9, 2014 at 9:34 PM, Julien Grall <julien.grall@linaro.org> wrote:
>>> (Adding Christoffer)
>>>
>>> Hi Ian,
>>>
>>> On 09/09/14 07:34, Ian Campbell wrote:
>>>>
>>>> On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
>>>>>
>>>>> - Only common device properties (interrupts, regs) are written to
>>>>> the guest device tree. Device that needs other properties may not
>>>>> work.
>>>>
>>>>
>>>> So I've glanced through the later (more toolstack oriented) bits from
>>>> towards the end but I think there's a question of the target users which
>>>> needs thinking about before I can have a sensible opinion on those.
>>>>
>>>> As I see it the main purpose of this series is to get the underlying
>>>> plumbing in place (wiring up iommus, routing IRQs etc) to support guests
>>>> with passthrough devices, for embedded folks to use and to provide a
>>>> basis for eventual PCI passthrough functionality. I really want to see
>>>> this stuff in 4.5
>>>>
>>>> What I'm concerned about is the toolstack side. TBH I'm not very keen on
>>>> the thing with exposing very DT specific stuff like compatible strings
>>>> down from the hypervisor via domctls.
>>>> It's not really clear how best to expose this functionality, I have a
>>>> feeling that this series either goes too far or not far enough and ends
>>>> up not really satisfying anyone.
>>>
>>>
>>> I don't see many other solutions to get the compatible strings. There is no
>>> easy way to get the property from DOM0, unless we introduce a new driver in
>>> Linux.
>>>
>>
>> The toolstack you are using to create your guest must necessarily know
>> which guest it is creating, including device properties of a device a
>> user wishes to assign It can know this because it's hardcoded,
>> included in some config files, or supplied directly by the user. I
>> think you really want to decouple the hardware description method for
>> Dom0 from retrieving resource description about your device. Can't you
>> simply reference the device to Linux through its sysfs handle and use
>> your Xen-passthrough-layer-hypercall-magic (which I know nothing
>> about) to have Dom0 tell Xen to map/route the relevant resources?
>
> By Xen-passthrough-layer-hypercall-magic do you mean the thing which
> lets the userspace toolstack make hypercalls (which is called "privcmd"
> FWIW) or are you talking about some specific passtrhough related kernel
> driver (like VFIO? which has no Xen equivalent right now)
>
> If you mean the former then I think Julien's code already does this --
> it makes hypercalls telling Xen to map certain MMIO regions to guests.
> What's in question is where the inputs to those hypercalls came from.
>
>>>> My suspicion is that regular folks won't really be using passthrough
>>>> until it is via PCI and that in the meantime this functionality is only
>>>> going to be used by e.g. people building embedded system and superkeen
>>>> early adopters both of whom know what they are doing and can tolerate
>>>> some hacks etc to get things working (and I think that's fine, it's
>>>> still a worthwhile set of things to get into 4.5 and those folks are
>>>> worth supporting).
>>>>
>>>> I'm also worried that we may be committing ourselves to a libxl API
>>>> already without really working through all the issues (e.g. other
>>>> properties).
>>>>
>>>> Given that I wonder if we wouldn't be better off for 4.5 supporting
>>>> something much simpler at the toolstack level, namely allowing users to
>>>> use iomem= and irq= in their domain config to map platform devices
>>>> through (already works with your series today?)
>>>
>>>
>>> This would need a bit a plumbing for irq part to allow the user choosing the
>>> VIRQ (like Arianna did for MMIO range).
>>>
>>
>> My Xen knowledge is limited here. Is the iomem= and irq= commands
>> given to your Dom0 toolstack, Dom0, or Xen itself?
>
> They are things that the user can write into the guest cfg file,
> containing lists of the relevant resources. e.g. to pass IRQ 42 to the
> guest: irqs = [ 42 ]
>
> The toolstack parses that and makes hypercalls while building a guest to
> tell Xen to map those regions through.
>
>> How does a user
>> know which physical address in Xen's physical address space needs to
>> be remapped based on the hardware description language for Dom0?
>
> This is an interesting question. The short answer for platform device
> type things is "they just know" (they presumably have datasheets etc).
> But I think the underlying question here is whether they are given in PA
> space or dom0 IPA space, right?
>
> The way that this works on x86 is that dom0 sees the real underlying
> MMIO addresses in e.g. PCI BARs and /proc/iomem etc. So the hypercalls
> to map MMIO regions to guests all take real physical addresses and
> nothing has to worry about IPA vs PA issues.
>
> On ARM things are potentially a bit more complex because dom0 is running
> with second stage paging. However we always map the MMIO regions 1:1 for
> dom0, and I think we will always have to do that (the "1:1 workaround"
> refers to RAM regions only). So I think we can continue to treat these
> things as proper physical addresses and don't need to introduce variants
> of the hypercalls which work in terms of IPAs (or you could argue that
> they already do and the translation is currently a nop).
The MMIO regions for platform device to passthrough are not mapped into
DOM0 memory. Although, the device is still described in the device tree.
I'm not sure if it's useful for map the MMIO in DOM0 for a simple
passthrough as for now we don't have any "platform-back" drivers in DOM0
that will reset the device.
Maybe we should stay consistent with the IRQ assignment, as we can't
assign to DOM0 for now (this is because there is not driver to unmap them).
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-09 19:34 ` Julien Grall
2014-09-10 9:22 ` Christoffer Dall
@ 2014-09-10 10:11 ` Ian Campbell
2014-09-10 18:45 ` Julien Grall
1 sibling, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-10 10:11 UTC (permalink / raw)
To: Julien Grall; +Cc: xen-devel, tim, Christoffer Dall, stefano.stabellini
On Tue, 2014-09-09 at 12:34 -0700, Julien Grall wrote:
> (Adding Christoffer)
>
> Hi Ian,
>
> On 09/09/14 07:34, Ian Campbell wrote:
> > On Thu, 2014-07-31 at 16:00 +0100, Julien Grall wrote:
> >> - Only common device properties (interrupts, regs) are written to
> >> the guest device tree. Device that needs other properties may not work.
> >
> > So I've glanced through the later (more toolstack oriented) bits from
> > towards the end but I think there's a question of the target users which
> > needs thinking about before I can have a sensible opinion on those.
> >
> > As I see it the main purpose of this series is to get the underlying
> > plumbing in place (wiring up iommus, routing IRQs etc) to support guests
> > with passthrough devices, for embedded folks to use and to provide a
> > basis for eventual PCI passthrough functionality. I really want to see
> > this stuff in 4.5
> >
> > What I'm concerned about is the toolstack side. TBH I'm not very keen on
> > the thing with exposing very DT specific stuff like compatible strings
> > down from the hypervisor via domctls.
> > It's not really clear how best to expose this functionality, I have a
> > feeling that this series either goes too far or not far enough and ends
> > up not really satisfying anyone.
>
> I don't see many other solutions to get the compatible strings. There is
> no easy way to get the property from DOM0, unless we introduce a new
> driver in Linux.
Right, they would have to come from the user (in my proposal via the DT
blob injection).
> > My suspicion is that regular folks won't really be using passthrough
> > until it is via PCI and that in the meantime this functionality is only
> > going to be used by e.g. people building embedded system and superkeen
> > early adopters both of whom know what they are doing and can tolerate
> > some hacks etc to get things working (and I think that's fine, it's
> > still a worthwhile set of things to get into 4.5 and those folks are
> > worth supporting).
> >
> > I'm also worried that we may be committing ourselves to a libxl API
> > already without really working through all the issues (e.g. other
> > properties).
> >
> > Given that I wonder if we wouldn't be better off for 4.5 supporting
> > something much simpler at the toolstack level, namely allowing users to
> > use iomem= and irq= in their domain config to map platform devices
> > through (already works with your series today?)
>
> This would need a bit a plumbing for irq part to allow the user choosing
> the VIRQ (like Arianna did for MMIO range).
Is it required, or can we just number them from IRQ32 onwards?
Adding the ability to specify would be a nice to have, but not essential
right now IMHO, although if it falls out nicely from the changes you are
already planning to make then great.
>
> > and perhaps a back door
> > to allow the injection a blob of DT into the guest's DT to describe
> > them. i.e. enough to actually get stuff done but not pretending to be
> > too finely integrated.
>
> I would be fine with this solution for Xen 4.5. I will give a look to
> see what could be done.
>
> > Then we can revisit the "proper" toolstack side for 4.6. Otherwise I
> > fear that by the time we get the toolstack side sorted out to our
> > satisfaction the basic functionality (which seems to be largely done)
> > will have missed 4.5.
>
> Depending of the use cases, your solution based on "iomem", "irq" and DT
> blob might be enough for device platform passthrough.
I hope so.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 10:11 ` Ian Campbell
@ 2014-09-10 18:45 ` Julien Grall
2014-09-11 8:58 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-10 18:45 UTC (permalink / raw)
To: Ian Campbell
Cc: xen-devel, tim, Christoffer Dall, stefano.stabellini, Andrii Tseglytskyi
Hi Ian,
On 10/09/14 03:11, Ian Campbell wrote:
>>> My suspicion is that regular folks won't really be using passthrough
>>> until it is via PCI and that in the meantime this functionality is only
>>> going to be used by e.g. people building embedded system and superkeen
>>> early adopters both of whom know what they are doing and can tolerate
>>> some hacks etc to get things working (and I think that's fine, it's
>>> still a worthwhile set of things to get into 4.5 and those folks are
>>> worth supporting).
>>>
>>> I'm also worried that we may be committing ourselves to a libxl API
>>> already without really working through all the issues (e.g. other
>>> properties).
>>>
>>> Given that I wonder if we wouldn't be better off for 4.5 supporting
>>> something much simpler at the toolstack level, namely allowing users to
>>> use iomem= and irq= in their domain config to map platform devices
>>> through (already works with your series today?)
>>
>> This would need a bit a plumbing for irq part to allow the user choosing
>> the VIRQ (like Arianna did for MMIO range).
>
> Is it required, or can we just number them from IRQ32 onwards?
The current code is actually numbering from IRQ32 onwards. I was mostly
thinking of the Andrii use case. Shall we handle it for Xen 4.5?
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-10 18:45 ` Julien Grall
@ 2014-09-11 8:58 ` Ian Campbell
2014-09-11 19:11 ` Julien Grall
0 siblings, 1 reply; 119+ messages in thread
From: Ian Campbell @ 2014-09-11 8:58 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, Christoffer Dall, stefano.stabellini, Andrii Tseglytskyi
On Wed, 2014-09-10 at 11:45 -0700, Julien Grall wrote:
> Hi Ian,
>
> On 10/09/14 03:11, Ian Campbell wrote:
> >>> My suspicion is that regular folks won't really be using passthrough
> >>> until it is via PCI and that in the meantime this functionality is only
> >>> going to be used by e.g. people building embedded system and superkeen
> >>> early adopters both of whom know what they are doing and can tolerate
> >>> some hacks etc to get things working (and I think that's fine, it's
> >>> still a worthwhile set of things to get into 4.5 and those folks are
> >>> worth supporting).
> >>>
> >>> I'm also worried that we may be committing ourselves to a libxl API
> >>> already without really working through all the issues (e.g. other
> >>> properties).
> >>>
> >>> Given that I wonder if we wouldn't be better off for 4.5 supporting
> >>> something much simpler at the toolstack level, namely allowing users to
> >>> use iomem= and irq= in their domain config to map platform devices
> >>> through (already works with your series today?)
> >>
> >> This would need a bit a plumbing for irq part to allow the user choosing
> >> the VIRQ (like Arianna did for MMIO range).
> >
> > Is it required, or can we just number them from IRQ32 onwards?
>
> The current code is actually numbering from IRQ32 onwards.
In the hypervisor though, I thought? I meant do it in the toolstack,
which is where you can more easily handle overrides like user requests
for specific virq:pirq mappings (as a future extension).
IOW the *hypercall* interface now should take both a pirq and a virq and
map on to the other, but there is no need right now (unless you want to)
to plumb that all the way out to (lib)xl.
> I was mostly
> thinking of the Andrii use case. Shall we handle it for Xen 4.5?
I think it's a nice to have if we can do it in time, but I wouldn't want
to risk the rest of the series missing the boat because of it.
Ian.
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-11 8:58 ` Ian Campbell
@ 2014-09-11 19:11 ` Julien Grall
2014-09-12 10:16 ` Ian Campbell
0 siblings, 1 reply; 119+ messages in thread
From: Julien Grall @ 2014-09-11 19:11 UTC (permalink / raw)
To: Ian Campbell
Cc: xen-devel, tim, Christoffer Dall, stefano.stabellini, Andrii Tseglytskyi
Hi Ian,
On 11/09/14 01:58, Ian Campbell wrote:
> On Wed, 2014-09-10 at 11:45 -0700, Julien Grall wrote:
>> Hi Ian,
>>
>> On 10/09/14 03:11, Ian Campbell wrote:
>>>>> My suspicion is that regular folks won't really be using passthrough
>>>>> until it is via PCI and that in the meantime this functionality is only
>>>>> going to be used by e.g. people building embedded system and superkeen
>>>>> early adopters both of whom know what they are doing and can tolerate
>>>>> some hacks etc to get things working (and I think that's fine, it's
>>>>> still a worthwhile set of things to get into 4.5 and those folks are
>>>>> worth supporting).
>>>>>
>>>>> I'm also worried that we may be committing ourselves to a libxl API
>>>>> already without really working through all the issues (e.g. other
>>>>> properties).
>>>>>
>>>>> Given that I wonder if we wouldn't be better off for 4.5 supporting
>>>>> something much simpler at the toolstack level, namely allowing users to
>>>>> use iomem= and irq= in their domain config to map platform devices
>>>>> through (already works with your series today?)
>>>>
>>>> This would need a bit a plumbing for irq part to allow the user choosing
>>>> the VIRQ (like Arianna did for MMIO range).
>>>
>>> Is it required, or can we just number them from IRQ32 onwards?
>>
>> The current code is actually numbering from IRQ32 onwards.
>
> In the hypervisor though, I thought? I meant do it in the toolstack,
> which is where you can more easily handle overrides like user requests
> for specific virq:pirq mappings (as a future extension).
>
> IOW the *hypercall* interface now should take both a pirq and a virq and
> map on to the other, but there is no need right now (unless you want to)
> to plumb that all the way out to (lib)xl.
>
>> I was mostly
>> thinking of the Andrii use case. Shall we handle it for Xen 4.5?
>
> I think it's a nice to have if we can do it in time, but I wouldn't want
> to risk the rest of the series missing the boat because of it.
I though more about the possibility for Xen 4.5. We could keep VIRQ ==
PIRQ when the user specificy the list of interrupts via "irqs=". And
then revisit the solution for Xen 4.6.
The new DOMCTL to configure the number of SPIS will be still there to
avoid allocation on domain that doesn't require anything.
On the overall, this solution will require less code from the toolstack
point of view. This would mean that PHYSDEVOP_map_pirq on ARM will
always require an index and the pirq.
AFAIU, on x86 it's possible to specify the pirq only when this hypercall
is called from an HVM.
Regards,
--
Julien Grall
^ permalink raw reply [flat|nested] 119+ messages in thread
* Re: [PATCH v2 00/21] xen/arm: Add support for non-pci passthrough
2014-09-11 19:11 ` Julien Grall
@ 2014-09-12 10:16 ` Ian Campbell
0 siblings, 0 replies; 119+ messages in thread
From: Ian Campbell @ 2014-09-12 10:16 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, tim, Christoffer Dall, stefano.stabellini, Andrii Tseglytskyi
On Thu, 2014-09-11 at 12:11 -0700, Julien Grall wrote:
> Hi Ian,
>
> On 11/09/14 01:58, Ian Campbell wrote:
> > On Wed, 2014-09-10 at 11:45 -0700, Julien Grall wrote:
> >> Hi Ian,
> >>
> >> On 10/09/14 03:11, Ian Campbell wrote:
> >>>>> My suspicion is that regular folks won't really be using passthrough
> >>>>> until it is via PCI and that in the meantime this functionality is only
> >>>>> going to be used by e.g. people building embedded system and superkeen
> >>>>> early adopters both of whom know what they are doing and can tolerate
> >>>>> some hacks etc to get things working (and I think that's fine, it's
> >>>>> still a worthwhile set of things to get into 4.5 and those folks are
> >>>>> worth supporting).
> >>>>>
> >>>>> I'm also worried that we may be committing ourselves to a libxl API
> >>>>> already without really working through all the issues (e.g. other
> >>>>> properties).
> >>>>>
> >>>>> Given that I wonder if we wouldn't be better off for 4.5 supporting
> >>>>> something much simpler at the toolstack level, namely allowing users to
> >>>>> use iomem= and irq= in their domain config to map platform devices
> >>>>> through (already works with your series today?)
> >>>>
> >>>> This would need a bit a plumbing for irq part to allow the user choosing
> >>>> the VIRQ (like Arianna did for MMIO range).
> >>>
> >>> Is it required, or can we just number them from IRQ32 onwards?
> >>
> >> The current code is actually numbering from IRQ32 onwards.
> >
> > In the hypervisor though, I thought? I meant do it in the toolstack,
> > which is where you can more easily handle overrides like user requests
> > for specific virq:pirq mappings (as a future extension).
> >
> > IOW the *hypercall* interface now should take both a pirq and a virq and
> > map on to the other, but there is no need right now (unless you want to)
> > to plumb that all the way out to (lib)xl.
> >
> >> I was mostly
> >> thinking of the Andrii use case. Shall we handle it for Xen 4.5?
> >
> > I think it's a nice to have if we can do it in time, but I wouldn't want
> > to risk the rest of the series missing the boat because of it.
>
> I though more about the possibility for Xen 4.5. We could keep VIRQ ==
> PIRQ when the user specificy the list of interrupts via "irqs=". And
> then revisit the solution for Xen 4.6.
I guess I don't mind if we do 1:1 or autoincrementing for 4.5.
> The new DOMCTL to configure the number of SPIS will be still there to
> avoid allocation on domain that doesn't require anything.
>
> On the overall, this solution will require less code from the toolstack
> point of view. This would mean that PHYSDEVOP_map_pirq on ARM will
> always require an index and the pirq.
>
> AFAIU, on x86 it's possible to specify the pirq only when this hypercall
> is called from an HVM.
That's probably because on x86 PV there is no virq, only evtchns. We
should be following the HVM style though.
>
> Regards,
>
^ permalink raw reply [flat|nested] 119+ messages in thread