* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-09 1:00 ` Stefano Stabellini
@ 2022-04-09 1:44 ` Stefano Stabellini
2022-04-11 10:44 ` Bertrand Marquis
2022-04-11 15:28 ` Rahul Singh
2022-04-11 18:16 ` Julien Grall
2 siblings, 1 reply; 21+ messages in thread
From: Stefano Stabellini @ 2022-04-09 1:44 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Rahul Singh, xen-devel, bertrand.marquis, Andrew Cooper,
George Dunlap, Jan Beulich, Julien Grall, Wei Liu
[-- Attachment #1: Type: text/plain, Size: 8207 bytes --]
On Fri, 8 Apr 2022, Stefano Stabellini wrote:
> On Wed, 23 Mar 2022, Rahul Singh wrote:
> > in dom0less system. This patch introduce the new feature to support the
> > signaling between two domUs in dom0less system.
> >
> > Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> > ---
> > docs/designs/dom0less-evtchn.md | 96 +++++++++++++++++++++++++++++++++
> > 1 file changed, 96 insertions(+)
> > create mode 100644 docs/designs/dom0less-evtchn.md
> >
> > diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
> > new file mode 100644
> > index 0000000000..6a1b7e8c22
> > --- /dev/null
> > +++ b/docs/designs/dom0less-evtchn.md
> > @@ -0,0 +1,96 @@
> > +# Signaling support between two domUs on dom0less system
> > +
> > +## Current state: Draft version
> > +
> > +## Proposer(s): Rahul Singh, Bertrand Marquis
> > +
> > +## Problem Statement:
> > +
> > +The goal of this work is to define a simple signaling system between Xen guests
> > +in dom0less systems.
> > +
> > +In dom0less system, we cannot make use of xenbus and xenstore that are used in
> > +normal systems with dynamic VMs to communicate between domains by providing a
> > +bus abstraction for paravirtualized drivers.
> > +
> > +One possible solution to implement the signaling system between domUs is based
> > +on event channels.
>
> I suggest to reword this as follows:
>
> ---
> Dom0less guests would benefit from a statically-defined memory sharing
> and signally system for communication. One that would be immediately
> available at boot without any need for dynamic configurations.
>
> In embedded a great variety of guest operating system kernels exist,
> many of which don't have support for xenstore, grant table or other
> complex drivers. Some of them are small kernel-space applications (often
> called "baremetal", not to be confused with the term "baremetal" used in
> datacenter which means "without hypervisors") or RTOSes. Additionally,
> for safety reasons, users often need to be able to configure the full
> system statically so that it can be verified statically.
>
> Event channels are very simple and can be added even to baremetal
> applications. This proposal introduces a way to define them statically
> to make them suitable to dom0less embedded deployments.
> ---
>
>
> > +## Proposal:
> > +
> > +Event channels are the basic primitive provided by Xen for event notifications.
> > +An event channel is a logical connection between 2 domains (more specifically
> > +between dom1,port1 and dom2,port2). They essentially store one bit of
> > +information, the event of interest is signalled by transitioning this bit from
> > +0 to 1. An event is an equivalent of a hardware interrupt.
> > +
> > +Notifications are received by a guest via an interrupt from Xen to the guest,
> > +indicating when an event arrives (setting the bit). Further notifications are
> > +masked until the bit is cleared again. When a domain wants to wait for data it
> > +will block until an event arrives, and then send an event to signal that data
> > +has been consumed. Events are delivered asynchronously to guests and are
> > +enqueued when the guest is not running.
> > +
> > +Event channel communication will be established statically between two domU
> > +guests before unpausing the domains after domain creation. Event channel
> > +connection information between domUs will be passed to XEN via device tree
> > +node.
> > +
> > +Under the /chosen node, there needs to be sub nodes with compatible
> > +"xen,evtchn" that descibes the event channel connection between two domUs.
> > +
> > +The event channel sub-node has the following properties:
> > +
> > +- compatible
> > +
> > + "xen,evtchn"
> > +
> > +- xen,evtchn
> > +
> > + The property is four numbers of tuples of
> > + (local-port-domU1,domU1-phandle,local-port-domU2,domU2-phandle) where:
> > +
> > + local-port-domU1 is an integer value that will be used to allocte local
> > + port for domU1 to send an event notification to the remote domain.
> > +
> > + domU1-phandle is a single phandle to an domain to which local-port-domU1
> > + will be allocated.
> > +
> > + local-port-domU2 is an integer value that will be used to allocte local
> > + port for domU2 to send an event notification to the remote domain.
> > +
> > + domU2-phandle is a single phandle to an domain to which local-port-domU2
> > + will be allocated.
> > +
> > +Example:
> > +
> > + chosen {
> > + ....
> > +
> > + domU1: domU1 {
> > + ......
> > + };
> > +
> > + domU2: domU2 {
> > + ......
> > + };
> > +
> > + evtchn@1 {
> > + compatible = "xen,evtchn";
> > + xen,evtchn = <0xa &domU1 0xb &domU2>;
> > + };
> > +
> > + evtchn@2 {
> > + compatible = "xen,evtchn";
> > + xen,evtchn = <0xc &domU1 0xd &domU2>;
> > + };
> > + };
>
> There is no need to use two evtchn nodes for this given that in device
> tree it is entirely normal to have multiple tuplets in a property. Also,
> it would be good to have a version number in the compatible string so
> that we can change version in the future.
>
> 1)
> chosen {
> ....
>
> domU1: domU1 {
> ......
> };
>
> domU2: domU2 {
> ......
> };
>
> evtchn {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xa &domU1 0xb &domU2 0xc &domU1 0xd &domU2>;
> };
> };
>
>
> I should mention that it would be also possible to use sub-nodes to
> express this information:
>
> 2)
> domU1: domU1 {
> ...
> /* one sub-node per local event channel */
> ec1: evtchn@a {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0xa &ec3>
> };
> ec2: evtchn@c {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xc &ec4>
> };
> };
>
> domU2: domU2 {
> ...
> ec3: evtchn@b {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xb &ec1>
> };
> ec4: evtchn@d {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xa &ec2>
> };
> };
> };
>
> This format has the advantage that doesn't need a new top-level node
> type under /chosen. That is desirable few a few reasons. Today we only
> have domains (dom0 is legacy). In the future we might have nested
> domains and non-Xen domains. With System Device Tree, domains are under
> /domains instead of /chosen.
>
> So normally I would argue to use the sub-node format because it doesn't
> need a new top-level node under /chosen. However, in this case it looks
> like the 1) format is simpler to write and also simpler to parse in Xen.
>
> In 1), we would need to loop over xen,evtchn and for each tuplet we
> would only need to fetch the foreign domid.
>
> In 2), we would need to check the compatible string of every
> "xen,evtchn-v1" node, and we would have to fetch from the phandle both
> the remote event channel number but also the domain-id of the parent.
>
> So it looks like 1) is better because it is much simpler to parse. Do
> you agree?
[...]
>
> I think this is fine in principle. Like Jan wrote, at some point we'll
> need to specify the device tree binding to expose this information to
> the guest.
Actually, thinking more about it, I think it is likely that the guest
device tree bindings will include information about how it is supposed
to be used. For instance, the domU device tree might pair an event
channel with a shared memory region so that the domU knows that they are
expected to be used together.
If Xen is to generate such a device tree for guests, then we need that
information also on the host device tree too (the one given to Xen and
discussed here.)
So, I think it would be a good idea to discuss the domU device tree
bindings for this, as part of this design document, even if we don't
implement it straight away.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-09 1:44 ` Stefano Stabellini
@ 2022-04-11 10:44 ` Bertrand Marquis
2022-04-14 1:14 ` Stefano Stabellini
0 siblings, 1 reply; 21+ messages in thread
From: Bertrand Marquis @ 2022-04-11 10:44 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Rahul Singh, xen-devel, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu
Hi Stefano,
> On 9 Apr 2022, at 02:44, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Fri, 8 Apr 2022, Stefano Stabellini wrote:
>> On Wed, 23 Mar 2022, Rahul Singh wrote:
>>> in dom0less system. This patch introduce the new feature to support the
>>> signaling between two domUs in dom0less system.
>>>
>>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>>> ---
>>> docs/designs/dom0less-evtchn.md | 96 +++++++++++++++++++++++++++++++++
>>> 1 file changed, 96 insertions(+)
>>> create mode 100644 docs/designs/dom0less-evtchn.md
>>>
>>> diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
>>> new file mode 100644
>>> index 0000000000..6a1b7e8c22
>>> --- /dev/null
>>> +++ b/docs/designs/dom0less-evtchn.md
>>> @@ -0,0 +1,96 @@
>>> +# Signaling support between two domUs on dom0less system
>>> +
>>> +## Current state: Draft version
>>> +
>>> +## Proposer(s): Rahul Singh, Bertrand Marquis
>>> +
>>> +## Problem Statement:
>>> +
>>> +The goal of this work is to define a simple signaling system between Xen guests
>>> +in dom0less systems.
>>> +
>>> +In dom0less system, we cannot make use of xenbus and xenstore that are used in
>>> +normal systems with dynamic VMs to communicate between domains by providing a
>>> +bus abstraction for paravirtualized drivers.
>>> +
>>> +One possible solution to implement the signaling system between domUs is based
>>> +on event channels.
>>
>> I suggest to reword this as follows:
>>
>> ---
>> Dom0less guests would benefit from a statically-defined memory sharing
>> and signally system for communication. One that would be immediately
>> available at boot without any need for dynamic configurations.
>>
>> In embedded a great variety of guest operating system kernels exist,
>> many of which don't have support for xenstore, grant table or other
>> complex drivers. Some of them are small kernel-space applications (often
>> called "baremetal", not to be confused with the term "baremetal" used in
>> datacenter which means "without hypervisors") or RTOSes. Additionally,
>> for safety reasons, users often need to be able to configure the full
>> system statically so that it can be verified statically.
>>
>> Event channels are very simple and can be added even to baremetal
>> applications. This proposal introduces a way to define them statically
>> to make them suitable to dom0less embedded deployments.
>> ---
>>
>>
>>> +## Proposal:
>>> +
>>> +Event channels are the basic primitive provided by Xen for event notifications.
>>> +An event channel is a logical connection between 2 domains (more specifically
>>> +between dom1,port1 and dom2,port2). They essentially store one bit of
>>> +information, the event of interest is signalled by transitioning this bit from
>>> +0 to 1. An event is an equivalent of a hardware interrupt.
>>> +
>>> +Notifications are received by a guest via an interrupt from Xen to the guest,
>>> +indicating when an event arrives (setting the bit). Further notifications are
>>> +masked until the bit is cleared again. When a domain wants to wait for data it
>>> +will block until an event arrives, and then send an event to signal that data
>>> +has been consumed. Events are delivered asynchronously to guests and are
>>> +enqueued when the guest is not running.
>>> +
>>> +Event channel communication will be established statically between two domU
>>> +guests before unpausing the domains after domain creation. Event channel
>>> +connection information between domUs will be passed to XEN via device tree
>>> +node.
>>> +
>>> +Under the /chosen node, there needs to be sub nodes with compatible
>>> +"xen,evtchn" that descibes the event channel connection between two domUs.
>>> +
>>> +The event channel sub-node has the following properties:
>>> +
>>> +- compatible
>>> +
>>> + "xen,evtchn"
>>> +
>>> +- xen,evtchn
>>> +
>>> + The property is four numbers of tuples of
>>> + (local-port-domU1,domU1-phandle,local-port-domU2,domU2-phandle) where:
>>> +
>>> + local-port-domU1 is an integer value that will be used to allocte local
>>> + port for domU1 to send an event notification to the remote domain.
>>> +
>>> + domU1-phandle is a single phandle to an domain to which local-port-domU1
>>> + will be allocated.
>>> +
>>> + local-port-domU2 is an integer value that will be used to allocte local
>>> + port for domU2 to send an event notification to the remote domain.
>>> +
>>> + domU2-phandle is a single phandle to an domain to which local-port-domU2
>>> + will be allocated.
>>> +
>>> +Example:
>>> +
>>> + chosen {
>>> + ....
>>> +
>>> + domU1: domU1 {
>>> + ......
>>> + };
>>> +
>>> + domU2: domU2 {
>>> + ......
>>> + };
>>> +
>>> + evtchn@1 {
>>> + compatible = "xen,evtchn";
>>> + xen,evtchn = <0xa &domU1 0xb &domU2>;
>>> + };
>>> +
>>> + evtchn@2 {
>>> + compatible = "xen,evtchn";
>>> + xen,evtchn = <0xc &domU1 0xd &domU2>;
>>> + };
>>> + };
>>
>> There is no need to use two evtchn nodes for this given that in device
>> tree it is entirely normal to have multiple tuplets in a property. Also,
>> it would be good to have a version number in the compatible string so
>> that we can change version in the future.
>>
>> 1)
>> chosen {
>> ....
>>
>> domU1: domU1 {
>> ......
>> };
>>
>> domU2: domU2 {
>> ......
>> };
>>
>> evtchn {
>> compatible = "xen,evtchn-v1";
>> xen,evtchn = <0xa &domU1 0xb &domU2 0xc &domU1 0xd &domU2>;
>> };
>> };
>>
>>
>> I should mention that it would be also possible to use sub-nodes to
>> express this information:
>>
>> 2)
>> domU1: domU1 {
>> ...
>> /* one sub-node per local event channel */
>> ec1: evtchn@a {
>> compatible = "xen,evtchn-v1";
>> /* local-evtchn link-to-foreign-evtchn */
>> xen,evtchn = <0xa &ec3>
>> };
>> ec2: evtchn@c {
>> compatible = "xen,evtchn-v1";
>> xen,evtchn = <0xc &ec4>
>> };
>> };
>>
>> domU2: domU2 {
>> ...
>> ec3: evtchn@b {
>> compatible = "xen,evtchn-v1";
>> xen,evtchn = <0xb &ec1>
>> };
>> ec4: evtchn@d {
>> compatible = "xen,evtchn-v1";
>> xen,evtchn = <0xa &ec2>
>> };
>> };
>> };
>>
>> This format has the advantage that doesn't need a new top-level node
>> type under /chosen. That is desirable few a few reasons. Today we only
>> have domains (dom0 is legacy). In the future we might have nested
>> domains and non-Xen domains. With System Device Tree, domains are under
>> /domains instead of /chosen.
>>
>> So normally I would argue to use the sub-node format because it doesn't
>> need a new top-level node under /chosen. However, in this case it looks
>> like the 1) format is simpler to write and also simpler to parse in Xen.
>>
>> In 1), we would need to loop over xen,evtchn and for each tuplet we
>> would only need to fetch the foreign domid.
>>
>> In 2), we would need to check the compatible string of every
>> "xen,evtchn-v1" node, and we would have to fetch from the phandle both
>> the remote event channel number but also the domain-id of the parent.
>>
>> So it looks like 1) is better because it is much simpler to parse. Do
>> you agree?
>
> [...]
>>
>> I think this is fine in principle. Like Jan wrote, at some point we'll
>> need to specify the device tree binding to expose this information to
>> the guest.
>
> Actually, thinking more about it, I think it is likely that the guest
> device tree bindings will include information about how it is supposed
> to be used. For instance, the domU device tree might pair an event
> channel with a shared memory region so that the domU knows that they are
> expected to be used together.
>
> If Xen is to generate such a device tree for guests, then we need that
> information also on the host device tree too (the one given to Xen and
> discussed here.)
>
> So, I think it would be a good idea to discuss the domU device tree
> bindings for this, as part of this design document, even if we don't
> implement it straight away.
What you mention here is actually combining 2 different solutions inside
Xen to build a custom communication solution.
My assumption here is that the user will actually create the device tree
nodes he wants to do that and we should not create guest node entries
as it would enforce some design.
If everything can be statically defined for Xen then the user can also
statically define node entries inside his guest to make use of the events
and the shared memories.
For example one might need more than one event to build a communication
system, or more than one shared memory or could build something
communicating with multiple guest thus requiring even more events and
shared memories.
Cheers
Bertrand
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-11 10:44 ` Bertrand Marquis
@ 2022-04-14 1:14 ` Stefano Stabellini
2022-04-14 13:18 ` Bertrand Marquis
2022-04-19 17:16 ` Rahul Singh
0 siblings, 2 replies; 21+ messages in thread
From: Stefano Stabellini @ 2022-04-14 1:14 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Stefano Stabellini, Rahul Singh, xen-devel, Andrew Cooper,
George Dunlap, Jan Beulich, Julien Grall, Wei Liu
On Mon, 11 Apr 2022, Bertrand Marquis wrote:
> What you mention here is actually combining 2 different solutions inside
> Xen to build a custom communication solution.
> My assumption here is that the user will actually create the device tree
> nodes he wants to do that and we should not create guest node entries
> as it would enforce some design.
>
> If everything can be statically defined for Xen then the user can also
> statically define node entries inside his guest to make use of the events
> and the shared memories.
>
> For example one might need more than one event to build a communication
> system, or more than one shared memory or could build something
> communicating with multiple guest thus requiring even more events and
> shared memories.
Hi Bertrand, Rahul,
If the guests are allowed some level of dynamic discovery, this feature
is not needed. They can discover the shared memory location from the
domU device tree, then proceed to allocate evtchns as needed and tell
the other end the evtchn numbers over shared memory. I already have an
example of it here:
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2251030537/Xen+Shared+Memory+and+Interrupts+Between+VMs
What if the guest doesn't support device tree at runtime, like baremetal
or Zephyr? The shared memory address can be hardcoded or generated from
device tree at build time. That's no problem. Then, the event channels
can still be allocated at runtime and passed to the other end over
shared memory. That's what the example on the wikipage does.
When are static event channels actually useful? When the application
cannot allocate the event channels at runtime at all. The reason for the
restriction could be related to safety (no dynamic allocations at
runtime) or convenience (everything else is fully static, why should the
event channel numbers be dynamic?)
Given the above, I can see why there is no need to describe the static
event channel info in the domU device tree: static event channels are
only useful in fully static configurations, and in those configurations
the domU device tree dynamically generated by Xen is not needed. I can
see where you are coming from.
The workflow that we have been trying to enable with the System Device
Tree effort (System Device Tree is similar to a normal Device Tree plus
the xen,domains nodes) is the following:
S-DT ---[lopper]---> Linux DT
L--> Zephyr DT ---[Zephyr build]---> Zephyr .h files
S-DT contains all the needed information for both the regular Linux DT
generation and also the Zephyr/RTOS/baremetal header files generation,
that happens at build time.
S-DT is not the same as the Xen device tree, but so far it has been
conceptually and practically similar. I always imagine that the bindings
we have in Xen we'll also have corresponding bindings in System Device
Tree.
For this workflow to work S-DT needs all the info so that both Linux DT
and Zephyr DT and Zephyr .h files can be generated.
Does this proposal contain enough information so that Zephyr .h files
could be statically generated with the event channel numbers and static
shared memory regions addresses?
I am not sure. Maybe not?
It is possible that the shared memory usage is so application specific
that there is no point in even talking about it. But I think that
introducing a simple bundle of both event channels and shared memory
would help a lot.
Something like the following in the Xen device tree would be enough to
specify an arbitrary number of event channels connected with the same
domains sharing the memory region.
It looks like that if we did the below, we would carry a lot more useful
information compared to the original proposal alone. We could add a
similar xen,notificaiton property to the domU reserved-memory region in
device tree generated by Xen for consistency, so that everything
available to the domU is described fully in device tree.
domU1 {
compatible = "xen,domain";
/* one sub-node per local event channel */
ec1: evtchn@1 {
compatible = "xen,evtchn-v1";
/* local-evtchn link-to-foreign-evtchn */
xen,evtchn = <0x1 &ec3>
};
ec2: evtchn@2 {
compatible = "xen,evtchn-v1";
xen,evtchn = <0x2 &ec4>
};
/*
* shared memory region between DomU1 and DomU2.
*/
domU1-shared-mem@50000000 {
compatible = "xen,domain-shared-memory-v1";
xen,shm-id = <0x1>;
xen,shared-mem = <0x50000000 0x20000000 0x60000000>;
/* this is new */
xen,notification = <&ec1 &ec2>;
}
};
domU2 {
compatible = "xen,domain";
/* one sub-node per local event channel */
ec3: evtchn@3 {
compatible = "xen,evtchn-v1";
/* local-evtchn link-to-foreign-evtchn */
xen,evtchn = <0x3 &ec1>
};
ec4: evtchn@4 {
compatible = "xen,evtchn-v1";
xen,evtchn = <0x4 &ec2>
};
/*
* shared memory region between domU1 and domU2.
*/
domU2-shared-mem@50000000 {
compatible = "xen,domain-shared-memory-v1";
xen,shm-id = <0x1>;
xen,shared-mem = <0x50000000 0x20000000 0x70000000>;
/* this is new */
xen,notification = <&ec3 &ec4>;
}
};
The good thing about this is that:
- it is very flexible
- nothing to do in this series, except switching to the
one-subnode-per-evtchn model, which we called 2) in the previous email
- there were good reasons to use the one-subnode-per-evtchn model anyway
- the xen,notification property can be added later without issues, after Penny's series
There are a couple of ways to implement the xen,notification property
but we don't need to discuss them now.
Short Summary
------------
I think it is fine to only introduce the Xen device tree binding for
static event channels without domU binding, but I prefer if we switched
to using proposal 2) "one subnode per event channel".
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-14 1:14 ` Stefano Stabellini
@ 2022-04-14 13:18 ` Bertrand Marquis
2022-04-14 17:11 ` Stefano Stabellini
2022-04-19 17:16 ` Rahul Singh
1 sibling, 1 reply; 21+ messages in thread
From: Bertrand Marquis @ 2022-04-14 13:18 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Rahul Singh, xen-devel, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu
Hi Stefano,
> On 14 Apr 2022, at 02:14, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Mon, 11 Apr 2022, Bertrand Marquis wrote:
>> What you mention here is actually combining 2 different solutions inside
>> Xen to build a custom communication solution.
>> My assumption here is that the user will actually create the device tree
>> nodes he wants to do that and we should not create guest node entries
>> as it would enforce some design.
>>
>> If everything can be statically defined for Xen then the user can also
>> statically define node entries inside his guest to make use of the events
>> and the shared memories.
>>
>> For example one might need more than one event to build a communication
>> system, or more than one shared memory or could build something
>> communicating with multiple guest thus requiring even more events and
>> shared memories.
>
> Hi Bertrand, Rahul,
>
> If the guests are allowed some level of dynamic discovery, this feature
> is not needed. They can discover the shared memory location from the
> domU device tree, then proceed to allocate evtchns as needed and tell
> the other end the evtchn numbers over shared memory. I already have an
> example of it here:
>
> https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2251030537/Xen+Shared+Memory+and+Interrupts+Between+VMs
>
> What if the guest doesn't support device tree at runtime, like baremetal
> or Zephyr? The shared memory address can be hardcoded or generated from
> device tree at build time. That's no problem. Then, the event channels
> can still be allocated at runtime and passed to the other end over
> shared memory. That's what the example on the wikipage does.
>
>
> When are static event channels actually useful? When the application
> cannot allocate the event channels at runtime at all. The reason for the
> restriction could be related to safety (no dynamic allocations at
> runtime) or convenience (everything else is fully static, why should the
> event channel numbers be dynamic?)
An other use case here is dom0less: you cannot have dom0 create them.
>
> Given the above, I can see why there is no need to describe the static
> event channel info in the domU device tree: static event channels are
> only useful in fully static configurations, and in those configurations
> the domU device tree dynamically generated by Xen is not needed. I can
> see where you are coming from.
>
>
> The workflow that we have been trying to enable with the System Device
> Tree effort (System Device Tree is similar to a normal Device Tree plus
> the xen,domains nodes) is the following:
>
> S-DT ---[lopper]---> Linux DT
> L--> Zephyr DT ---[Zephyr build]---> Zephyr .h files
>
> S-DT contains all the needed information for both the regular Linux DT
> generation and also the Zephyr/RTOS/baremetal header files generation,
> that happens at build time.
>
> S-DT is not the same as the Xen device tree, but so far it has been
> conceptually and practically similar. I always imagine that the bindings
> we have in Xen we'll also have corresponding bindings in System Device
> Tree.
>
> For this workflow to work S-DT needs all the info so that both Linux DT
> and Zephyr DT and Zephyr .h files can be generated.
>
> Does this proposal contain enough information so that Zephyr .h files
> could be statically generated with the event channel numbers and static
> shared memory regions addresses?
>
> I am not sure. Maybe not?
Yes it should be possible to have all infos as the integrator will setup the
system and will decide upfront the address and the event(s) number(s).
>
>
> It is possible that the shared memory usage is so application specific
> that there is no point in even talking about it. But I think that
> introducing a simple bundle of both event channels and shared memory
> would help a lot.
>
> Something like the following in the Xen device tree would be enough to
> specify an arbitrary number of event channels connected with the same
> domains sharing the memory region.
>
> It looks like that if we did the below, we would carry a lot more useful
> information compared to the original proposal alone. We could add a
> similar xen,notificaiton property to the domU reserved-memory region in
> device tree generated by Xen for consistency, so that everything
> available to the domU is described fully in device tree.
>
>
> domU1 {
> compatible = "xen,domain";
>
> /* one sub-node per local event channel */
> ec1: evtchn@1 {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0x1 &ec3>
> };
> ec2: evtchn@2 {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0x2 &ec4>
> };
> /*
> * shared memory region between DomU1 and DomU2.
> */
> domU1-shared-mem@50000000 {
> compatible = "xen,domain-shared-memory-v1";
> xen,shm-id = <0x1>;
> xen,shared-mem = <0x50000000 0x20000000 0x60000000>;
> /* this is new */
> xen,notification = <&ec1 &ec2>;
> }
> };
>
> domU2 {
> compatible = "xen,domain";
>
> /* one sub-node per local event channel */
> ec3: evtchn@3 {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0x3 &ec1>
> };
> ec4: evtchn@4 {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0x4 &ec2>
> };
> /*
> * shared memory region between domU1 and domU2.
> */
> domU2-shared-mem@50000000 {
> compatible = "xen,domain-shared-memory-v1";
> xen,shm-id = <0x1>;
> xen,shared-mem = <0x50000000 0x20000000 0x70000000>;
> /* this is new */
> xen,notification = <&ec3 &ec4>;
> }
> };
Few remarks/questions on this:
- this is not a shared memory anymore as you add a notification system to it
- what if someone wants to use only a shared memory, or an event, what should xen do ?
- in xen device tree, how do you associate the event with the shared memory ?
>
>
>
> The good thing about this is that:
>
> - it is very flexible
> - nothing to do in this series, except switching to the
> one-subnode-per-evtchn model, which we called 2) in the previous email
> - there were good reasons to use the one-subnode-per-evtchn model anyway
> - the xen,notification property can be added later without issues, after Penny's series
>
> There are a couple of ways to implement the xen,notification property
> but we don't need to discuss them now.
I think there is something to do here but we need a bit more discussion and this can be done later.
Right now I am not quite sure we will not add something that will end up not being used.
>
>
> Short Summary
> ------------
> I think it is fine to only introduce the Xen device tree binding for
> static event channels without domU binding, but I prefer if we switched
> to using proposal 2) "one subnode per event channel".
I will let Rahul answer on that.
Cheers
Bertrand
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-14 13:18 ` Bertrand Marquis
@ 2022-04-14 17:11 ` Stefano Stabellini
0 siblings, 0 replies; 21+ messages in thread
From: Stefano Stabellini @ 2022-04-14 17:11 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Stefano Stabellini, Rahul Singh, xen-devel, Andrew Cooper,
George Dunlap, Jan Beulich, Julien Grall, Wei Liu
On Thu, 14 Apr 2022, Bertrand Marquis wrote:
> > On 14 Apr 2022, at 02:14, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >
> > On Mon, 11 Apr 2022, Bertrand Marquis wrote:
> >> What you mention here is actually combining 2 different solutions inside
> >> Xen to build a custom communication solution.
> >> My assumption here is that the user will actually create the device tree
> >> nodes he wants to do that and we should not create guest node entries
> >> as it would enforce some design.
> >>
> >> If everything can be statically defined for Xen then the user can also
> >> statically define node entries inside his guest to make use of the events
> >> and the shared memories.
> >>
> >> For example one might need more than one event to build a communication
> >> system, or more than one shared memory or could build something
> >> communicating with multiple guest thus requiring even more events and
> >> shared memories.
> >
> > Hi Bertrand, Rahul,
> >
> > If the guests are allowed some level of dynamic discovery, this feature
> > is not needed. They can discover the shared memory location from the
> > domU device tree, then proceed to allocate evtchns as needed and tell
> > the other end the evtchn numbers over shared memory. I already have an
> > example of it here:
> >
> > https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2251030537/Xen+Shared+Memory+and+Interrupts+Between+VMs
> >
> > What if the guest doesn't support device tree at runtime, like baremetal
> > or Zephyr? The shared memory address can be hardcoded or generated from
> > device tree at build time. That's no problem. Then, the event channels
> > can still be allocated at runtime and passed to the other end over
> > shared memory. That's what the example on the wikipage does.
> >
> >
> > When are static event channels actually useful? When the application
> > cannot allocate the event channels at runtime at all. The reason for the
> > restriction could be related to safety (no dynamic allocations at
> > runtime) or convenience (everything else is fully static, why should the
> > event channel numbers be dynamic?)
>
> An other use case here is dom0less: you cannot have dom0 create them.
>
> >
> > Given the above, I can see why there is no need to describe the static
> > event channel info in the domU device tree: static event channels are
> > only useful in fully static configurations, and in those configurations
> > the domU device tree dynamically generated by Xen is not needed. I can
> > see where you are coming from.
> >
> >
> > The workflow that we have been trying to enable with the System Device
> > Tree effort (System Device Tree is similar to a normal Device Tree plus
> > the xen,domains nodes) is the following:
> >
> > S-DT ---[lopper]---> Linux DT
> > L--> Zephyr DT ---[Zephyr build]---> Zephyr .h files
> >
> > S-DT contains all the needed information for both the regular Linux DT
> > generation and also the Zephyr/RTOS/baremetal header files generation,
> > that happens at build time.
> >
> > S-DT is not the same as the Xen device tree, but so far it has been
> > conceptually and practically similar. I always imagine that the bindings
> > we have in Xen we'll also have corresponding bindings in System Device
> > Tree.
> >
> > For this workflow to work S-DT needs all the info so that both Linux DT
> > and Zephyr DT and Zephyr .h files can be generated.
> >
> > Does this proposal contain enough information so that Zephyr .h files
> > could be statically generated with the event channel numbers and static
> > shared memory regions addresses?
> >
> > I am not sure. Maybe not?
>
> Yes it should be possible to have all infos as the integrator will setup the
> system and will decide upfront the address and the event(s) number(s).
>
> >
> >
> > It is possible that the shared memory usage is so application specific
> > that there is no point in even talking about it. But I think that
> > introducing a simple bundle of both event channels and shared memory
> > would help a lot.
> >
> > Something like the following in the Xen device tree would be enough to
> > specify an arbitrary number of event channels connected with the same
> > domains sharing the memory region.
> >
> > It looks like that if we did the below, we would carry a lot more useful
> > information compared to the original proposal alone. We could add a
> > similar xen,notificaiton property to the domU reserved-memory region in
> > device tree generated by Xen for consistency, so that everything
> > available to the domU is described fully in device tree.
> >
> >
> > domU1 {
> > compatible = "xen,domain";
> >
> > /* one sub-node per local event channel */
> > ec1: evtchn@1 {
> > compatible = "xen,evtchn-v1";
> > /* local-evtchn link-to-foreign-evtchn */
> > xen,evtchn = <0x1 &ec3>
> > };
> > ec2: evtchn@2 {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0x2 &ec4>
> > };
> > /*
> > * shared memory region between DomU1 and DomU2.
> > */
> > domU1-shared-mem@50000000 {
> > compatible = "xen,domain-shared-memory-v1";
> > xen,shm-id = <0x1>;
> > xen,shared-mem = <0x50000000 0x20000000 0x60000000>;
> > /* this is new */
> > xen,notification = <&ec1 &ec2>;
> > }
> > };
> >
> > domU2 {
> > compatible = "xen,domain";
> >
> > /* one sub-node per local event channel */
> > ec3: evtchn@3 {
> > compatible = "xen,evtchn-v1";
> > /* local-evtchn link-to-foreign-evtchn */
> > xen,evtchn = <0x3 &ec1>
> > };
> > ec4: evtchn@4 {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0x4 &ec2>
> > };
> > /*
> > * shared memory region between domU1 and domU2.
> > */
> > domU2-shared-mem@50000000 {
> > compatible = "xen,domain-shared-memory-v1";
> > xen,shm-id = <0x1>;
> > xen,shared-mem = <0x50000000 0x20000000 0x70000000>;
> > /* this is new */
> > xen,notification = <&ec3 &ec4>;
> > }
> > };
>
> Few remarks/questions on this:
> - this is not a shared memory anymore as you add a notification system to it
> - what if someone wants to use only a shared memory, or an event, what should xen do ?
They still can. xen,notification would only be an optional property, not
a mandatory property. So it is still possible to have shared memory
without notifications (skip the xen,notification property), or event
channels without shared memory (do not link the evtchn to
xen,notification).
> - in xen device tree, how do you associate the event with the shared memory ?
I don't think I understand the question. The example above shows how to
associate the event with the shared memory: the only additional thing
needed (compared to proposal 2 already discussed) is the new optional
property xen,notification.
Xen itself wouldn't have to do anything special when xen,notification is
specified, but would add a similar optional xen,notification property to
the generated domU device tree.
> > The good thing about this is that:
> >
> > - it is very flexible
> > - nothing to do in this series, except switching to the
> > one-subnode-per-evtchn model, which we called 2) in the previous email
> > - there were good reasons to use the one-subnode-per-evtchn model anyway
> > - the xen,notification property can be added later without issues, after Penny's series
> >
> > There are a couple of ways to implement the xen,notification property
> > but we don't need to discuss them now.
>
> I think there is something to do here but we need a bit more discussion and this can be done later.
> Right now I am not quite sure we will not add something that will end up not being used.
Yes, I am not asking to add xen,notification now, neither to the Xen
device tree or the domU device tree. I am only trying to make sure it
would be possible do something like it without major changes to the
existing device tree. And I think it is possible if we use proposal 2).
> > Short Summary
> > ------------
> > I think it is fine to only introduce the Xen device tree binding for
> > static event channels without domU binding, but I prefer if we switched
> > to using proposal 2) "one subnode per event channel".
>
> I will let Rahul answer on that.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-14 1:14 ` Stefano Stabellini
2022-04-14 13:18 ` Bertrand Marquis
@ 2022-04-19 17:16 ` Rahul Singh
2022-04-20 0:25 ` Stefano Stabellini
1 sibling, 1 reply; 21+ messages in thread
From: Rahul Singh @ 2022-04-19 17:16 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Bertrand Marquis, xen-devel, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu
Hello Stefano,
> On 14 Apr 2022, at 2:14 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Mon, 11 Apr 2022, Bertrand Marquis wrote:
>> What you mention here is actually combining 2 different solutions inside
>> Xen to build a custom communication solution.
>> My assumption here is that the user will actually create the device tree
>> nodes he wants to do that and we should not create guest node entries
>> as it would enforce some design.
>>
>> If everything can be statically defined for Xen then the user can also
>> statically define node entries inside his guest to make use of the events
>> and the shared memories.
>>
>> For example one might need more than one event to build a communication
>> system, or more than one shared memory or could build something
>> communicating with multiple guest thus requiring even more events and
>> shared memories.
>
> Hi Bertrand, Rahul,
>
> If the guests are allowed some level of dynamic discovery, this feature
> is not needed. They can discover the shared memory location from the
> domU device tree, then proceed to allocate evtchns as needed and tell
> the other end the evtchn numbers over shared memory. I already have an
> example of it here:
>
> https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2251030537/Xen+Shared+Memory+and+Interrupts+Between+VMs
>
> What if the guest doesn't support device tree at runtime, like baremetal
> or Zephyr? The shared memory address can be hardcoded or generated from
> device tree at build time. That's no problem. Then, the event channels
> can still be allocated at runtime and passed to the other end over
> shared memory. That's what the example on the wikipage does.
>
>
> When are static event channels actually useful? When the application
> cannot allocate the event channels at runtime at all. The reason for the
> restriction could be related to safety (no dynamic allocations at
> runtime) or convenience (everything else is fully static, why should the
> event channel numbers be dynamic?)
>
> Given the above, I can see why there is no need to describe the static
> event channel info in the domU device tree: static event channels are
> only useful in fully static configurations, and in those configurations
> the domU device tree dynamically generated by Xen is not needed. I can
> see where you are coming from.
>
>
> The workflow that we have been trying to enable with the System Device
> Tree effort (System Device Tree is similar to a normal Device Tree plus
> the xen,domains nodes) is the following:
>
> S-DT ---[lopper]---> Linux DT
> L--> Zephyr DT ---[Zephyr build]---> Zephyr .h files
>
> S-DT contains all the needed information for both the regular Linux DT
> generation and also the Zephyr/RTOS/baremetal header files generation,
> that happens at build time.
>
> S-DT is not the same as the Xen device tree, but so far it has been
> conceptually and practically similar. I always imagine that the bindings
> we have in Xen we'll also have corresponding bindings in System Device
> Tree.
>
> For this workflow to work S-DT needs all the info so that both Linux DT
> and Zephyr DT and Zephyr .h files can be generated.
>
> Does this proposal contain enough information so that Zephyr .h files
> could be statically generated with the event channel numbers and static
> shared memory regions addresses?
>
> I am not sure. Maybe not?
>
>
> It is possible that the shared memory usage is so application specific
> that there is no point in even talking about it. But I think that
> introducing a simple bundle of both event channels and shared memory
> would help a lot.
>
> Something like the following in the Xen device tree would be enough to
> specify an arbitrary number of event channels connected with the same
> domains sharing the memory region.
>
> It looks like that if we did the below, we would carry a lot more useful
> information compared to the original proposal alone. We could add a
> similar xen,notificaiton property to the domU reserved-memory region in
> device tree generated by Xen for consistency, so that everything
> available to the domU is described fully in device tree.
>
>
> domU1 {
> compatible = "xen,domain";
>
> /* one sub-node per local event channel */
> ec1: evtchn@1 {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0x1 &ec3>
> };
> ec2: evtchn@2 {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0x2 &ec4>
> };
> /*
> * shared memory region between DomU1 and DomU2.
> */
> domU1-shared-mem@50000000 {
> compatible = "xen,domain-shared-memory-v1";
> xen,shm-id = <0x1>;
> xen,shared-mem = <0x50000000 0x20000000 0x60000000>;
> /* this is new */
> xen,notification = <&ec1 &ec2>;
> }
> };
>
> domU2 {
> compatible = "xen,domain";
>
> /* one sub-node per local event channel */
> ec3: evtchn@3 {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0x3 &ec1>
> };
> ec4: evtchn@4 {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0x4 &ec2>
> };
> /*
> * shared memory region between domU1 and domU2.
> */
> domU2-shared-mem@50000000 {
> compatible = "xen,domain-shared-memory-v1";
> xen,shm-id = <0x1>;
> xen,shared-mem = <0x50000000 0x20000000 0x70000000>;
> /* this is new */
> xen,notification = <&ec3 &ec4>;
> }
> };
>
>
>
> The good thing about this is that:
>
> - it is very flexible
> - nothing to do in this series, except switching to the
> one-subnode-per-evtchn model, which we called 2) in the previous email
> - there were good reasons to use the one-subnode-per-evtchn model anyway
> - the xen,notification property can be added later without issues, after Penny's series
>
> There are a couple of ways to implement the xen,notification property
> but we don't need to discuss them now.
>
>
> Short Summary
> ------------
> I think it is fine to only introduce the Xen device tree binding for
> static event channels without domU binding, but I prefer if we switched
> to using proposal 2) "one subnode per event channel".
Thanks for detailed explanation.I just did the basic implementation of the 2) proposal and
it looks simple and not very complex as I thought earlier.Therefore I am ok to switch to proposal 2).
If everyone is ok with the 2) proposal I will send the v2 of design doc after fixing all the comments.
Regards,
Rahul
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-19 17:16 ` Rahul Singh
@ 2022-04-20 0:25 ` Stefano Stabellini
0 siblings, 0 replies; 21+ messages in thread
From: Stefano Stabellini @ 2022-04-20 0:25 UTC (permalink / raw)
To: Rahul Singh
Cc: Stefano Stabellini, Bertrand Marquis, xen-devel, Andrew Cooper,
George Dunlap, Jan Beulich, Julien Grall, Wei Liu
On Tue, 19 Apr 2022, Rahul Singh wrote:
> > On 14 Apr 2022, at 2:14 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
> >
> > On Mon, 11 Apr 2022, Bertrand Marquis wrote:
> >> What you mention here is actually combining 2 different solutions inside
> >> Xen to build a custom communication solution.
> >> My assumption here is that the user will actually create the device tree
> >> nodes he wants to do that and we should not create guest node entries
> >> as it would enforce some design.
> >>
> >> If everything can be statically defined for Xen then the user can also
> >> statically define node entries inside his guest to make use of the events
> >> and the shared memories.
> >>
> >> For example one might need more than one event to build a communication
> >> system, or more than one shared memory or could build something
> >> communicating with multiple guest thus requiring even more events and
> >> shared memories.
> >
> > Hi Bertrand, Rahul,
> >
> > If the guests are allowed some level of dynamic discovery, this feature
> > is not needed. They can discover the shared memory location from the
> > domU device tree, then proceed to allocate evtchns as needed and tell
> > the other end the evtchn numbers over shared memory. I already have an
> > example of it here:
> >
> > https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2251030537/Xen+Shared+Memory+and+Interrupts+Between+VMs
> >
> > What if the guest doesn't support device tree at runtime, like baremetal
> > or Zephyr? The shared memory address can be hardcoded or generated from
> > device tree at build time. That's no problem. Then, the event channels
> > can still be allocated at runtime and passed to the other end over
> > shared memory. That's what the example on the wikipage does.
> >
> >
> > When are static event channels actually useful? When the application
> > cannot allocate the event channels at runtime at all. The reason for the
> > restriction could be related to safety (no dynamic allocations at
> > runtime) or convenience (everything else is fully static, why should the
> > event channel numbers be dynamic?)
> >
> > Given the above, I can see why there is no need to describe the static
> > event channel info in the domU device tree: static event channels are
> > only useful in fully static configurations, and in those configurations
> > the domU device tree dynamically generated by Xen is not needed. I can
> > see where you are coming from.
> >
> >
> > The workflow that we have been trying to enable with the System Device
> > Tree effort (System Device Tree is similar to a normal Device Tree plus
> > the xen,domains nodes) is the following:
> >
> > S-DT ---[lopper]---> Linux DT
> > L--> Zephyr DT ---[Zephyr build]---> Zephyr .h files
> >
> > S-DT contains all the needed information for both the regular Linux DT
> > generation and also the Zephyr/RTOS/baremetal header files generation,
> > that happens at build time.
> >
> > S-DT is not the same as the Xen device tree, but so far it has been
> > conceptually and practically similar. I always imagine that the bindings
> > we have in Xen we'll also have corresponding bindings in System Device
> > Tree.
> >
> > For this workflow to work S-DT needs all the info so that both Linux DT
> > and Zephyr DT and Zephyr .h files can be generated.
> >
> > Does this proposal contain enough information so that Zephyr .h files
> > could be statically generated with the event channel numbers and static
> > shared memory regions addresses?
> >
> > I am not sure. Maybe not?
> >
> >
> > It is possible that the shared memory usage is so application specific
> > that there is no point in even talking about it. But I think that
> > introducing a simple bundle of both event channels and shared memory
> > would help a lot.
> >
> > Something like the following in the Xen device tree would be enough to
> > specify an arbitrary number of event channels connected with the same
> > domains sharing the memory region.
> >
> > It looks like that if we did the below, we would carry a lot more useful
> > information compared to the original proposal alone. We could add a
> > similar xen,notificaiton property to the domU reserved-memory region in
> > device tree generated by Xen for consistency, so that everything
> > available to the domU is described fully in device tree.
> >
> >
> > domU1 {
> > compatible = "xen,domain";
> >
> > /* one sub-node per local event channel */
> > ec1: evtchn@1 {
> > compatible = "xen,evtchn-v1";
> > /* local-evtchn link-to-foreign-evtchn */
> > xen,evtchn = <0x1 &ec3>
> > };
> > ec2: evtchn@2 {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0x2 &ec4>
> > };
> > /*
> > * shared memory region between DomU1 and DomU2.
> > */
> > domU1-shared-mem@50000000 {
> > compatible = "xen,domain-shared-memory-v1";
> > xen,shm-id = <0x1>;
> > xen,shared-mem = <0x50000000 0x20000000 0x60000000>;
> > /* this is new */
> > xen,notification = <&ec1 &ec2>;
> > }
> > };
> >
> > domU2 {
> > compatible = "xen,domain";
> >
> > /* one sub-node per local event channel */
> > ec3: evtchn@3 {
> > compatible = "xen,evtchn-v1";
> > /* local-evtchn link-to-foreign-evtchn */
> > xen,evtchn = <0x3 &ec1>
> > };
> > ec4: evtchn@4 {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0x4 &ec2>
> > };
> > /*
> > * shared memory region between domU1 and domU2.
> > */
> > domU2-shared-mem@50000000 {
> > compatible = "xen,domain-shared-memory-v1";
> > xen,shm-id = <0x1>;
> > xen,shared-mem = <0x50000000 0x20000000 0x70000000>;
> > /* this is new */
> > xen,notification = <&ec3 &ec4>;
> > }
> > };
> >
> >
> >
> > The good thing about this is that:
> >
> > - it is very flexible
> > - nothing to do in this series, except switching to the
> > one-subnode-per-evtchn model, which we called 2) in the previous email
> > - there were good reasons to use the one-subnode-per-evtchn model anyway
> > - the xen,notification property can be added later without issues, after Penny's series
> >
> > There are a couple of ways to implement the xen,notification property
> > but we don't need to discuss them now.
> >
> >
> > Short Summary
> > ------------
> > I think it is fine to only introduce the Xen device tree binding for
> > static event channels without domU binding, but I prefer if we switched
> > to using proposal 2) "one subnode per event channel".
>
> Thanks for detailed explanation.I just did the basic implementation of the 2) proposal and
> it looks simple and not very complex as I thought earlier.Therefore I am ok to switch to proposal 2).
>
> If everyone is ok with the 2) proposal I will send the v2 of design doc after fixing all the comments.
That's great, thank you Rahul!
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-09 1:00 ` Stefano Stabellini
2022-04-09 1:44 ` Stefano Stabellini
@ 2022-04-11 15:28 ` Rahul Singh
2022-04-11 18:16 ` Julien Grall
2 siblings, 0 replies; 21+ messages in thread
From: Rahul Singh @ 2022-04-11 15:28 UTC (permalink / raw)
To: Stefano Stabellini
Cc: xen-devel, Bertrand Marquis, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Wei Liu
Hi Stefano,
Thanks for reviewing the design.
> On 9 Apr 2022, at 2:00 am, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Wed, 23 Mar 2022, Rahul Singh wrote:
>> in dom0less system. This patch introduce the new feature to support the
>> signaling between two domUs in dom0less system.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> docs/designs/dom0less-evtchn.md | 96 +++++++++++++++++++++++++++++++++
>> 1 file changed, 96 insertions(+)
>> create mode 100644 docs/designs/dom0less-evtchn.md
>>
>> diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
>> new file mode 100644
>> index 0000000000..6a1b7e8c22
>> --- /dev/null
>> +++ b/docs/designs/dom0less-evtchn.md
>> @@ -0,0 +1,96 @@
>> +# Signaling support between two domUs on dom0less system
>> +
>> +## Current state: Draft version
>> +
>> +## Proposer(s): Rahul Singh, Bertrand Marquis
>> +
>> +## Problem Statement:
>> +
>> +The goal of this work is to define a simple signaling system between Xen guests
>> +in dom0less systems.
>> +
>> +In dom0less system, we cannot make use of xenbus and xenstore that are used in
>> +normal systems with dynamic VMs to communicate between domains by providing a
>> +bus abstraction for paravirtualized drivers.
>> +
>> +One possible solution to implement the signaling system between domUs is based
>> +on event channels.
>
> I suggest to reword this as follows:
>
> ---
> Dom0less guests would benefit from a statically-defined memory sharing
> and signally system for communication. One that would be immediately
> available at boot without any need for dynamic configurations.
>
> In embedded a great variety of guest operating system kernels exist,
> many of which don't have support for xenstore, grant table or other
> complex drivers. Some of them are small kernel-space applications (often
> called "baremetal", not to be confused with the term "baremetal" used in
> datacenter which means "without hypervisors") or RTOSes. Additionally,
> for safety reasons, users often need to be able to configure the full
> system statically so that it can be verified statically.
>
> Event channels are very simple and can be added even to baremetal
> applications. This proposal introduces a way to define them statically
> to make them suitable to dom0less embedded deployments.
> ---
>
Ok. This is really a good explanation I will add this in next version.
>
>> +## Proposal:
>> +
>> +Event channels are the basic primitive provided by Xen for event notifications.
>> +An event channel is a logical connection between 2 domains (more specifically
>> +between dom1,port1 and dom2,port2). They essentially store one bit of
>> +information, the event of interest is signalled by transitioning this bit from
>> +0 to 1. An event is an equivalent of a hardware interrupt.
>> +
>> +Notifications are received by a guest via an interrupt from Xen to the guest,
>> +indicating when an event arrives (setting the bit). Further notifications are
>> +masked until the bit is cleared again. When a domain wants to wait for data it
>> +will block until an event arrives, and then send an event to signal that data
>> +has been consumed. Events are delivered asynchronously to guests and are
>> +enqueued when the guest is not running.
>> +
>> +Event channel communication will be established statically between two domU
>> +guests before unpausing the domains after domain creation. Event channel
>> +connection information between domUs will be passed to XEN via device tree
>> +node.
>> +
>> +Under the /chosen node, there needs to be sub nodes with compatible
>> +"xen,evtchn" that descibes the event channel connection between two domUs.
>> +
>> +The event channel sub-node has the following properties:
>> +
>> +- compatible
>> +
>> + "xen,evtchn"
>> +
>> +- xen,evtchn
>> +
>> + The property is four numbers of tuples of
>> + (local-port-domU1,domU1-phandle,local-port-domU2,domU2-phandle) where:
>> +
>> + local-port-domU1 is an integer value that will be used to allocte local
>> + port for domU1 to send an event notification to the remote domain.
>> +
>> + domU1-phandle is a single phandle to an domain to which local-port-domU1
>> + will be allocated.
>> +
>> + local-port-domU2 is an integer value that will be used to allocte local
>> + port for domU2 to send an event notification to the remote domain.
>> +
>> + domU2-phandle is a single phandle to an domain to which local-port-domU2
>> + will be allocated.
>> +
>> +Example:
>> +
>> + chosen {
>> + ....
>> +
>> + domU1: domU1 {
>> + ......
>> + };
>> +
>> + domU2: domU2 {
>> + ......
>> + };
>> +
>> + evtchn@1 {
>> + compatible = "xen,evtchn";
>> + xen,evtchn = <0xa &domU1 0xb &domU2>;
>> + };
>> +
>> + evtchn@2 {
>> + compatible = "xen,evtchn";
>> + xen,evtchn = <0xc &domU1 0xd &domU2>;
>> + };
>> + };
>
> There is no need to use two evtchn nodes for this given that in device
> tree it is entirely normal to have multiple tuplets in a property. Also,
> it would be good to have a version number in the compatible string so
> that we can change version in the future.
>
> 1)
> chosen {
> ....
>
> domU1: domU1 {
> ......
> };
>
> domU2: domU2 {
> ......
> };
>
> evtchn {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xa &domU1 0xb &domU2 0xc &domU1 0xd &domU2>;
> };
> };
>
I agree if we can have multiple tuples in a property. I will modify the design in next version
to have multiple tuples in a property.
>
> I should mention that it would be also possible to use sub-nodes to
> express this information:
>
> 2)
> domU1: domU1 {
> ...
> /* one sub-node per local event channel */
> ec1: evtchn@a {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0xa &ec3>
> };
> ec2: evtchn@c {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xc &ec4>
> };
> };
>
> domU2: domU2 {
> ...
> ec3: evtchn@b {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xb &ec1>
> };
> ec4: evtchn@d {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xa &ec2>
> };
> };
> };
>
> This format has the advantage that doesn't need a new top-level node
> type under /chosen. That is desirable few a few reasons. Today we only
> have domains (dom0 is legacy). In the future we might have nested
> domains and non-Xen domains. With System Device Tree, domains are under
> /domains instead of /chosen.
>
> So normally I would argue to use the sub-node format because it doesn't
> need a new top-level node under /chosen. However, in this case it looks
> like the 1) format is simpler to write and also simpler to parse in Xen.
>
> In 1), we would need to loop over xen,evtchn and for each tuplet we
> would only need to fetch the foreign domid.
>
> In 2), we would need to check the compatible string of every
> "xen,evtchn-v1" node, and we would have to fetch from the phandle both
> the remote event channel number but also the domain-id of the parent.
>
> So it looks like 1) is better because it is much simpler to parse. Do
> you agree?
Yes I agree with you, for this case we need to parse all the "xen,evtchn-v1” compatible node
and from that node, we need to find the remote event channel and dom-id from the phandle.
I started from this configuration and later realize that if we use this configuration code will become
more complex and defining the event-channel connection in DT will also not be simple.
>
>> +In above example two event channel comunication will be established between
>> +domU1 and domU2.
>> +
>> + domU1 (port 0xa) <-----------------> domU2 (port 0xb)
>> + domU1 (port 0xc) <-----------------> domU2 (port 0xd)
>> +
>> +domU1 and domU2 can send the signal to remote domain via hypercall
>> +EVTCHNOP_send(.) on local port.
>
> I think this is fine in principle. Like Jan wrote, at some point we'll
> need to specify the device tree binding to expose this information to
> the guest.
Regards,
Rahul
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-09 1:00 ` Stefano Stabellini
2022-04-09 1:44 ` Stefano Stabellini
2022-04-11 15:28 ` Rahul Singh
@ 2022-04-11 18:16 ` Julien Grall
2022-04-12 20:39 ` Stefano Stabellini
2 siblings, 1 reply; 21+ messages in thread
From: Julien Grall @ 2022-04-11 18:16 UTC (permalink / raw)
To: Stefano Stabellini, Rahul Singh
Cc: xen-devel, bertrand.marquis, Andrew Cooper, George Dunlap,
Jan Beulich, Wei Liu
Hi,
On 09/04/2022 02:00, Stefano Stabellini wrote:
> On Wed, 23 Mar 2022, Rahul Singh wrote:
>> in dom0less system. This patch introduce the new feature to support the
>> signaling between two domUs in dom0less system.
>>
>> Signed-off-by: Rahul Singh <rahul.singh@arm.com>
>> ---
>> docs/designs/dom0less-evtchn.md | 96 +++++++++++++++++++++++++++++++++
>> 1 file changed, 96 insertions(+)
>> create mode 100644 docs/designs/dom0less-evtchn.md
>>
>> diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
>> new file mode 100644
>> index 0000000000..6a1b7e8c22
>> --- /dev/null
>> +++ b/docs/designs/dom0less-evtchn.md
>> @@ -0,0 +1,96 @@
>> +# Signaling support between two domUs on dom0less system
>> +
>> +## Current state: Draft version
>> +
>> +## Proposer(s): Rahul Singh, Bertrand Marquis
>> +
>> +## Problem Statement:
>> +
>> +The goal of this work is to define a simple signaling system between Xen guests
>> +in dom0less systems.
>> +
>> +In dom0less system, we cannot make use of xenbus and xenstore that are used in
>> +normal systems with dynamic VMs to communicate between domains by providing a
>> +bus abstraction for paravirtualized drivers.
>> +
>> +One possible solution to implement the signaling system between domUs is based
>> +on event channels.
>
> I suggest to reword this as follows:
>
> ---
> Dom0less guests would benefit from a statically-defined memory sharing
> and signally system for communication. One that would be immediately
> available at boot without any need for dynamic configurations.
>
> In embedded a great variety of guest operating system kernels exist,
> many of which don't have support for xenstore, grant table or other
> complex drivers. Some of them are small kernel-space applications (often
> called "baremetal", not to be confused with the term "baremetal" used in
> datacenter which means "without hypervisors") or RTOSes. Additionally,
> for safety reasons, users often need to be able to configure the full
> system statically so that it can be verified statically.
>
> Event channels are very simple and can be added even to baremetal
> applications. This proposal introduces a way to define them statically
> to make them suitable to dom0less embedded deployments.
> ---
>
>
>> +## Proposal:
>> +
>> +Event channels are the basic primitive provided by Xen for event notifications.
>> +An event channel is a logical connection between 2 domains (more specifically
>> +between dom1,port1 and dom2,port2). They essentially store one bit of
>> +information, the event of interest is signalled by transitioning this bit from
>> +0 to 1. An event is an equivalent of a hardware interrupt.
>> +
>> +Notifications are received by a guest via an interrupt from Xen to the guest,
>> +indicating when an event arrives (setting the bit). Further notifications are
>> +masked until the bit is cleared again. When a domain wants to wait for data it
>> +will block until an event arrives, and then send an event to signal that data
>> +has been consumed. Events are delivered asynchronously to guests and are
>> +enqueued when the guest is not running.
>> +
>> +Event channel communication will be established statically between two domU
>> +guests before unpausing the domains after domain creation. Event channel
>> +connection information between domUs will be passed to XEN via device tree
>> +node.
>> +
>> +Under the /chosen node, there needs to be sub nodes with compatible
>> +"xen,evtchn" that descibes the event channel connection between two domUs.
>> +
>> +The event channel sub-node has the following properties:
>> +
>> +- compatible
>> +
>> + "xen,evtchn"
>> +
>> +- xen,evtchn
>> +
>> + The property is four numbers of tuples of
>> + (local-port-domU1,domU1-phandle,local-port-domU2,domU2-phandle) where:
>> +
>> + local-port-domU1 is an integer value that will be used to allocte local
>> + port for domU1 to send an event notification to the remote domain.
>> +
>> + domU1-phandle is a single phandle to an domain to which local-port-domU1
>> + will be allocated.
>> +
>> + local-port-domU2 is an integer value that will be used to allocte local
>> + port for domU2 to send an event notification to the remote domain.
>> +
>> + domU2-phandle is a single phandle to an domain to which local-port-domU2
>> + will be allocated.
>> +
>> +Example:
>> +
>> + chosen {
>> + ....
>> +
>> + domU1: domU1 {
>> + ......
>> + };
>> +
>> + domU2: domU2 {
>> + ......
>> + };
>> +
>> + evtchn@1 {
>> + compatible = "xen,evtchn";
>> + xen,evtchn = <0xa &domU1 0xb &domU2>;
>> + };
>> +
>> + evtchn@2 {
>> + compatible = "xen,evtchn";
>> + xen,evtchn = <0xc &domU1 0xd &domU2>;
>> + };
>> + };
>
> There is no need to use two evtchn nodes for this given that in device
> tree it is entirely normal to have multiple tuplets in a property. Also,
> it would be good to have a version number in the compatible string so
> that we can change version in the future.
>
> 1)
> chosen {
> ....
>
> domU1: domU1 {
> ......
> };
>
> domU2: domU2 {
> ......
> };
>
> evtchn {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xa &domU1 0xb &domU2 0xc &domU1 0xd &domU2>;
> };
> };
>
>
> I should mention that it would be also possible to use sub-nodes to
> express this information:
>
> 2)
> domU1: domU1 {
> ...
> /* one sub-node per local event channel */
> ec1: evtchn@a {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign-evtchn */
> xen,evtchn = <0xa &ec3>
> };
> ec2: evtchn@c {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xc &ec4>
> };
> };
>
> domU2: domU2 {
> ...
> ec3: evtchn@b {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xb &ec1>
> };
> ec4: evtchn@d {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xa &ec2>
> };
> };
> };
As for 1), you could combine all the ports in one node.
>
> This format has the advantage that doesn't need a new top-level node
> type under /chosen. That is desirable few a few reasons. Today we only
> have domains (dom0 is legacy). In the future we might have nested
> domains and non-Xen domains. With System Device Tree, domains are under
> /domains instead of /chosen.
>
> So normally I would argue to use the sub-node format because it doesn't
> need a new top-level node under /chosen. However, in this case it looks
> like the 1) format is simpler to write and also simpler to parse in Xen.
I am not sure for both. For the writing part: In one hand, it is nice to
have all the event channels defined in one place. One the other hand, it
is more messy if you want to visually check that you don't duplicate the
event channels.
It may also end up to be complex to read if you have many events
channels. So if we go with 1), I think we would want to allow multiple
nodes to help the user to keep a clean/readable DT.
>
> In 1), we would need to loop over xen,evtchn and for each tuplet we
> would only need to fetch the foreign domid.
You will need to loop over all the nodes in chosen to find "xen,evtchn"
and also fetch two phandles.
>
> In 2), we would need to check the compatible string of every
> "xen,evtchn-v1" node, and we would have to fetch from the phandle both
> the remote event channel number but also the domain-id of the parent.
>
> So it looks like 1) is better because it is much simpler to parse. Do
> you agree?
See above, I am not sure the code to parse will end up to be much bigger
because we would still need to loop on the nodes in chosen and fetch two
phandles.
That said, we are potentially going to need to loop on more nodes.
So overall, I am 50/50 on which one to chose.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-11 18:16 ` Julien Grall
@ 2022-04-12 20:39 ` Stefano Stabellini
2022-04-13 9:51 ` Julien Grall
0 siblings, 1 reply; 21+ messages in thread
From: Stefano Stabellini @ 2022-04-12 20:39 UTC (permalink / raw)
To: Julien Grall
Cc: Stefano Stabellini, Rahul Singh, xen-devel, bertrand.marquis,
Andrew Cooper, George Dunlap, Jan Beulich, Wei Liu
[-- Attachment #1: Type: text/plain, Size: 8055 bytes --]
On Mon, 11 Apr 2022, Julien Grall wrote:
> On 09/04/2022 02:00, Stefano Stabellini wrote:
> > On Wed, 23 Mar 2022, Rahul Singh wrote:
> > > in dom0less system. This patch introduce the new feature to support the
> > > signaling between two domUs in dom0less system.
> > >
> > > Signed-off-by: Rahul Singh <rahul.singh@arm.com>
> > > ---
> > > docs/designs/dom0less-evtchn.md | 96 +++++++++++++++++++++++++++++++++
> > > 1 file changed, 96 insertions(+)
> > > create mode 100644 docs/designs/dom0less-evtchn.md
> > >
> > > diff --git a/docs/designs/dom0less-evtchn.md
> > > b/docs/designs/dom0less-evtchn.md
> > > new file mode 100644
> > > index 0000000000..6a1b7e8c22
> > > --- /dev/null
> > > +++ b/docs/designs/dom0less-evtchn.md
> > > @@ -0,0 +1,96 @@
> > > +# Signaling support between two domUs on dom0less system
> > > +
> > > +## Current state: Draft version
> > > +
> > > +## Proposer(s): Rahul Singh, Bertrand Marquis
> > > +
> > > +## Problem Statement:
> > > +
> > > +The goal of this work is to define a simple signaling system between Xen
> > > guests
> > > +in dom0less systems.
> > > +
> > > +In dom0less system, we cannot make use of xenbus and xenstore that are
> > > used in
> > > +normal systems with dynamic VMs to communicate between domains by
> > > providing a
> > > +bus abstraction for paravirtualized drivers.
> > > +
> > > +One possible solution to implement the signaling system between domUs is
> > > based
> > > +on event channels.
> >
> > I suggest to reword this as follows:
> >
> > ---
> > Dom0less guests would benefit from a statically-defined memory sharing
> > and signally system for communication. One that would be immediately
> > available at boot without any need for dynamic configurations.
> >
> > In embedded a great variety of guest operating system kernels exist,
> > many of which don't have support for xenstore, grant table or other
> > complex drivers. Some of them are small kernel-space applications (often
> > called "baremetal", not to be confused with the term "baremetal" used in
> > datacenter which means "without hypervisors") or RTOSes. Additionally,
> > for safety reasons, users often need to be able to configure the full
> > system statically so that it can be verified statically.
> >
> > Event channels are very simple and can be added even to baremetal
> > applications. This proposal introduces a way to define them statically
> > to make them suitable to dom0less embedded deployments.
> > ---
> >
> >
> > > +## Proposal:
> > > +
> > > +Event channels are the basic primitive provided by Xen for event
> > > notifications.
> > > +An event channel is a logical connection between 2 domains (more
> > > specifically
> > > +between dom1,port1 and dom2,port2). They essentially store one bit of
> > > +information, the event of interest is signalled by transitioning this bit
> > > from
> > > +0 to 1. An event is an equivalent of a hardware interrupt.
> > > +
> > > +Notifications are received by a guest via an interrupt from Xen to the
> > > guest,
> > > +indicating when an event arrives (setting the bit). Further notifications
> > > are
> > > +masked until the bit is cleared again. When a domain wants to wait for
> > > data it
> > > +will block until an event arrives, and then send an event to signal that
> > > data
> > > +has been consumed. Events are delivered asynchronously to guests and are
> > > +enqueued when the guest is not running.
> > > +
> > > +Event channel communication will be established statically between two
> > > domU
> > > +guests before unpausing the domains after domain creation. Event channel
> > > +connection information between domUs will be passed to XEN via device
> > > tree
> > > +node.
> > > +
> > > +Under the /chosen node, there needs to be sub nodes with compatible
> > > +"xen,evtchn" that descibes the event channel connection between two
> > > domUs.
> > > +
> > > +The event channel sub-node has the following properties:
> > > +
> > > +- compatible
> > > +
> > > + "xen,evtchn"
> > > +
> > > +- xen,evtchn
> > > +
> > > + The property is four numbers of tuples of
> > > + (local-port-domU1,domU1-phandle,local-port-domU2,domU2-phandle)
> > > where:
> > > +
> > > + local-port-domU1 is an integer value that will be used to allocte
> > > local
> > > + port for domU1 to send an event notification to the remote domain.
> > > +
> > > + domU1-phandle is a single phandle to an domain to which
> > > local-port-domU1
> > > + will be allocated.
> > > +
> > > + local-port-domU2 is an integer value that will be used to allocte
> > > local
> > > + port for domU2 to send an event notification to the remote domain.
> > > +
> > > + domU2-phandle is a single phandle to an domain to which
> > > local-port-domU2
> > > + will be allocated.
> > > +
> > > +Example:
> > > +
> > > + chosen {
> > > + ....
> > > +
> > > + domU1: domU1 {
> > > + ......
> > > + };
> > > +
> > > + domU2: domU2 {
> > > + ......
> > > + };
> > > +
> > > + evtchn@1 {
> > > + compatible = "xen,evtchn";
> > > + xen,evtchn = <0xa &domU1 0xb &domU2>;
> > > + };
> > > +
> > > + evtchn@2 {
> > > + compatible = "xen,evtchn";
> > > + xen,evtchn = <0xc &domU1 0xd &domU2>;
> > > + };
> > > + };
> >
> > There is no need to use two evtchn nodes for this given that in device
> > tree it is entirely normal to have multiple tuplets in a property. Also,
> > it would be good to have a version number in the compatible string so
> > that we can change version in the future.
> >
> > 1)
> > chosen {
> > ....
> >
> > domU1: domU1 {
> > ......
> > };
> >
> > domU2: domU2 {
> > ......
> > };
> >
> > evtchn {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0xa &domU1 0xb &domU2 0xc &domU1 0xd &domU2>;
> > };
> > };
> >
> >
> > I should mention that it would be also possible to use sub-nodes to
> > express this information:
> >
> > 2)
> > domU1: domU1 {
> > ...
> > /* one sub-node per local event channel */
> > ec1: evtchn@a {
> > compatible = "xen,evtchn-v1";
> > /* local-evtchn link-to-foreign-evtchn */
> > xen,evtchn = <0xa &ec3>
> > };
> > ec2: evtchn@c {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0xc &ec4>
> > };
> > };
> >
> > domU2: domU2 {
> > ...
> > ec3: evtchn@b {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0xb &ec1>
> > };
> > ec4: evtchn@d {
> > compatible = "xen,evtchn-v1";
> > xen,evtchn = <0xd &ec2>
> > };
> > };
> > };
>
> As for 1), you could combine all the ports in one node.
I thought about it but I couldn't come up with a way to do that which
retains the simplicity of this example. The problem is that in device
tree you can only link to nodes, not to individual properties. So I
think we would have to have separate nodes for each event channel so
that we could separately link to them.
Otherwise, we would have to add the foreign event channel number in
addition to the link to be able to distinguish them. And that would
result in duplicated information. E.g.:
domU1: domU1 {
...
/* one sub-node per local event channel */
ec1: evtchn@a {
compatible = "xen,evtchn-v1";
/* local-evtchn link-to-foreign foreign-evtchn */
xen,evtchn = <0xa &ec2 0xa 0xc &ec2 0xd>
};
};
domU2: domU2 {
...
ec2: evtchn@b {
compatible = "xen,evtchn-v1";
xen,evtchn = <0xb &ec1 0xa 0xd &ec1 0xc>
};
};
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..
2022-04-12 20:39 ` Stefano Stabellini
@ 2022-04-13 9:51 ` Julien Grall
0 siblings, 0 replies; 21+ messages in thread
From: Julien Grall @ 2022-04-13 9:51 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Rahul Singh, xen-devel, bertrand.marquis, Andrew Cooper,
George Dunlap, Jan Beulich, Wei Liu
Hi Stefano,
On 12/04/2022 21:39, Stefano Stabellini wrote:
>>> I should mention that it would be also possible to use sub-nodes to
>>> express this information:
>>>
>>> 2)
>>> domU1: domU1 {
>>> ...
>>> /* one sub-node per local event channel */
>>> ec1: evtchn@a {
>>> compatible = "xen,evtchn-v1";
>>> /* local-evtchn link-to-foreign-evtchn */
>>> xen,evtchn = <0xa &ec3>
>>> };
>>> ec2: evtchn@c {
>>> compatible = "xen,evtchn-v1";
>>> xen,evtchn = <0xc &ec4>
>>> };
>>> };
>>>
>>> domU2: domU2 {
>>> ...
>>> ec3: evtchn@b {
>>> compatible = "xen,evtchn-v1";
>>> xen,evtchn = <0xb &ec1>
>>> };
>>> ec4: evtchn@d {
>>> compatible = "xen,evtchn-v1";
>>> xen,evtchn = <0xd &ec2>
>>> };
>>> };
>>> };
>>
>> As for 1), you could combine all the ports in one node.
>
> I thought about it but I couldn't come up with a way to do that which
> retains the simplicity of this example. The problem is that in device
> tree you can only link to nodes, not to individual properties. So I
> think we would have to have separate nodes for each event channel so
> that we could separately link to them.
Ah yes. I overlooked it when making the suggestion. That said, I think
my point in my previous e-mail stands.
If this is too verbose, then we could provide macro to generate the
event channel node.
>
> Otherwise, we would have to add the foreign event channel number in
> addition to the link to be able to distinguish them. And that would
> result in duplicated information. E.g.:
This is not indeed not great.
>
> domU1: domU1 {
> ...
> /* one sub-node per local event channel */
> ec1: evtchn@a {
> compatible = "xen,evtchn-v1";
> /* local-evtchn link-to-foreign foreign-evtchn */
> xen,evtchn = <0xa &ec2 0xa 0xc &ec2 0xd>
> };
> };
>
> domU2: domU2 {
> ...
> ec2: evtchn@b {
> compatible = "xen,evtchn-v1";
> xen,evtchn = <0xb &ec1 0xa 0xd &ec1 0xc>
> };
> };
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 21+ messages in thread