Re: Using DT overlays for adding virtual hardware

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2016-06-08 18:31, Pantelis Antoniou wrote:
> Hi Mark,
> 
>> On Jun 8, 2016, at 19:23 , Mark Rutland <mark.rutland@arm.com> wrote:
>>
>> On Wed, Jun 08, 2016 at 06:57:37PM +0300, Pantelis Antoniou wrote:
>>> Hi Mark,
>>>
>>>> On Jun 8, 2016, at 18:17 , Mark Rutland <mark.rutland@arm.com> wrote:
>>>>
>>>> On Wed, Jun 08, 2016 at 04:16:32PM +0200, Jan Kiszka wrote:
>>>>> Hi all,
>>>>>
>>>>> already started the discussion off-list with Pantelis, but it's better
>>>>> done in public:
>>>>>
>>>>> I'm currently exploring ways to make Linux recognize dynamically added
>>>>> virtual hardware when running under the Jailhouse hypervisor [1]. We
>>>>> need to load drivers for inter-partition communication devices that only
>>>>> appear after Jailhouse started (which is done from within Linux, i.e.
>>>>> long after boot) or when a partition was added later on. Probably, we
>>>>> will simply add a virtual PCI host bridge on systems without physical
>>>>> PCI and let the IPC device be explored that way (already works on x86).
>>>>> Still, that leaves us with hotplug and unplug on hypervisor activation
>>>>> and deactivation.
>>>>
>>>> If I've understood correctly you want to use overlays to inject the
>>>> virtual PCI host bridge?
>>>>
>>>> Given that you know precisely what you want to inject, I'm not sure I
>>>> see the value of using an overlay. 
>>>>
>>>> Is there some reason you can't just create a device without having to go
>>>> via an intermediate step? As I understand it, Xen does that for (some)
>>>> virtual devices provided to Dom0 and DomU.
>>>
>>> As far as I understand it PCI is just one of the cases. You could conceivably
>>> inject any kind of virtio device like serial/storage networking etc.
>>
>> Sure, but we already have PCI transport for virtio devices, and per the
>> above PCI is the transport used on x86, so I assume that the devices we
>> really care about are going to be PCI anyhow.
>>
> 
> PCI on VMs is a hack, it’s all emulated.
> 
> We’re using it as crutch because it’s ubiquitous and is capable
> of probing, but it comes with a considerable amount of baggage.
> Jailhouse is a particular kind of a hypervisor where it is intended for
> safety critical applications and designed to be certified as such.
> The less amount of code it contains the better, and much easier to certify. 

That's true, but we already have to live with PCI on x86, thus the code
is there, and it's more and more present on ARM[64] as well.

We are trying hard to make it simple, primarily for the hypervisor, and
if it is simpler to plug a trivial virtual PCI bridge than to emulate
individual platform IPC devices - also fine.

> 
>>> The question is since overlays exist and do work, why should he do anything else
>>> besides using them?
>>
>> For one thing, they only work with DT, and there are ACPI ARM server
>> platforms out there, for which people may wish to use jailhouse. Tying
>> this to DT is not necessarily the best idea.
>>
> 
> I just don’t see how an ACPI based hypervisor can ever be certified for
> safety critical applications. It might be possible but it should be
> an enormous undertaking; perhaps a subset without AML, but then again
> can you even boot an ACPI box without it?

ACPI is out of scope for us. We will probably continue to feed the
hypervisor with static platform information, generated in advance and
validated. Can be DT-based one day, but even that is more complex to
parse than our current structures.

But does ACPI usually mean that the kernel no longer has DT support and
would not be able to handle any overlay? That could be a killer.

> 
> DT is safer since it contains state only.
> 
>> To be clear, I'm not arguing *against* overlays as such, just making
>> sure that we're not prematurely choosing a solution just becasue it's
>> the one we're aware of.

I'm open for any suggestion that is simple. Maybe we can extend a
trivial existing pci host driver (like pci-host-generic) to work also
without DT overlays - also fine, at least from Jailhose POV. However,
any unneeded kernel patch is even better.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux