* About creating machines on the command line
@ 2021-01-11 14:50 Luc Michel
2021-01-11 20:04 ` BALATON Zoltan
` (2 more replies)
0 siblings, 3 replies; 16+ messages in thread
From: Luc Michel @ 2021-01-11 14:50 UTC (permalink / raw)
To: qemu-devel
Cc: Damien Hedde, peter.maydell, richard.fuhler, berrange, sam.grove,
Mark Burton, f4bug, armbru, edgar.iglesias
Hi,
We would like to work on improving QEMU to be able to create custom
machines from the command line. The goal here is to get feedback from
the community and shape the future developments.
The use case mainly comes from people working with tools to customize
their designs, such as SiFive Core Designer
(https://scs.sifive.com/core-designer). This kind of tools may allow
creation or customization of a whole SoC, from the number of cores, to
the memory and IRQ mapping of peripherals etc.
The ultimate goal would be to be able to create any kind of machine on
the command line. However we are aware that this is a substantial amount
of changes in QEMU.
In its current state, QEMU allows for very limited customization of
existing machines on the command line. We identified the following
limitations (feel free to add to the list):
- Most devices are not user creatable. Moreover, sysbus devices must
be explicitly allowed by a machine to be creatable through `-device`,
- Memory regions cannot be created on the command line,
- Device MMIO regions cannot be mapped on a bus from the command line,
- GPIOs and clocks cannot be wired from the command line,
- CPUs are not sysbus devices (and not user-creatable). They need
special care when creating them regarding system reset. Not being on a
bus means that they must be reset manually on system reset. This is done
in machines by registering a QEMU reset handler.
- Machine specific boot code is usually hard-coded into the machine
itself. Some architectures (e.g. ARM) do factorize bootloader related
code, but there is no standard way of doing that in QEMU.
We don't want to address all those limitations at once. We plan to start
with the following scenario:
- Start with a base machine that would handle CPU creation and
bootloader stuff. Note that the "none" machine is probably not
sufficient in its current shape. It does allow only one CPU and
obviously does not handle the boot process.
- Allow for this machine every sysbus devices we want to be user
command-line creatable (and mark them user_creatable if needed)
- Add command line options to create memory regions (probably ram
ones at first)
- Add command line options to map a memory region (including sysbus
device MMIO regions) onto another (memory_region_add_subregion)
- Add command line options to connect GPIOs and clocks.
This would hopefully allow for simple machines creation. We would then
be able to use either the command line or the `-readconfig` option to
create the machine.
Note that we are not planning to use QMP/HMP for now. From our
understanding, a `device_add` request is always considered as hot-plug,
which is not what we want here.
Please tell us what do you think about this plan. Any feedback is
appreciated. Then we can discuss the details of how to do this properly.
Thanks!
--
Luc
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-11 14:50 About creating machines on the command line Luc Michel
@ 2021-01-11 20:04 ` BALATON Zoltan
2021-01-11 20:28 ` Liviu Ionescu
2021-01-14 9:30 ` Luc Michel
2021-01-14 11:37 ` Daniel P. Berrangé
2021-02-03 16:55 ` Philippe Mathieu-Daudé
2 siblings, 2 replies; 16+ messages in thread
From: BALATON Zoltan @ 2021-01-11 20:04 UTC (permalink / raw)
To: Luc Michel
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, armbru,
Mark Burton, qemu-devel, richard.fuhler, edgar.iglesias, f4bug
On Mon, 11 Jan 2021, Luc Michel wrote:
> Hi,
>
> We would like to work on improving QEMU to be able to create custom machines
> from the command line. The goal here is to get feedback from the community
> and shape the future developments.
>
> The use case mainly comes from people working with tools to customize their
> designs, such as SiFive Core Designer (https://scs.sifive.com/core-designer).
> This kind of tools may allow creation or customization of a whole SoC, from
> the number of cores, to the memory and IRQ mapping of peripherals etc.
>
> The ultimate goal would be to be able to create any kind of machine on the
> command line. However we are aware that this is a substantial amount of
> changes in QEMU.
>
> In its current state, QEMU allows for very limited customization of existing
> machines on the command line. We identified the following limitations (feel
> free to add to the list):
>
> - Most devices are not user creatable. Moreover, sysbus devices must be
> explicitly allowed by a machine to be creatable through `-device`,
>
> - Memory regions cannot be created on the command line,
>
> - Device MMIO regions cannot be mapped on a bus from the command line,
>
> - GPIOs and clocks cannot be wired from the command line,
>
> - CPUs are not sysbus devices (and not user-creatable). They need special
> care when creating them regarding system reset. Not being on a bus means that
> they must be reset manually on system reset. This is done in machines by
> registering a QEMU reset handler.
>
> - Machine specific boot code is usually hard-coded into the machine itself.
> Some architectures (e.g. ARM) do factorize bootloader related code, but there
> is no standard way of doing that in QEMU.
>
> We don't want to address all those limitations at once. We plan to start with
> the following scenario:
>
> - Start with a base machine that would handle CPU creation and bootloader
> stuff. Note that the "none" machine is probably not sufficient in its current
> shape. It does allow only one CPU and obviously does not handle the boot
> process.
>
> - Allow for this machine every sysbus devices we want to be user
> command-line creatable (and mark them user_creatable if needed)
>
> - Add command line options to create memory regions (probably ram ones at
> first)
>
> - Add command line options to map a memory region (including sysbus device
> MMIO regions) onto another (memory_region_add_subregion)
>
> - Add command line options to connect GPIOs and clocks.
I'm not sure the command line would be the best interface for this, it's
already hard enough to get a simple VM running, adding more options would
just make it more obscure for mere users who just want to use existing
machines to create a VM. So I think some kind of config file to define the
machine would be better.
Another issue with command line was backward compatibility that it's hard
to change if it's used as an interface in scripts, etc. My personal
opinion is that the command line should be a user interface to run VMs but
for management apps and this kind of dynamic machine creation a better
defined API might be better. But I'm not in any way authorative to decide
so this is just my input to this.
> This would hopefully allow for simple machines creation. We would then be
> able to use either the command line or the `-readconfig` option to create the
> machine.
It was found that readconfig is limited and cannot even handle its current
purpose so it probably would need to be rewritten. This might be a good
opportunity for that. Maybe you need a better format than ini file that
can actually describe the machine parts and their connections you want to
create in a managable way. Question arises how general this has to be to
describe all the properties of objects and their connections and if you'll
end up with somthing like still needing to write a board code which is the
current situation just instead of writing C you'd use some other language
and call it a "config file". What do you really want here? Avoid writing C
or avoid recompiling QEMU to experiment with devices? In the latter case
better support from the QEMU monitor to create machines interactively
could also be a solution and if you could run scripts with monitor
commands in it that could be your config file.
> Note that we are not planning to use QMP/HMP for now. From our understanding,
> a `device_add` request is always considered as hot-plug, which is not what we
> want here.
Is that something that cannot be solved? Maybe by adding another command
or a state where starting with -S and adding devices before continue would
not be hotplug?
These are just some random thoughts I had, feel free to ignore any of it.
Regards,
BALATON Zoltan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-11 20:04 ` BALATON Zoltan
@ 2021-01-11 20:28 ` Liviu Ionescu
2021-01-14 10:56 ` Luc Michel
2021-01-14 9:30 ` Luc Michel
1 sibling, 1 reply; 16+ messages in thread
From: Liviu Ionescu @ 2021-01-11 20:28 UTC (permalink / raw)
To: BALATON Zoltan
Cc: Damien Hedde, Peter Maydell, richard.fuhler,
"Daniel P. Berrangé",
sam.grove, Mark Burton, Markus Armbruster, QEMU Developers,
edgar.iglesias, Luc Michel, f4bug
I know it does not answer your question directly, but you might find useful to take a look at the solution used in the xPack QEMU Arm (formerly GNU ARM Eclipse QEMU).
In this fork I dynamically create the peripheral registers when QEMU starts, by reading a JSON file, converted from the CMSIS SVD files.
Unfortunately this is not enough, since the actions associated with each register are not defined in these JSONs, and need to be hardcoded in code.
But I think that the mechanism to create peripherals and registers might be adapted to your use case too.
Regards,
Liviu
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-11 20:04 ` BALATON Zoltan
2021-01-11 20:28 ` Liviu Ionescu
@ 2021-01-14 9:30 ` Luc Michel
1 sibling, 0 replies; 16+ messages in thread
From: Luc Michel @ 2021-01-14 9:30 UTC (permalink / raw)
To: BALATON Zoltan
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, armbru,
Mark Burton, qemu-devel, richard.fuhler, edgar.iglesias, f4bug
Hi Zoltan,
On 1/11/21 9:04 PM, BALATON Zoltan wrote:
> On Mon, 11 Jan 2021, Luc Michel wrote:
>> Hi,
>>
>> We would like to work on improving QEMU to be able to create custom
>> machines from the command line. The goal here is to get feedback from
>> the community and shape the future developments.
>>
>> The use case mainly comes from people working with tools to customize
>> their designs, such as SiFive Core Designer
>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>> creation or customization of a whole SoC, from the number of cores, to
>> the memory and IRQ mapping of peripherals etc.
>>
>> The ultimate goal would be to be able to create any kind of machine on
>> the command line. However we are aware that this is a substantial
>> amount of changes in QEMU.
>>
>> In its current state, QEMU allows for very limited customization of
>> existing machines on the command line. We identified the following
>> limitations (feel free to add to the list):
>>
>> - Most devices are not user creatable. Moreover, sysbus devices must
>> be explicitly allowed by a machine to be creatable through `-device`,
>>
>> - Memory regions cannot be created on the command line,
>>
>> - Device MMIO regions cannot be mapped on a bus from the command line,
>>
>> - GPIOs and clocks cannot be wired from the command line,
>>
>> - CPUs are not sysbus devices (and not user-creatable). They need
>> special care when creating them regarding system reset. Not being on a
>> bus means that they must be reset manually on system reset. This is
>> done in machines by registering a QEMU reset handler.
>>
>> - Machine specific boot code is usually hard-coded into the machine
>> itself. Some architectures (e.g. ARM) do factorize bootloader related
>> code, but there is no standard way of doing that in QEMU.
>>
>> We don't want to address all those limitations at once. We plan to
>> start with the following scenario:
>>
>> - Start with a base machine that would handle CPU creation and
>> bootloader stuff. Note that the "none" machine is probably not
>> sufficient in its current shape. It does allow only one CPU and
>> obviously does not handle the boot process.
>>
>> - Allow for this machine every sysbus devices we want to be user
>> command-line creatable (and mark them user_creatable if needed)
>>
>> - Add command line options to create memory regions (probably ram
>> ones at first)
>>
>> - Add command line options to map a memory region (including sysbus
>> device MMIO regions) onto another (memory_region_add_subregion)
>>
>> - Add command line options to connect GPIOs and clocks.
>
> I'm not sure the command line would be the best interface for this, it's
> already hard enough to get a simple VM running, adding more options
> would just make it more obscure for mere users who just want to use
> existing machines to create a VM. So I think some kind of config file to
> define the machine would be better.
OK. My first thought here was to be not too much "intrusive" in the QEMU
source code, so adding a couple a command line options seemed to be the
way to go, as -device was already doing what we wanted.
>
> Another issue with command line was backward compatibility that it's
> hard to change if it's used as an interface in scripts, etc. My personal
> opinion is that the command line should be a user interface to run VMs
> but for management apps and this kind of dynamic machine creation a
> better defined API might be better. But I'm not in any way authorative
> to decide so this is just my input to this.
Makes sense. I tend to agree with you. I'd appreciate more feedback on
this to know if it's indeed a shared opinion.
>
>> This would hopefully allow for simple machines creation. We would then
>> be able to use either the command line or the `-readconfig` option to
>> create the machine.
>
> It was found that readconfig is limited and cannot even handle its
> current purpose so it probably would need to be rewritten. This might be
> a good opportunity for that. Maybe you need a better format than ini
> file that can actually describe the machine parts and their connections
> you want to create in a managable way. Question arises how general this
> has to be to describe all the properties of objects and their
> connections and if you'll end up with somthing like still needing to
> write a board code which is the current situation just instead of
> writing C you'd use some other language and call it a "config file".
> What do you really want here? Avoid writing C or avoid recompiling QEMU
> to experiment with devices?
Yes the idea is definitely to have a mean to create custom machines
without the need of recompiling QEMU.
> In the latter case better support from the
> QEMU monitor to create machines interactively could also be a solution
> and if you could run scripts with monitor commands in it that could be
> your config file.
Yes. We plan to have a script that will serve as a "frontend" anyways to
generate the necessary commands (being on the command line, or monitor
command or...)
>
>> Note that we are not planning to use QMP/HMP for now. From our
>> understanding, a `device_add` request is always considered as
>> hot-plug, which is not what we want here.
>
> Is that something that cannot be solved? Maybe by adding another command
> or a state where starting with -S and adding devices before continue
> would not be hotplug?
Sure. The goal here was to avoid adding yet another run state to QEMU.
The command line happened to kind of support what we plan to do. If
people are OK with enhancing the monitor for this, going the QMP way
could be a solution. Does anyone has an opinion on this?
>
> These are just some random thoughts I had, feel free to ignore any of it.
Thanks for your feedback!
>
> Regards,
> BALATON Zoltan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-11 20:28 ` Liviu Ionescu
@ 2021-01-14 10:56 ` Luc Michel
2021-01-14 11:07 ` Liviu Ionescu
0 siblings, 1 reply; 16+ messages in thread
From: Luc Michel @ 2021-01-14 10:56 UTC (permalink / raw)
To: Liviu Ionescu, BALATON Zoltan
Cc: Damien Hedde, Peter Maydell, richard.fuhler,
Daniel P. Berrangé,
sam.grove, Mark Burton, Markus Armbruster, QEMU Developers,
edgar.iglesias, f4bug
Hi Liviu,
On 1/11/21 9:28 PM, Liviu Ionescu wrote:
> I know it does not answer your question directly, but you might find useful to take a look at the solution used in the xPack QEMU Arm (formerly GNU ARM Eclipse QEMU).
>
> In this fork I dynamically create the peripheral registers when QEMU starts, by reading a JSON file, converted from the CMSIS SVD files.
>
> Unfortunately this is not enough, since the actions associated with each register are not defined in these JSONs, and need to be hardcoded in code.
>
>
> But I think that the mechanism to create peripherals and registers might be adapted to your use case too.
Thanks, I'll have a look. But yes the goal here is to rely on QEMU's
already existing device models to dynamically create a machine.
Luc
>
> Regards,
>
> Liviu
>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-14 10:56 ` Luc Michel
@ 2021-01-14 11:07 ` Liviu Ionescu
0 siblings, 0 replies; 16+ messages in thread
From: Liviu Ionescu @ 2021-01-14 11:07 UTC (permalink / raw)
To: Luc Michel
Cc: Damien Hedde, Peter Maydell, richard.fuhler,
"Daniel P. Berrangé",
sam.grove, QEMU Developers, Mark Burton, Markus Armbruster,
edgar.iglesias, f4bug
> On 14 Jan 2021, at 12:56, Luc Michel <luc.michel@greensocs.com> wrote:
>
> ... But yes the goal here is to rely on QEMU's already existing device models to dynamically create a machine.
I did not check the current QEMU device models, so I might be wrong, but I think that the scope might be slightly different.
In my fork I read the JSON with the detailed peripherals definitions, which include not only memory address, but registers, sizes, alignment, bit-fields, access right, reset values, etc, and from these definitions I automatically create the QEMU objects.
The result is a device with all registers available, you can already read/write all of them, just that there are no actions associated yet, and these actions need to be added manually.
There is also a tool that generates the C code for this, so basically I can turn a CMSIS SVD definition into a complete QEMU device, partly functional (without actions).
I plan to further improve this mechanism, but it'll probably take some time.
Regards,
Liviu
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-11 14:50 About creating machines on the command line Luc Michel
2021-01-11 20:04 ` BALATON Zoltan
@ 2021-01-14 11:37 ` Daniel P. Berrangé
2021-01-14 17:11 ` Kashyap Chamarthy
2021-02-03 16:55 ` Philippe Mathieu-Daudé
2 siblings, 1 reply; 16+ messages in thread
From: Daniel P. Berrangé @ 2021-01-14 11:37 UTC (permalink / raw)
To: Luc Michel
Cc: Damien Hedde, peter.maydell, sam.grove, armbru, Mark Burton,
qemu-devel, richard.fuhler, edgar.iglesias, f4bug
On Mon, Jan 11, 2021 at 03:50:58PM +0100, Luc Michel wrote:
> Hi,
>
> We would like to work on improving QEMU to be able to create custom machines
> from the command line. The goal here is to get feedback from the community
> and shape the future developments.
>
> The use case mainly comes from people working with tools to customize their
> designs, such as SiFive Core Designer
> (https://scs.sifive.com/core-designer). This kind of tools may allow
> creation or customization of a whole SoC, from the number of cores, to the
> memory and IRQ mapping of peripherals etc.
>
> The ultimate goal would be to be able to create any kind of machine on the
> command line. However we are aware that this is a substantial amount of
> changes in QEMU.
>
> In its current state, QEMU allows for very limited customization of existing
> machines on the command line. We identified the following limitations (feel
> free to add to the list):
>
> - Most devices are not user creatable. Moreover, sysbus devices must be
> explicitly allowed by a machine to be creatable through `-device`,
>
> - Memory regions cannot be created on the command line,
>
> - Device MMIO regions cannot be mapped on a bus from the command line,
>
> - GPIOs and clocks cannot be wired from the command line,
>
> - CPUs are not sysbus devices (and not user-creatable). They need special
> care when creating them regarding system reset. Not being on a bus means
> that they must be reset manually on system reset. This is done in machines
> by registering a QEMU reset handler.
>
> - Machine specific boot code is usually hard-coded into the machine
> itself. Some architectures (e.g. ARM) do factorize bootloader related code,
> but there is no standard way of doing that in QEMU.
>
> We don't want to address all those limitations at once. We plan to start
> with the following scenario:
>
> - Start with a base machine that would handle CPU creation and bootloader
> stuff. Note that the "none" machine is probably not sufficient in its
> current shape. It does allow only one CPU and obviously does not handle the
> boot process.
>
> - Allow for this machine every sysbus devices we want to be user
> command-line creatable (and mark them user_creatable if needed)
>
> - Add command line options to create memory regions (probably ram ones at
> first)
>
> - Add command line options to map a memory region (including sysbus device
> MMIO regions) onto another (memory_region_add_subregion)
>
> - Add command line options to connect GPIOs and clocks.
>
> This would hopefully allow for simple machines creation. We would then be
> able to use either the command line or the `-readconfig` option to create
> the machine.
>
> Note that we are not planning to use QMP/HMP for now. From our
> understanding, a `device_add` request is always considered as hot-plug,
> which is not what we want here.
>
> Please tell us what do you think about this plan. Any feedback is
> appreciated. Then we can discuss the details of how to do this properly.
There's a general desire amongst QEMU maintainers to move to a world
where QAPI is used for describing everything. In this vision, eventually
all current command line options would be replaced with QMP commands
and QAPI objects specs.
In this world -readconfig is likely to be deleted. Also this means we
have a bias against adding new command line options to current QEMU.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-14 11:37 ` Daniel P. Berrangé
@ 2021-01-14 17:11 ` Kashyap Chamarthy
2021-01-14 17:26 ` Daniel P. Berrangé
0 siblings, 1 reply; 16+ messages in thread
From: Kashyap Chamarthy @ 2021-01-14 17:11 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Damien Hedde, peter.maydell, richard.fuhler, sam.grove,
Mark Burton, armbru, qemu-devel, edgar.iglesias, Luc Michel,
f4bug
On Thu, Jan 14, 2021 at 11:37:06AM +0000, Daniel P. Berrangé wrote:
> On Mon, Jan 11, 2021 at 03:50:58PM +0100, Luc Michel wrote:
[...]
> > This would hopefully allow for simple machines creation. We would then be
> > able to use either the command line or the `-readconfig` option to create
> > the machine.
> >
> > Note that we are not planning to use QMP/HMP for now. From our
> > understanding, a `device_add` request is always considered as hot-plug,
> > which is not what we want here.
> >
> > Please tell us what do you think about this plan. Any feedback is
> > appreciated. Then we can discuss the details of how to do this properly.
>
> There's a general desire amongst QEMU maintainers to move to a world
> where QAPI is used for describing everything. In this vision, eventually
> all current command line options would be replaced with QMP commands
> and QAPI objects specs.
>
> In this world -readconfig is likely to be deleted.
In that case, I hope the above intention / direction will be documented
somewhere more clearly. In the past I ran into at least a couple of
companies that use QEMU in production and heavily rely on '-readconfig',
despite knowing some of its shortcomings. There might be others out
there.
> Also this means we have a bias against adding new command line options
> to current QEMU.
[...]
--
/kashyap
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-14 17:11 ` Kashyap Chamarthy
@ 2021-01-14 17:26 ` Daniel P. Berrangé
0 siblings, 0 replies; 16+ messages in thread
From: Daniel P. Berrangé @ 2021-01-14 17:26 UTC (permalink / raw)
To: Kashyap Chamarthy
Cc: Damien Hedde, peter.maydell, richard.fuhler, sam.grove,
Mark Burton, armbru, qemu-devel, edgar.iglesias, Luc Michel,
f4bug
On Thu, Jan 14, 2021 at 06:11:50PM +0100, Kashyap Chamarthy wrote:
> On Thu, Jan 14, 2021 at 11:37:06AM +0000, Daniel P. Berrangé wrote:
> > On Mon, Jan 11, 2021 at 03:50:58PM +0100, Luc Michel wrote:
>
> [...]
>
> > > This would hopefully allow for simple machines creation. We would then be
> > > able to use either the command line or the `-readconfig` option to create
> > > the machine.
> > >
> > > Note that we are not planning to use QMP/HMP for now. From our
> > > understanding, a `device_add` request is always considered as hot-plug,
> > > which is not what we want here.
> > >
> > > Please tell us what do you think about this plan. Any feedback is
> > > appreciated. Then we can discuss the details of how to do this properly.
> >
> > There's a general desire amongst QEMU maintainers to move to a world
> > where QAPI is used for describing everything. In this vision, eventually
> > all current command line options would be replaced with QMP commands
> > and QAPI objects specs.
> >
> > In this world -readconfig is likely to be deleted.
>
> In that case, I hope the above intention / direction will be documented
> somewhere more clearly. In the past I ran into at least a couple of
> companies that use QEMU in production and heavily rely on '-readconfig',
> despite knowing some of its shortcomings. There might be others out
> there.
Such a change would go through the deprecation process as normal so
they would have 2 releases prior warning.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-01-11 14:50 About creating machines on the command line Luc Michel
2021-01-11 20:04 ` BALATON Zoltan
2021-01-14 11:37 ` Daniel P. Berrangé
@ 2021-02-03 16:55 ` Philippe Mathieu-Daudé
2021-02-03 17:09 ` graf--- via
2 siblings, 1 reply; 16+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-02-03 16:55 UTC (permalink / raw)
To: Luc Michel, qemu-devel, Nikos Dragazis, Maxime Coquelin,
Thanos Makatos, Andra-Irina Paraschiv, John G. Johnson,
Jan Kiszka, Alexander Graf
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, Mark Burton,
richard.fuhler, armbru, edgar.iglesias
On 1/11/21 3:50 PM, Luc Michel wrote:
> Hi,
>
> We would like to work on improving QEMU to be able to create custom
> machines from the command line. The goal here is to get feedback from
> the community and shape the future developments.
Cc'ing John who is working on command line, and some developers from
the "inter-VM device emulation interface" call.
https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>
> The use case mainly comes from people working with tools to customize
> their designs, such as SiFive Core Designer
> (https://scs.sifive.com/core-designer). This kind of tools may allow
> creation or customization of a whole SoC, from the number of cores, to
> the memory and IRQ mapping of peripherals etc.
>
> The ultimate goal would be to be able to create any kind of machine on
> the command line. However we are aware that this is a substantial amount
> of changes in QEMU.
>
> In its current state, QEMU allows for very limited customization of
> existing machines on the command line. We identified the following
> limitations (feel free to add to the list):
>
> - Most devices are not user creatable. Moreover, sysbus devices must
> be explicitly allowed by a machine to be creatable through `-device`,
>
> - Memory regions cannot be created on the command line,
>
> - Device MMIO regions cannot be mapped on a bus from the command line,
>
> - GPIOs and clocks cannot be wired from the command line,
>
> - CPUs are not sysbus devices (and not user-creatable). They need
> special care when creating them regarding system reset. Not being on a
> bus means that they must be reset manually on system reset. This is done
> in machines by registering a QEMU reset handler.
>
> - Machine specific boot code is usually hard-coded into the machine
> itself. Some architectures (e.g. ARM) do factorize bootloader related
> code, but there is no standard way of doing that in QEMU.
>
> We don't want to address all those limitations at once. We plan to start
> with the following scenario:
>
> - Start with a base machine that would handle CPU creation and
> bootloader stuff. Note that the "none" machine is probably not
> sufficient in its current shape. It does allow only one CPU and
> obviously does not handle the boot process.
>
> - Allow for this machine every sysbus devices we want to be user
> command-line creatable (and mark them user_creatable if needed)
>
> - Add command line options to create memory regions (probably ram ones
> at first)
>
> - Add command line options to map a memory region (including sysbus
> device MMIO regions) onto another (memory_region_add_subregion)
>
> - Add command line options to connect GPIOs and clocks.
>
> This would hopefully allow for simple machines creation. We would then
> be able to use either the command line or the `-readconfig` option to
> create the machine.
>
> Note that we are not planning to use QMP/HMP for now. From our
> understanding, a `device_add` request is always considered as hot-plug,
> which is not what we want here.
>
> Please tell us what do you think about this plan. Any feedback is
> appreciated. Then we can discuss the details of how to do this properly.
> Thanks!
>
> --
> Luc
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-02-03 16:55 ` Philippe Mathieu-Daudé
@ 2021-02-03 17:09 ` graf--- via
2021-02-04 20:29 ` John Snow
2021-02-05 10:43 ` Luc Michel
0 siblings, 2 replies; 16+ messages in thread
From: graf--- via @ 2021-02-03 17:09 UTC (permalink / raw)
To: Philippe Mathieu-Daudé,
Luc Michel, qemu-devel, Nikos Dragazis, Maxime Coquelin,
Thanos Makatos, Andra-Irina Paraschiv, John G. Johnson,
Jan Kiszka
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, Mark Burton,
richard.fuhler, armbru, edgar.iglesias
On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
>
> On 1/11/21 3:50 PM, Luc Michel wrote:
>> Hi,
>>
>> We would like to work on improving QEMU to be able to create custom
>> machines from the command line. The goal here is to get feedback from
>> the community and shape the future developments.
>
> Cc'ing John who is working on command line, and some developers from
> the "inter-VM device emulation interface" call.
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>
>>
>> The use case mainly comes from people working with tools to customize
>> their designs, such as SiFive Core Designer
>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>> creation or customization of a whole SoC, from the number of cores, to
>> the memory and IRQ mapping of peripherals etc.
>>
>> The ultimate goal would be to be able to create any kind of machine on
>> the command line. However we are aware that this is a substantial amount
>> of changes in QEMU.
Is the command line really the right abstraction level here? Wouldn't it
make more sense to have a QOM / <scripting language> bridge that allows
you to create and connect QOM objects using for example Python?
You could then have machine descriptions in a script, which could be
generated by the SoC customization tools.
In combination with plugin support for platform devices, that should
allow pretty much any customization you would need to happen, no?
Alex
>>
>> In its current state, QEMU allows for very limited customization of
>> existing machines on the command line. We identified the following
>> limitations (feel free to add to the list):
>>
>> - Most devices are not user creatable. Moreover, sysbus devices must
>> be explicitly allowed by a machine to be creatable through `-device`,
>>
>> - Memory regions cannot be created on the command line,
>>
>> - Device MMIO regions cannot be mapped on a bus from the command line,
>>
>> - GPIOs and clocks cannot be wired from the command line,
>>
>> - CPUs are not sysbus devices (and not user-creatable). They need
>> special care when creating them regarding system reset. Not being on a
>> bus means that they must be reset manually on system reset. This is done
>> in machines by registering a QEMU reset handler.
>>
>> - Machine specific boot code is usually hard-coded into the machine
>> itself. Some architectures (e.g. ARM) do factorize bootloader related
>> code, but there is no standard way of doing that in QEMU.
>>
>> We don't want to address all those limitations at once. We plan to start
>> with the following scenario:
>>
>> - Start with a base machine that would handle CPU creation and
>> bootloader stuff. Note that the "none" machine is probably not
>> sufficient in its current shape. It does allow only one CPU and
>> obviously does not handle the boot process.
>>
>> - Allow for this machine every sysbus devices we want to be user
>> command-line creatable (and mark them user_creatable if needed)
>>
>> - Add command line options to create memory regions (probably ram ones
>> at first)
>>
>> - Add command line options to map a memory region (including sysbus
>> device MMIO regions) onto another (memory_region_add_subregion)
>>
>> - Add command line options to connect GPIOs and clocks.
>>
>> This would hopefully allow for simple machines creation. We would then
>> be able to use either the command line or the `-readconfig` option to
>> create the machine.
>>
>> Note that we are not planning to use QMP/HMP for now. From our
>> understanding, a `device_add` request is always considered as hot-plug,
>> which is not what we want here.
>>
>> Please tell us what do you think about this plan. Any feedback is
>> appreciated. Then we can discuss the details of how to do this properly.
>> Thanks!
>>
>> --
>> Luc
>>
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-02-03 17:09 ` graf--- via
@ 2021-02-04 20:29 ` John Snow
2021-02-05 16:04 ` Luc Michel
2021-02-05 10:43 ` Luc Michel
1 sibling, 1 reply; 16+ messages in thread
From: John Snow @ 2021-02-04 20:29 UTC (permalink / raw)
To: Alexander Graf, Philippe Mathieu-Daudé,
Luc Michel, qemu-devel, Nikos Dragazis, Maxime Coquelin,
Thanos Makatos, Andra-Irina Paraschiv, John G. Johnson,
Jan Kiszka
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, Mark Burton,
richard.fuhler, armbru, edgar.iglesias
On 2/3/21 12:09 PM, graf--- via wrote:
>
>
> On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
>>
>> On 1/11/21 3:50 PM, Luc Michel wrote:
>>> Hi,
>>>
>>> We would like to work on improving QEMU to be able to create custom
>>> machines from the command line. The goal here is to get feedback from
>>> the community and shape the future developments.
>>
>> Cc'ing John who is working on command line, and some developers from
>> the "inter-VM device emulation interface" call.
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>>
I have a bunch of extremely naive questions about what it would take to
generalize this kind of interface; my familiarity is largely with x86
and so I actually just have a series of very basic questions about the
scope of this work.
>>>
>>> The use case mainly comes from people working with tools to customize
>>> their designs, such as SiFive Core Designer
>>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>>> creation or customization of a whole SoC, from the number of cores, to
>>> the memory and IRQ mapping of peripherals etc.
>>>
>>> The ultimate goal would be to be able to create any kind of machine on
>>> the command line. However we are aware that this is a substantial amount
>>> of changes in QEMU.
>
> Is the command line really the right abstraction level here? Wouldn't it
> make more sense to have a QOM / <scripting language> bridge that allows
> you to create and connect QOM objects using for example Python?
>
> You could then have machine descriptions in a script, which could be
> generated by the SoC customization tools.
>
> In combination with plugin support for platform devices, that should
> allow pretty much any customization you would need to happen, no?
>
>
> Alex
>
Right, I am just reading this proposal as "We want an interface language
that allows us to construct more arbitrary configurations."
In this case, something QMP/QOM-based would be the right place to do it
and not the CLI as it exists today. Small details in the grand scheme of
things the kind of internal modeling cleanup this idea seems to imply.
>>>
>>> In its current state, QEMU allows for very limited customization of
>>> existing machines on the command line. We identified the following
>>> limitations (feel free to add to the list):
>>>
>>> - Most devices are not user creatable. Moreover, sysbus devices must
>>> be explicitly allowed by a machine to be creatable through `-device`,
>>>
>>> - Memory regions cannot be created on the command line,
>>>
>>> - Device MMIO regions cannot be mapped on a bus from the command
>>> line,
>>>
>>> - GPIOs and clocks cannot be wired from the command line,
>>>
>>> - CPUs are not sysbus devices (and not user-creatable). They need
>>> special care when creating them regarding system reset. Not being on a
>>> bus means that they must be reset manually on system reset. This is done
>>> in machines by registering a QEMU reset handler.
>>>
>>> - Machine specific boot code is usually hard-coded into the machine
>>> itself. Some architectures (e.g. ARM) do factorize bootloader related
>>> code, but there is no standard way of doing that in QEMU.
>>>
>>> We don't want to address all those limitations at once. We plan to start
>>> with the following scenario:
>>>
It does sound like a staggering level of work.
>>> - Start with a base machine that would handle CPU creation and
>>> bootloader stuff. Note that the "none" machine is probably not
>>> sufficient in its current shape. It does allow only one CPU and
>>> obviously does not handle the boot process.
>>>
I suppose this implies a type of machine that doesn't have migration
compatibility. e.g. your "blank" might be different from someone else's
"blank".
Or at least, not migration compatibility in its current form. Some
thought might need to be put into how to negotiate migration
compatibility between two instances of QEMU; it sounds like a hard problem.
(Jokingly, I'd want to call a machine like this "Apple Pie", but that
would probably be too confusing for both Apple and Raspberry Pi
enthusiasts.)
>>> - Allow for this machine every sysbus devices we want to be user
>>> command-line creatable (and mark them user_creatable if needed)
>>>
Is it even possible to enumerate, for every sysbus device in QEMU, what
the interface requirements are?
e.g. I assume most need a memory region, or maybe an IO region, or maybe
an IRQ resource of some kind, or ...
doing this kind of dynamic bringup requires a lot of configuration data
that is at-present just hardcoded absolutely everywhere.
>>> - Add command line options to create memory regions (probably ram
>>> ones
>>> at first)
>>>
Naive question: what does adding a memory region mean to you in this
context?
For whom (which device, spec, standard, instruction set, etc?) are these
regions tangible abstractions?
>>> - Add command line options to map a memory region (including sysbus
>>> device MMIO regions) onto another (memory_region_add_subregion)
>>>
More naive questions:
The assumption here being that sysbus devices can be modeled using their
own private memory addressing namespace; and we configure them by
mapping that space onto the machine's physical address space.
Instead of, say, making the memory address of that device a property of
the device.
Yes?
>>> - Add command line options to connect GPIOs and clocks.
>>>
Similar to elsewhere, I wonder how much work it is to generalize this
kind of pluggability so that all existing clocks and GPIO devices can be
modeled in the same way.
(I say this meaning: Wow, I have no idea. I know a lot of the ones we
have are hard-coded. What does a configuration interface for this look
like?)
>>> This would hopefully allow for simple machines creation. We would then
>>> be able to use either the command line or the `-readconfig` option to
>>> create the machine.
>>>
>>> Note that we are not planning to use QMP/HMP for now. From our
>>> understanding, a `device_add` request is always considered as hot-plug,
>>> which is not what we want here.
>>>
We're working our way towards being able to use QMP to define machines
prior to execution start in order to have a QMP-only interaction with
QEMU possible.
You likely want to invest in that kind of a strategy where you define
structures that describe the machines you want to build.
>>> Please tell us what do you think about this plan. Any feedback is
>>> appreciated. Then we can discuss the details of how to do this
>>> properly.
>>> Thanks!
>>>
>>> --
>>> Luc
>>>
>
>
>
> Amazon Development Center Germany GmbH
> Krausenstr. 38
> 10117 Berlin
> Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
> Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
> Sitz: Berlin
> Ust-ID: DE 289 237 879
>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-02-03 17:09 ` graf--- via
2021-02-04 20:29 ` John Snow
@ 2021-02-05 10:43 ` Luc Michel
2021-02-10 12:13 ` Alexander Graf
1 sibling, 1 reply; 16+ messages in thread
From: Luc Michel @ 2021-02-05 10:43 UTC (permalink / raw)
To: Alexander Graf, Philippe Mathieu-Daudé,
qemu-devel, Nikos Dragazis, Maxime Coquelin, Thanos Makatos,
Andra-Irina Paraschiv, John G. Johnson, Jan Kiszka
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, Mark Burton,
richard.fuhler, armbru, edgar.iglesias
On 2/3/21 6:09 PM, Alexander Graf wrote:
>
>
> On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
>>
>> On 1/11/21 3:50 PM, Luc Michel wrote:
>>> Hi,
>>>
>>> We would like to work on improving QEMU to be able to create custom
>>> machines from the command line. The goal here is to get feedback from
>>> the community and shape the future developments.
>>
>> Cc'ing John who is working on command line, and some developers from
>> the "inter-VM device emulation interface" call.
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>>
>>>
>>> The use case mainly comes from people working with tools to customize
>>> their designs, such as SiFive Core Designer
>>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>>> creation or customization of a whole SoC, from the number of cores, to
>>> the memory and IRQ mapping of peripherals etc.
>>>
>>> The ultimate goal would be to be able to create any kind of machine on
>>> the command line. However we are aware that this is a substantial amount
>>> of changes in QEMU.
>
> Is the command line really the right abstraction level here? Wouldn't it
> make more sense to have a QOM / <scripting language> bridge that allows
> you to create and connect QOM objects using for example Python?
Yes, after some discussions with the community, we are now working on
improving QMP to achieve this. We first started with the idea of the
command line because it seems to be the place where we had "almost"
everything we needed already. In either cases we are planning to use a
front-end script to go from e.g. a DTB to whatever QEMU interface we
will end up using.
>
> You could then have machine descriptions in a script, which could be
> generated by the SoC customization tools.
Yes, most likely a DTB in our case.
>
> In combination with plugin support for platform devices, that should
> allow pretty much any customization you would need to happen, no?
I'm not sure what you mean by "plugin support for platform devices". If
you refer to the current QEMU plugin API, it's pretty much limited to
TCG. If you suggest this API could be enhanced to allow for custom
devices to be added by plugins, for now we are planning to use only
existing device models. It's more a matter of having flexibility on
connections between the devices than on the devices themselves (but yes
that would be a nice next step).
Thanks,
--
Luc
>
>
> Alex
>
>>>
>>> In its current state, QEMU allows for very limited customization of
>>> existing machines on the command line. We identified the following
>>> limitations (feel free to add to the list):
>>>
>>> - Most devices are not user creatable. Moreover, sysbus devices must
>>> be explicitly allowed by a machine to be creatable through `-device`,
>>>
>>> - Memory regions cannot be created on the command line,
>>>
>>> - Device MMIO regions cannot be mapped on a bus from the command
>>> line,
>>>
>>> - GPIOs and clocks cannot be wired from the command line,
>>>
>>> - CPUs are not sysbus devices (and not user-creatable). They need
>>> special care when creating them regarding system reset. Not being on a
>>> bus means that they must be reset manually on system reset. This is done
>>> in machines by registering a QEMU reset handler.
>>>
>>> - Machine specific boot code is usually hard-coded into the machine
>>> itself. Some architectures (e.g. ARM) do factorize bootloader related
>>> code, but there is no standard way of doing that in QEMU.
>>>
>>> We don't want to address all those limitations at once. We plan to start
>>> with the following scenario:
>>>
>>> - Start with a base machine that would handle CPU creation and
>>> bootloader stuff. Note that the "none" machine is probably not
>>> sufficient in its current shape. It does allow only one CPU and
>>> obviously does not handle the boot process.
>>>
>>> - Allow for this machine every sysbus devices we want to be user
>>> command-line creatable (and mark them user_creatable if needed)
>>>
>>> - Add command line options to create memory regions (probably ram
>>> ones
>>> at first)
>>>
>>> - Add command line options to map a memory region (including sysbus
>>> device MMIO regions) onto another (memory_region_add_subregion)
>>>
>>> - Add command line options to connect GPIOs and clocks.
>>>
>>> This would hopefully allow for simple machines creation. We would then
>>> be able to use either the command line or the `-readconfig` option to
>>> create the machine.
>>>
>>> Note that we are not planning to use QMP/HMP for now. From our
>>> understanding, a `device_add` request is always considered as hot-plug,
>>> which is not what we want here.
>>>
>>> Please tell us what do you think about this plan. Any feedback is
>>> appreciated. Then we can discuss the details of how to do this
>>> properly.
>>> Thanks!
>>>
>>> --
>>> Luc
>>>
>
>
>
> Amazon Development Center Germany GmbH
> Krausenstr. 38
> 10117 Berlin
> Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
> Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
> Sitz: Berlin
> Ust-ID: DE 289 237 879
>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-02-04 20:29 ` John Snow
@ 2021-02-05 16:04 ` Luc Michel
0 siblings, 0 replies; 16+ messages in thread
From: Luc Michel @ 2021-02-05 16:04 UTC (permalink / raw)
To: John Snow, Alexander Graf, Philippe Mathieu-Daudé,
qemu-devel, Nikos Dragazis, Maxime Coquelin, Thanos Makatos,
Andra-Irina Paraschiv, John G. Johnson, Jan Kiszka
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, Mark Burton,
richard.fuhler, armbru, edgar.iglesias
On 2/4/21 9:29 PM, John Snow wrote:
> On 2/3/21 12:09 PM, graf--- via wrote:
>>
>>
>> On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
>>>
>>> On 1/11/21 3:50 PM, Luc Michel wrote:
>>>> Hi,
>>>>
>>>> We would like to work on improving QEMU to be able to create custom
>>>> machines from the command line. The goal here is to get feedback from
>>>> the community and shape the future developments.
>>>
>>> Cc'ing John who is working on command line, and some developers from
>>> the "inter-VM device emulation interface" call.
>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>>>
>
> I have a bunch of extremely naive questions about what it would take to
> generalize this kind of interface; my familiarity is largely with x86
> and so I actually just have a series of very basic questions about the
> scope of this work.
>
>>>>
>>>> The use case mainly comes from people working with tools to customize
>>>> their designs, such as SiFive Core Designer
>>>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>>>> creation or customization of a whole SoC, from the number of cores, to
>>>> the memory and IRQ mapping of peripherals etc.
>>>>
>>>> The ultimate goal would be to be able to create any kind of machine on
>>>> the command line. However we are aware that this is a substantial
>>>> amount
>>>> of changes in QEMU.
>>
>> Is the command line really the right abstraction level here? Wouldn't
>> it make more sense to have a QOM / <scripting language> bridge that
>> allows you to create and connect QOM objects using for example Python?
>>
>> You could then have machine descriptions in a script, which could be
>> generated by the SoC customization tools.
>>
>> In combination with plugin support for platform devices, that should
>> allow pretty much any customization you would need to happen, no?
>>
>>
>> Alex
>>
>
> Right, I am just reading this proposal as "We want an interface language
> that allows us to construct more arbitrary configurations."
Yes exactly.
>
> In this case, something QMP/QOM-based would be the right place to do it
> and not the CLI as it exists today.
Yes, we realized this through the discussions. We are now changing our
plan to focus on QMP.
> Small details in the grand scheme of
> things the kind of internal modeling cleanup this idea seems to imply.
>
>>>>
>>>> In its current state, QEMU allows for very limited customization of
>>>> existing machines on the command line. We identified the following
>>>> limitations (feel free to add to the list):
>>>>
>>>> - Most devices are not user creatable. Moreover, sysbus devices must
>>>> be explicitly allowed by a machine to be creatable through `-device`,
>>>>
>>>> - Memory regions cannot be created on the command line,
>>>>
>>>> - Device MMIO regions cannot be mapped on a bus from the command
>>>> line,
>>>>
>>>> - GPIOs and clocks cannot be wired from the command line,
>>>>
>>>> - CPUs are not sysbus devices (and not user-creatable). They need
>>>> special care when creating them regarding system reset. Not being on a
>>>> bus means that they must be reset manually on system reset. This is
>>>> done
>>>> in machines by registering a QEMU reset handler.
>>>>
>>>> - Machine specific boot code is usually hard-coded into the machine
>>>> itself. Some architectures (e.g. ARM) do factorize bootloader related
>>>> code, but there is no standard way of doing that in QEMU.
>>>>
>>>> We don't want to address all those limitations at once. We plan to
>>>> start
>>>> with the following scenario:
>>>>
>
> It does sound like a staggering level of work.
>
>>>> - Start with a base machine that would handle CPU creation and
>>>> bootloader stuff. Note that the "none" machine is probably not
>>>> sufficient in its current shape. It does allow only one CPU and
>>>> obviously does not handle the boot process.
>>>>
>
> I suppose this implies a type of machine that doesn't have migration
> compatibility. e.g. your "blank" might be different from someone else's
> "blank".
>
> Or at least, not migration compatibility in its current form. Some
> thought might need to be put into how to negotiate migration
> compatibility between two instances of QEMU; it sounds like a hard problem.
>
> (Jokingly, I'd want to call a machine like this "Apple Pie", but that
> would probably be too confusing for both Apple and Raspberry Pi
> enthusiasts.)
Migration is definitely an issue. We didn't think too much about it yet
because the use-case is more firmware/OS development oriented (i.e. I
fix, compile and re-run my code in QEMU to test/debug). Migration is not
a must-have for this use-case.
>
>>>> - Allow for this machine every sysbus devices we want to be user
>>>> command-line creatable (and mark them user_creatable if needed)
>>>>
>
> Is it even possible to enumerate, for every sysbus device in QEMU, what
> the interface requirements are?
>
> e.g. I assume most need a memory region, or maybe an IO region, or maybe
> an IRQ resource of some kind, or ...
>
> doing this kind of dynamic bringup requires a lot of configuration data
> that is at-present just hardcoded absolutely everywhere.
As of today, as I understand it, what a sysbus device exposes is part of
its "specification" (i.e. in best cases, it is documented in a comment
somewhere in its header file). Apart from that there is no way of
knowing that the memory region number 0 is e.g for the ARM GIC, the
distributor interface, while the number 1 is the CPU interface.
An introspection mechanism would be very helpful here. A simple addition
would be to be able to name those regions at sysbus device init time,
the same way we can name GPIOs and clocks. Having a proper name is
already a good piece of "self-documentation" for the device, and it
could be made observable by the user (as properties are today in QMP).
>
>>>> - Add command line options to create memory regions (probably ram
>>>> ones
>>>> at first)
>>>>
>
> Naive question: what does adding a memory region mean to you in this
> context?
>
> For whom (which device, spec, standard, instruction set, etc?) are these
> regions tangible abstractions?
Good question. What we want here is basically being able to create the
memories of our platform (rom, ram). Those can be viewed as devices
really, and this is how in our current prototype we made them (a simple
sysbus device that embeds the memory region and exposes it through SBD
API). I think this is the good level of abstraction to choose because it
aligns nicely with the rest. Such a simple device would make sense in
vanilla IMO. Creating the memories of the platform would then resort to
creating a device (with parameters to set the size, set if it is
read-only, ...) and connecting it to a bus.
>
>>>> - Add command line options to map a memory region (including sysbus
>>>> device MMIO regions) onto another (memory_region_add_subregion)
>>>>
>
> More naive questions:
>
> The assumption here being that sysbus devices can be modeled using their
> own private memory addressing namespace; and we configure them by
> mapping that space onto the machine's physical address space.
>
> Instead of, say, making the memory address of that device a property of
> the device.
>
> Yes?
Yes memory regions can be created by sysbus devices and exposed at the
sysbus device API level. Then the parent device instantiating this
device (or the machine itself) can use the SBD API to fetch and connect
them to other memory regions. This is then up to the parent to choose
where and how to map the child region(s).
>
>>>> - Add command line options to connect GPIOs and clocks.
>>>>
>
> Similar to elsewhere, I wonder how much work it is to generalize this
> kind of pluggability so that all existing clocks and GPIO devices can be
> modeled in the same way.
>
> (I say this meaning: Wow, I have no idea. I know a lot of the ones we
> have are hard-coded. What does a configuration interface for this look
> like?)
By hard-coded, you mean the connections? Because otherwise the GPIO and
Clock APIs are well defined already. Exposing this to QMP does not seem
to be an issue. I could be wrong though.
>
>>>> This would hopefully allow for simple machines creation. We would then
>>>> be able to use either the command line or the `-readconfig` option to
>>>> create the machine.
>>>>
>>>> Note that we are not planning to use QMP/HMP for now. From our
>>>> understanding, a `device_add` request is always considered as hot-plug,
>>>> which is not what we want here.
>>>>
>
> We're working our way towards being able to use QMP to define machines
> prior to execution start in order to have a QMP-only interaction with
> QEMU possible.
>
> You likely want to invest in that kind of a strategy where you define
> structures that describe the machines you want to build.
OK. It is clear to us that QMP is the way to go now. The plan is for us
to work and come up with a first RFC series.
Thanks a lot!
--
Luc
>
>>>> Please tell us what do you think about this plan. Any feedback is
>>>> appreciated. Then we can discuss the details of how to do this
>>>> properly.
>>>> Thanks!
>>>>
>>>> --
>>>> Luc
>>>>
>>
>>
>>
>> Amazon Development Center Germany GmbH
>> Krausenstr. 38
>> 10117 Berlin
>> Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
>> Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
>> Sitz: Berlin
>> Ust-ID: DE 289 237 879
>>
>>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-02-05 10:43 ` Luc Michel
@ 2021-02-10 12:13 ` Alexander Graf
2021-02-13 13:58 ` Luc Michel
0 siblings, 1 reply; 16+ messages in thread
From: Alexander Graf @ 2021-02-10 12:13 UTC (permalink / raw)
To: Luc Michel, Philippe Mathieu-Daudé,
qemu-devel, Nikos Dragazis, Maxime Coquelin, Thanos Makatos,
Andra-Irina Paraschiv, John G. Johnson, Jan Kiszka
Cc: Damien Hedde, peter.maydell, richard.fuhler, berrange, sam.grove,
Mark Burton, armbru, edgar.iglesias
On 05.02.21 11:43, Luc Michel wrote:
>
> On 2/3/21 6:09 PM, Alexander Graf wrote:
>>
>>
>> On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
>>>
>>> On 1/11/21 3:50 PM, Luc Michel wrote:
>>>> Hi,
>>>>
>>>> We would like to work on improving QEMU to be able to create custom
>>>> machines from the command line. The goal here is to get feedback from
>>>> the community and shape the future developments.
>>>
>>> Cc'ing John who is working on command line, and some developers from
>>> the "inter-VM device emulation interface" call.
>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>>>
>>>>
>>>> The use case mainly comes from people working with tools to customize
>>>> their designs, such as SiFive Core Designer
>>>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>>>> creation or customization of a whole SoC, from the number of cores, to
>>>> the memory and IRQ mapping of peripherals etc.
>>>>
>>>> The ultimate goal would be to be able to create any kind of machine on
>>>> the command line. However we are aware that this is a substantial
>>>> amount
>>>> of changes in QEMU.
>>
>> Is the command line really the right abstraction level here? Wouldn't it
>> make more sense to have a QOM / <scripting language> bridge that allows
>> you to create and connect QOM objects using for example Python?
> Yes, after some discussions with the community, we are now working on
> improving QMP to achieve this. We first started with the idea of the
> command line because it seems to be the place where we had "almost"
> everything we needed already. In either cases we are planning to use a
> front-end script to go from e.g. a DTB to whatever QEMU interface we
> will end up using.
>
>>
>> You could then have machine descriptions in a script, which could be
>> generated by the SoC customization tools.
> Yes, most likely a DTB in our case.
So, something like this?
https://github.com/Xilinx/qemu/blob/master/hw/arm/arm_generic_fdt.c
I'm not sure how DTB would translate 1:1 into qom properties for a reeal
generic mechanism? And how easy it would be to extract target device
specifics, such as interrupt descriptors or PCI ranges properties.
If you want to make this fully generic, try to ensure that you can model
weird beasts such as the q35 machine model.
Either way, the idea has come up multiple times before. I don't know of
anyone who successfully managed to push it upstream yet, but I
definitely hope we will be at a point where you can describe machine
models in something different than C.
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: About creating machines on the command line
2021-02-10 12:13 ` Alexander Graf
@ 2021-02-13 13:58 ` Luc Michel
0 siblings, 0 replies; 16+ messages in thread
From: Luc Michel @ 2021-02-13 13:58 UTC (permalink / raw)
To: Alexander Graf, Philippe Mathieu-Daudé,
qemu-devel, Nikos Dragazis, Maxime Coquelin, Thanos Makatos,
Andra-Irina Paraschiv, John G. Johnson, Jan Kiszka
Cc: Damien Hedde, peter.maydell, berrange, sam.grove, Mark Burton,
richard.fuhler, armbru, edgar.iglesias
On 2/10/21 1:13 PM, Alexander Graf wrote:
>
>
> On 05.02.21 11:43, Luc Michel wrote:
>>
>> On 2/3/21 6:09 PM, Alexander Graf wrote:
>>>
>>>
>>> On 03.02.21 17:55, Philippe Mathieu-Daudé wrote:
>>>>
>>>> On 1/11/21 3:50 PM, Luc Michel wrote:
>>>>> Hi,
>>>>>
>>>>> We would like to work on improving QEMU to be able to create custom
>>>>> machines from the command line. The goal here is to get feedback from
>>>>> the community and shape the future developments.
>>>>
>>>> Cc'ing John who is working on command line, and some developers from
>>>> the "inter-VM device emulation interface" call.
>>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg723252.html
>>>>
>>>>>
>>>>> The use case mainly comes from people working with tools to customize
>>>>> their designs, such as SiFive Core Designer
>>>>> (https://scs.sifive.com/core-designer). This kind of tools may allow
>>>>> creation or customization of a whole SoC, from the number of cores, to
>>>>> the memory and IRQ mapping of peripherals etc.
>>>>>
>>>>> The ultimate goal would be to be able to create any kind of machine on
>>>>> the command line. However we are aware that this is a substantial
>>>>> amount
>>>>> of changes in QEMU.
>>>
>>> Is the command line really the right abstraction level here? Wouldn't it
>>> make more sense to have a QOM / <scripting language> bridge that allows
>>> you to create and connect QOM objects using for example Python?
>> Yes, after some discussions with the community, we are now working on
>> improving QMP to achieve this. We first started with the idea of the
>> command line because it seems to be the place where we had "almost"
>> everything we needed already. In either cases we are planning to use a
>> front-end script to go from e.g. a DTB to whatever QEMU interface we
>> will end up using.
>>
>>>
>>> You could then have machine descriptions in a script, which could be
>>> generated by the SoC customization tools.
>> Yes, most likely a DTB in our case.
>
> So, something like this?
>
> https://github.com/Xilinx/qemu/blob/master/hw/arm/arm_generic_fdt.c
Yes, but we specifically want to avoid embedding the part that will
parse and translate the DTB into QEMU.
>
> I'm not sure how DTB would translate 1:1 into qom properties for a reeal
> generic mechanism? And how easy it would be to extract target device
> specifics, such as interrupt descriptors or PCI ranges properties.
I think there is no such 1:1 mapping. That's why we believe the tool(s)
that parses the DTB and translate it into QMP commands must stay out of
QEMU.
>
> If you want to make this fully generic, try to ensure that you can model
> weird beasts such as the q35 machine model.
>
> Either way, the idea has come up multiple times before. I don't know of
> anyone who successfully managed to push it upstream yet, but I
> definitely hope we will be at a point where you can describe machine
> models in something different than C.
We'll try an go step by step, to not raise all the issues at the same
time hopefully.
Thanks
--
Luc
>
>
> Alex
>
>
>
> Amazon Development Center Germany GmbH
> Krausenstr. 38
> 10117 Berlin
> Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
> Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
> Sitz: Berlin
> Ust-ID: DE 289 237 879
>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2021-02-13 14:02 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-11 14:50 About creating machines on the command line Luc Michel
2021-01-11 20:04 ` BALATON Zoltan
2021-01-11 20:28 ` Liviu Ionescu
2021-01-14 10:56 ` Luc Michel
2021-01-14 11:07 ` Liviu Ionescu
2021-01-14 9:30 ` Luc Michel
2021-01-14 11:37 ` Daniel P. Berrangé
2021-01-14 17:11 ` Kashyap Chamarthy
2021-01-14 17:26 ` Daniel P. Berrangé
2021-02-03 16:55 ` Philippe Mathieu-Daudé
2021-02-03 17:09 ` graf--- via
2021-02-04 20:29 ` John Snow
2021-02-05 16:04 ` Luc Michel
2021-02-05 10:43 ` Luc Michel
2021-02-10 12:13 ` Alexander Graf
2021-02-13 13:58 ` Luc Michel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.