[Qemu-devel] [RFC] Plan for moving forward with QOM

[Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

Hi,

I spent a couple hours today writing up some comparisons and an initial 
task list for converting qdev to QOM.  The main location of this is the 
wiki[1] but I thought I would inline it here for easier commenting.

I decided to do this because I wanted to avoid a massively long 00 patch 
explaining the rationale for the first batch of changes that I'm going 
to send out.

There is so much complexity in qdev and the device model in general that 
it's hard to come up with a concise document.  I'd really appreciate 
suggestions for topics to write up more rationale as that would help me 
avoid writing a book on the topic :-)

[1] http://wiki.qemu.org/Features/QOM

== Device Relationships ==

=== Device Relationships in QDev ===

The two main concepts in QDev are devices and busses.  A device is 
represented
by a DeviceState and a bus is represented by a BusState.  They do not 
share a
common base class.  Devices can have properties but busses cannot.  A device
has no direct relationship with other devices.  The only relationship is
indirect through busses.

A device may have zero or more busses associated with it via a has-a
relationship.  Each child bus may have multiple devices associated with 
it via
a reference.  All devices have a single parent bus and all busses have a 
single
parent device.  These relationships form a strict tree where every 
alternating
level is a bus level followed by a device level.  The root of the tree 
is the
main system bus often referred to as SysBus.

=== Device Relationships in QOM ===

Everything in QOM is a device.  The concept of busses are implemented as an
interface that a device implements.  Devices can have two types of 
relationships
with other devices: device composition or device backlinks.  Device 
composition
involves one device directly creating another device.  It will own life 
cycle
management for the created device and will generally propagate events to 
that
device (although there are exceptions).

Device backlinks allow one device to refer to another device in a looser
fashion.  While there can be only one device composition relationship that
exists between two specific devices, a device can participate in an 
arbitrary
number of backlinks.

Device composition and backlinks are both strongly typed and can be 
typed as a
specific device type or as an interface.  When typed as an interface, any
device that implements that interface can be used.

There is no explicit notion of parents in QOM.  A typical bus relationship
would the bus having a backlink to the child device, and the child 
device having
a backlink to the bus.

Without device backlinks, device composition forms a multi-rooted strict 
tree.
With backlinks, the result are multiple directed graphs.

== Naming ==

=== Naming in QDev ===

QDev has three namespaces: the device namespace, the bus namespace, and 
property
namespaces.

The device namespace contains the names of qdev devices.  qdev supports the
ability to have anonymous devices.  Anonymous devices are usually created
through composition and are anonymous because the user controls the device
namespace and the user has no way of allocating names for devices created
through composition.

The bus namespace is parallel to the device namespace.  Unlike the device
namespace, busses cannot be anonymous.  For busses that are created as a 
result
of composition, a name is derived either from the device name or via the 
type
name.

In qdev, implicit bus names are not considered stable and may change across
invocations and/or versions of QEMU.

Property namespaces are local to devices.  The 'id' property is reserved to
refer to the name of the device.  Property names do not reference child 
devices.

Paths cannot be meaningfully constructed in QDev.  Devices can only be 
addressed
directly by their name as there is no stable way to refer to busses under a
device, or children under a bus.

=== Naming in QOM ===

In QOM, there are only two namespaces, the device namespace and the property
namespace.

All devices have unique names.  There are no exceptions.  Devices created
through composition are given unique names by deriving the name based on the
parent device name and a special separator, "::", that cannot be used in 
user
supplied names.

Since a bus is-a device in QOM, there is no notion of having multiple busses
under the same device.  A device can implement multiple bus interfaces, 
but can
only be a single bus of any given bus interface.

Device names are completely independent of pathnames.  For devices that 
are no
user created, device names should be treated as opaque blobs with 
absolutely no
semantic meaning.

All device relationships are identified as named properties.  A QOM path 
name
consists of a named device, followed by a series of properties which may 
or may
not refer to other devices.  For instance, all of the following are 
valid paths:

  /i440fx/piix3/i8042/aux
  /i440fx/slot[1.0]/i8042/aux
  /i440fx/slot[1.0]/bus/piix3/i8042/aux

All of these path names are interpreted as follows:

  def resolve_pathname(pathname):
      root, props = pathname[1:].split('/')
      dev = find_device(root)
      for prop in props:
          device_name = get_property(dev, prop)
          dev = find_device(device_name)
      return dev

In this specific example, the i440fx object has two properties that both 
refer
to the PIIX3 device.  The 'piix3' device is a property that reflects a 
device
composition relationship.  The 'slot[1.0]' property represents a device 
backlink
relationship.

The PIIX3 device has a 'i8042' property based on device composition of 
the PC
Keyboard Controller device.  It also has a device backlink property, 
'bus', that
points to the bus that it sits on (which is the 'i440fx' object).

Finally, the PC Keyboard Controller device has an 'aux' property which is a
device backlink property that can point to a PS/2 Mouse device.

The full set of devices names and properties used in this example are below:

  Type: I440FX
  Is-a: Device
  Implements: PciBus
  Name: i440fx
  Properties:
   piix3: Composition<PIIX3>
   slot[1.0]: Backlink<PciDevice>

  Type: PIIX3
  Isa-a: PciDevice
  Implements: IsaBus
  Name: i440fx::piix3
  Properties:
   i8042: Composition<I8042>
   bus: Backlink<PciDevice> (inherited from PciDevice)

  Type: I8042
  Isa-a: Device
  Implements: Ps2Controller
  Name: i440fx::piix3::i8042
  Properties:
   aux: Backlink<Ps2Mouse>

== Device Properties ==

=== Properties in QDev ===

Device properties in qdev are bound to classes and map directly to 
elements of
the device structure.  They are strongly typed.  Each type is parsed 
directly
from a string representation.  There is no way to set qdev properties from
anything but a string.

Device properties in qdev are only settable during construction.  After
construction, they are read-only.  Devices cannot hook setting or 
getting of a
property.

=== Properties in QOM ===

Device properties in QOM are bound to devices and are implemented by 
closures
provided by the device.  A Visitor is passed to the closure which 
effectively
allows properties to be set/get through native C types.  Mapping to any 
other
type of representation (string or otherwise) is done by a Visitor with no
knowledge of the device or property.

By convention, most device properties are implemented by writing a C typed
public getter/setter for the device, and then using a property wrapper to
translate those typed functions to an untyped closure that accepts a 
Visitor.

QOM has no notion of construction.  All devices are created without 
properties.
Properties can be set/get after initialization.  Devices support the notion
of "realize" which roughly corresponds to construction.  More accurately, it
corresponds to the moment before a device will be first consumed by a guest.

"unrealize" roughly corresponds to reset.  A device may be realized and
unrealized many times during its lifecycle.

Properties are, by default, locked while the device is realized. 
Exceptions can
be made by the devices themselves in order to implement a way for a user to
interact with a device while it is realized.

Two special types of properties are plug<> and socket<> properties.  A 
plug<>
property is used to represent device composition.  When a plug<> property is
added, the child device has its life cycle automatically tied to the parent
device.  The socket<> property represents a device backlink.

The plug<> property type is always read-only and when read, will return the
name of the child device.

The socket<> property type is by default read/write and locked during 
realize.
Hotplug can be implemented by allowing a socket to be writable after 
realize and
installing a custom setter function that implements hotplug behavior.

== TODO ==

  1. Eliminate anonymous devices.
   a. Will require touching any place in the tree that creates a qdev 
object and
      giving a meaningful unique name.

  2. Refactor any device that creates 2 or more busses to only create a 
single
     bus. This will mean using composition.
   a. An example: IDE controller creates two busses, one for the primary 
and one
      for the secondary.  Instead, IDE device should have two IDE 
controller sub
      devices, each device having a single bus.

  3. Unify the bus/device namespaces
   a. This is a compatibility breaker
   b. Depends on (1) and (2)

  4. Modify qdev properties to allow devices to register setters/getters 
that use
     visitors to access properties.
   a. Implement existing properties in terms of visitors

  5. Modify qdev properties to be stored in the object, not in the class

  6. Expose children as named properties
   a. Read only to start with

  7. Change qdev to use QOM typing
   a. Depends on (3)
   b. Must change all init functions to use QOM init functions
   c. Change all DO_UPCASTS to QOM macro casts
   d. Can be largely done via sed

  8. Change children to be based on plug and socket properties
   a. Eliminate children list in each device
   b. Compatibility breaker

  9. Improve object model
   a. Compatibility breaker

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/14/2011 01:04 PM, Anthony Liguori wrote:
> Hi,
>
> I spent a couple hours today writing up some comparisons and an initial task
> list for converting qdev to QOM. The main location of this is the wiki[1] but I
> thought I would inline it here for easier commenting.
>
> I decided to do this because I wanted to avoid a massively long 00 patch
> explaining the rationale for the first batch of changes that I'm going to send out.
>
> There is so much complexity in qdev and the device model in general that it's
> hard to come up with a concise document. I'd really appreciate suggestions for
> topics to write up more rationale as that would help me avoid writing a book on
> the topic :-)
>
> [1] http://wiki.qemu.org/Features/QOM
>
> == Device Relationships ==
>
> === Device Relationships in QDev ===
>
> The two main concepts in QDev are devices and busses. A device is represented
> by a DeviceState and a bus is represented by a BusState. They do not share a
> common base class. Devices can have properties but busses cannot. A device
> has no direct relationship with other devices. The only relationship is
> indirect through busses.
>
> A device may have zero or more busses associated with it via a has-a
> relationship. Each child bus may have multiple devices associated with it via
> a reference. All devices have a single parent bus and all busses have a single
> parent device. These relationships form a strict tree where every alternating
> level is a bus level followed by a device level. The root of the tree is the
> main system bus often referred to as SysBus.
>
> === Device Relationships in QOM ===
>
> Everything in QOM is a device. The concept of busses are implemented as an
> interface that a device implements. Devices can have two types of relationships
> with other devices: device composition or device backlinks. Device composition
> involves one device directly creating another device. It will own life cycle
> management for the created device and will generally propagate events to that
> device (although there are exceptions).
>
> Device backlinks allow one device to refer to another device in a looser
> fashion. While there can be only one device composition relationship that
> exists between two specific devices, a device can participate in an arbitrary
> number of backlinks.
>
> Device composition and backlinks are both strongly typed and can be typed as a
> specific device type or as an interface. When typed as an interface, any
> device that implements that interface can be used.
>
> There is no explicit notion of parents in QOM. A typical bus relationship
> would the bus having a backlink to the child device, and the child device having
> a backlink to the bus.
>
> Without device backlinks, device composition forms a multi-rooted strict tree.
> With backlinks, the result are multiple directed graphs.

Peter often points out that he dislikes the directed nature of the graph links 
in QOM.  I assume he will again so I'll pre-emptively address it :-)

The reason they're directed is because that maps very well to C.  Here's an 
example of how these properties map to C:

struct I440FX {
   Device parent;

   PIIX3 piix3; // device composition
   PciDevice *slots[32]; // device backlink
};

An embedded struct is device composition, a pointer to a struct is a backlink.

Since with device composition, you can only have a link between two nodes, you 
could conceivably add a 'Device *parent' backlink to the Device class which 
could get filled out for any instance of device composition.  I dislike this 
approach because there's not a whole lot of useful things you can do with a 
Device and I suspect most people would upcast it to something else.  I think 
it's far better to have an explicitly typed backlink for those cases where 
composition also involves a bidirectional relationship between devices.

In this specific example, the PIIX3 is-a PciDevice and all PciDevices have a 
PciBus *bus backlink.  That's whole the piix3 should talk to the PciBus IMHO as 
opposed to relying on the composition link.

A bidirectional backlink is even harder to map to C.  I'm not really sure how 
you would do it.  I'm not necessarily opposed to notion of having bidirectional 
links by default but I just don't see a good mapping to something practical.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 4463 bytes --]

On 2011-09-14 20:04, Anthony Liguori wrote:
> Hi,
> 
> I spent a couple hours today writing up some comparisons and an initial 
> task list for converting qdev to QOM.  The main location of this is the 
> wiki[1] but I thought I would inline it here for easier commenting.
> 
> I decided to do this because I wanted to avoid a massively long 00 patch 
> explaining the rationale for the first batch of changes that I'm going 
> to send out.
> 
> There is so much complexity in qdev and the device model in general that 
> it's hard to come up with a concise document.  I'd really appreciate 
> suggestions for topics to write up more rationale as that would help me 
> avoid writing a book on the topic :-)
> 
> [1] http://wiki.qemu.org/Features/QOM
> 
> == Device Relationships ==
> 
> === Device Relationships in QDev ===
> 
> The two main concepts in QDev are devices and busses.  A device is 
> represented
> by a DeviceState and a bus is represented by a BusState.  They do not 
> share a
> common base class.  Devices can have properties but busses cannot.  A device
> has no direct relationship with other devices.  The only relationship is
> indirect through busses.
> 
> A device may have zero or more busses associated with it via a has-a
> relationship.  Each child bus may have multiple devices associated with 
> it via
> a reference.  All devices have a single parent bus and all busses have a 
> single
> parent device.  These relationships form a strict tree where every 
> alternating
> level is a bus level followed by a device level.  The root of the tree 
> is the
> main system bus often referred to as SysBus.
> 
> === Device Relationships in QOM ===
> 
> Everything in QOM is a device.  The concept of busses are implemented as an
> interface that a device implements.  Devices can have two types of 
> relationships
> with other devices: device composition or device backlinks. 

"Backlink" somehow implies to me that there is also a forward link. Why
"back"?

<snip>

> 
> === Naming in QOM ===
> 
> In QOM, there are only two namespaces, the device namespace and the property
> namespace.
> 
> All devices have unique names.

Globally unique?

>  There are no exceptions.  Devices created
> through composition are given unique names by deriving the name based on the
> parent device name and a special separator, "::", that cannot be used in 
> user
> supplied names.

That must be specified in more details. First of all, we likely need to
add a '::' prefix to reflect the root device so that no user can
override any auto-generated first-level device name.

What will be the unique name of some, say, smbus-eeprom?
::i440fx::PIIX4_PM::smbus-eeprom<something>? Will that something be
derived from its bus address? Or a counter that ran while all sibling
eeproms were created?

Such names can get fairly long I'm afraid...

> 
> Since a bus is-a device in QOM, there is no notion of having multiple busses
> under the same device.  A device can implement multiple bus interfaces, 
> but can
> only be a single bus of any given bus interface.
> 
> Device names are completely independent of pathnames.  For devices that 
> are no
> user created, device names should be treated as opaque blobs with 
> absolutely no
> semantic meaning.
> 
> All device relationships are identified as named properties.  A QOM path 
> name
> consists of a named device,

With a system root device called '/'. So '/' is another
character(-sequence) that is forbidden in device names.

> followed by a series of properties which may 
> or may
> not refer to other devices.  For instance, all of the following are 
> valid paths:
> 
>   /i440fx/piix3/i8042/aux
>   /i440fx/slot[1.0]/i8042/aux
>   /i440fx/slot[1.0]/bus/piix3/i8042/aux

Navigating along the properties sounds like a good idea. The properties
(hopefully) have telling names so that it is (generally) also user-friendly.

Nevertheless, I guess we should also establish and enforce some naming
rules for properties to avoid inconsistencies like "i2c" vs.
"my_i2c_devices" or just "d", ie. completely meaningless names. In the
end, variable (property) names now become part of the user interface.

This scheme also avoids having to merge the device namespace completely
into the path namespace, which would cause all kinds of conflicts again.

<snip>
So far for now.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/14/2011 02:30 PM, Jan Kiszka wrote:
> On 2011-09-14 20:04, Anthony Liguori wrote:
>> Hi,
>>
>> I spent a couple hours today writing up some comparisons and an initial
>> task list for converting qdev to QOM.  The main location of this is the
>> wiki[1] but I thought I would inline it here for easier commenting.
>>
>> I decided to do this because I wanted to avoid a massively long 00 patch
>> explaining the rationale for the first batch of changes that I'm going
>> to send out.
>>
>> There is so much complexity in qdev and the device model in general that
>> it's hard to come up with a concise document.  I'd really appreciate
>> suggestions for topics to write up more rationale as that would help me
>> avoid writing a book on the topic :-)
>>
>> [1] http://wiki.qemu.org/Features/QOM
>>
>> == Device Relationships ==
>>
>> === Device Relationships in QDev ===
>>
>> The two main concepts in QDev are devices and busses.  A device is
>> represented
>> by a DeviceState and a bus is represented by a BusState.  They do not
>> share a
>> common base class.  Devices can have properties but busses cannot.  A device
>> has no direct relationship with other devices.  The only relationship is
>> indirect through busses.
>>
>> A device may have zero or more busses associated with it via a has-a
>> relationship.  Each child bus may have multiple devices associated with
>> it via
>> a reference.  All devices have a single parent bus and all busses have a
>> single
>> parent device.  These relationships form a strict tree where every
>> alternating
>> level is a bus level followed by a device level.  The root of the tree
>> is the
>> main system bus often referred to as SysBus.
>>
>> === Device Relationships in QOM ===
>>
>> Everything in QOM is a device.  The concept of busses are implemented as an
>> interface that a device implements.  Devices can have two types of
>> relationships
>> with other devices: device composition or device backlinks.
>
> "Backlink" somehow implies to me that there is also a forward link. Why
> "back"?

Because I prefer to use confusing terminology :-)  link is definitely better. 
I'll s/backlink/link/g

>>
>> === Naming in QOM ===
>>
>> In QOM, there are only two namespaces, the device namespace and the property
>> namespace.
>>
>> All devices have unique names.
>
> Globally unique?

Yes.  It's also unique for all backends too (or will be eventually).

>>   There are no exceptions.  Devices created
>> through composition are given unique names by deriving the name based on the
>> parent device name and a special separator, "::", that cannot be used in
>> user
>> supplied names.
>
> That must be specified in more details. First of all, we likely need to
> add a '::' prefix to reflect the root device so that no user can
> override any auto-generated first-level device name.

Heh, I ended up doing this in my patch series as it turns out :-)

>
> What will be the unique name of some, say, smbus-eeprom?
> ::i440fx::PIIX4_PM::smbus-eeprom<something>?

The PIIX4_PM appears as a function on the PIIX4 device.  That means it will 
appear like:

::i440fx::piix4::pm::smbus-eeprom[N]

 > Will that something be
> derived from its bus address? Or a counter that ran while all sibling
> eeproms were created?

It's derived during creation.

> Such names can get fairly long I'm afraid...

A user should never even see these names.  A user probably will always interact 
with devices via paths.

We can also look at doing things like user-defined aliases or something like that.

>> Since a bus is-a device in QOM, there is no notion of having multiple busses
>> under the same device.  A device can implement multiple bus interfaces,
>> but can
>> only be a single bus of any given bus interface.
>>
>> Device names are completely independent of pathnames.  For devices that
>> are no
>> user created, device names should be treated as opaque blobs with
>> absolutely no
>> semantic meaning.
>>
>> All device relationships are identified as named properties.  A QOM path
>> name
>> consists of a named device,
>
> With a system root device called '/'. So '/' is another
> character(-sequence) that is forbidden in device names.

Yes, but there is no system root device.

In normal scenarios, the main device would be something like the i440fx.  That's 
a device that's explicitly created by the user.

>> followed by a series of properties which may
>> or may
>> not refer to other devices.  For instance, all of the following are
>> valid paths:
>>
>>    /i440fx/piix3/i8042/aux
>>    /i440fx/slot[1.0]/i8042/aux
>>    /i440fx/slot[1.0]/bus/piix3/i8042/aux
>
> Navigating along the properties sounds like a good idea. The properties
> (hopefully) have telling names so that it is (generally) also user-friendly.

Exactly.

> Nevertheless, I guess we should also establish and enforce some naming
> rules for properties to avoid inconsistencies like "i2c" vs.
> "my_i2c_devices" or just "d", ie. completely meaningless names. In the
> end, variable (property) names now become part of the user interface.

Indeed.

> This scheme also avoids having to merge the device namespace completely
> into the path namespace, which would cause all kinds of conflicts again.
>
> <snip>
> So far for now.

Thanks for taking a look!

Regards,

Anthony Liguori

> Jan
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
> Hi,
> 
> I spent a couple hours today writing up some comparisons and an
> initial task list for converting qdev to QOM.  The main location of
> this is the wiki[1] but I thought I would inline it here for easier
> commenting.
> 
> I decided to do this because I wanted to avoid a massively long 00
> patch explaining the rationale for the first batch of changes that
> I'm going to send out.
> 
> There is so much complexity in qdev and the device model in general
> that it's hard to come up with a concise document.  I'd really
> appreciate suggestions for topics to write up more rationale as that
> would help me avoid writing a book on the topic :-)

Thanks Anthony,

I'd appreciate if you could elaborate more on the backlinks. Also,
tiny code examples would help. Maybe you've got a git repo with
more code to link to?

Regarding the composition, a problem I face (which I mostly just
hack my way around) is that inter device connections may exist
at various layers at the same time. For example, a device that is
burried under a couple of busses/bridges may somehow be tighly
connected to another device at a very different location in 
the bus hierarchy by means of another overlayed interconnect (e.g
data channels, timing generation, IRQs etc). Maybe you could
clarify a bit more on how QOM handles these topics.

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/14/2011 03:00 PM, Edgar E. Iglesias wrote:
> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>> Hi,
>>
>> I spent a couple hours today writing up some comparisons and an
>> initial task list for converting qdev to QOM.  The main location of
>> this is the wiki[1] but I thought I would inline it here for easier
>> commenting.
>>
>> I decided to do this because I wanted to avoid a massively long 00
>> patch explaining the rationale for the first batch of changes that
>> I'm going to send out.
>>
>> There is so much complexity in qdev and the device model in general
>> that it's hard to come up with a concise document.  I'd really
>> appreciate suggestions for topics to write up more rationale as that
>> would help me avoid writing a book on the topic :-)
>
> Thanks Anthony,
>
> I'd appreciate if you could elaborate more on the backlinks. Also,
> tiny code examples would help. Maybe you've got a git repo with
> more code to link to?

Message-Id: <1311558293-5855-1-git-send-email-aliguori@us.ibm.com>

Or

http://repo.or.cz/w/qemu/aliguori.git/shortlog/refs/heads/qom

>
> Regarding the composition, a problem I face (which I mostly just
> hack my way around) is that inter device connections may exist
> at various layers at the same time. For example, a device that is
> burried under a couple of busses/bridges may somehow be tighly
> connected to another device at a very different location in
> the bus hierarchy by means of another overlayed interconnect (e.g
> data channels, timing generation, IRQs etc). Maybe you could
> clarify a bit more on how QOM handles these topics.

Links are meant to handle this.  You can have something like:

+ PlatformDevice1
+ PlatformDevice2
+ PciBus
   |
   +- PciDevice1
   +- PciDevice2
      |
      + I2CBus
        |
        +- I2CDevice1
           |- pd2: link<PlatformDevice2>

So you have I2CDevice that sites behind multiple levels of hierarchy, but has a 
direct relationship (via a link) to a platform device much higher in the 
hierarchy.  This is an simple demonstration of how you can create a graph in QOM.

qdev doesn't allow graphs in the object model which is the problem that you're 
having today.  That's because its a tree with alternating levels.  The top level 
is always a bus, the next level is always devices, the next level are always 
busses, etc.

With QOM, there are only devices, and devices can link to 0 or more other devices.

Regards,

Anthony Liguori


>
> Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Wed, Sep 14, 2011 at 03:22:29PM -0500, Anthony Liguori wrote:
> On 09/14/2011 03:00 PM, Edgar E. Iglesias wrote:
> >On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
> >>Hi,
> >>
> >>I spent a couple hours today writing up some comparisons and an
> >>initial task list for converting qdev to QOM.  The main location of
> >>this is the wiki[1] but I thought I would inline it here for easier
> >>commenting.
> >>
> >>I decided to do this because I wanted to avoid a massively long 00
> >>patch explaining the rationale for the first batch of changes that
> >>I'm going to send out.
> >>
> >>There is so much complexity in qdev and the device model in general
> >>that it's hard to come up with a concise document.  I'd really
> >>appreciate suggestions for topics to write up more rationale as that
> >>would help me avoid writing a book on the topic :-)
> >
> >Thanks Anthony,
> >
> >I'd appreciate if you could elaborate more on the backlinks. Also,
> >tiny code examples would help. Maybe you've got a git repo with
> >more code to link to?
> 
> Message-Id: <1311558293-5855-1-git-send-email-aliguori@us.ibm.com>
> 
> Or
> 
> http://repo.or.cz/w/qemu/aliguori.git/shortlog/refs/heads/qom
> 
> >
> >Regarding the composition, a problem I face (which I mostly just
> >hack my way around) is that inter device connections may exist
> >at various layers at the same time. For example, a device that is
> >burried under a couple of busses/bridges may somehow be tighly
> >connected to another device at a very different location in
> >the bus hierarchy by means of another overlayed interconnect (e.g
> >data channels, timing generation, IRQs etc). Maybe you could
> >clarify a bit more on how QOM handles these topics.
> 
> Links are meant to handle this.  You can have something like:
> 
> + PlatformDevice1
> + PlatformDevice2
> + PciBus
>   |
>   +- PciDevice1
>   +- PciDevice2
>      |
>      + I2CBus
>        |
>        +- I2CDevice1
>           |- pd2: link<PlatformDevice2>
> 
> So you have I2CDevice that sites behind multiple levels of
> hierarchy, but has a direct relationship (via a link) to a platform
> device much higher in the hierarchy.  This is an simple
> demonstration of how you can create a graph in QOM.
> 
> qdev doesn't allow graphs in the object model which is the problem
> that you're having today.  That's because its a tree with
> alternating levels.  The top level is always a bus, the next level
> is always devices, the next level are always busses, etc.
> 
> With QOM, there are only devices, and devices can link to 0 or more other devices.


Great, thanks for clarifying!

Maybe you can copy+paste this into your book :p

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Blue Swirl]

On Wed, Sep 14, 2011 at 8:22 PM, Anthony Liguori <aliguori@us.ibm.com> wrote:
> On 09/14/2011 03:00 PM, Edgar E. Iglesias wrote:
>>
>> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>>>
>>> Hi,
>>>
>>> I spent a couple hours today writing up some comparisons and an
>>> initial task list for converting qdev to QOM.  The main location of
>>> this is the wiki[1] but I thought I would inline it here for easier
>>> commenting.
>>>
>>> I decided to do this because I wanted to avoid a massively long 00
>>> patch explaining the rationale for the first batch of changes that
>>> I'm going to send out.
>>>
>>> There is so much complexity in qdev and the device model in general
>>> that it's hard to come up with a concise document.  I'd really
>>> appreciate suggestions for topics to write up more rationale as that
>>> would help me avoid writing a book on the topic :-)
>>
>> Thanks Anthony,
>>
>> I'd appreciate if you could elaborate more on the backlinks. Also,
>> tiny code examples would help. Maybe you've got a git repo with
>> more code to link to?
>
> Message-Id: <1311558293-5855-1-git-send-email-aliguori@us.ibm.com>
>
> Or
>
> http://repo.or.cz/w/qemu/aliguori.git/shortlog/refs/heads/qom

This set only contains the framework. Converting a few real life
devices would be helpful to see the framework in action.

Also, I didn't find anything about backlinks, did I miss something?

>>
>> Regarding the composition, a problem I face (which I mostly just
>> hack my way around) is that inter device connections may exist
>> at various layers at the same time. For example, a device that is
>> burried under a couple of busses/bridges may somehow be tighly
>> connected to another device at a very different location in
>> the bus hierarchy by means of another overlayed interconnect (e.g
>> data channels, timing generation, IRQs etc). Maybe you could
>> clarify a bit more on how QOM handles these topics.
>
> Links are meant to handle this.  You can have something like:
>
> + PlatformDevice1
> + PlatformDevice2
> + PciBus
>  |
>  +- PciDevice1
>  +- PciDevice2
>     |
>     + I2CBus
>       |
>       +- I2CDevice1
>          |- pd2: link<PlatformDevice2>
>
> So you have I2CDevice that sites behind multiple levels of hierarchy, but
> has a direct relationship (via a link) to a platform device much higher in
> the hierarchy.  This is an simple demonstration of how you can create a
> graph in QOM.
>
> qdev doesn't allow graphs in the object model which is the problem that
> you're having today.  That's because its a tree with alternating levels.
>  The top level is always a bus, the next level is always devices, the next
> level are always busses, etc.
>
> With QOM, there are only devices, and devices can link to 0 or more other
> devices.
>
> Regards,
>
> Anthony Liguori
>
>
>>
>> Cheers
>
>
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

On 2011-09-14 21:42, Anthony Liguori wrote:
>> Such names can get fairly long I'm afraid...
> 
> A user should never even see these names.  A user probably will always
> interact with devices via paths.

Right.
<scratching head>
But will those automatic names be used at all then?

> 
> We can also look at doing things like user-defined aliases or something
> like that.

...or a way to set the name of an auto-generated device via its pathname.

> 
>>> Since a bus is-a device in QOM, there is no notion of having multiple
>>> busses
>>> under the same device.  A device can implement multiple bus interfaces,
>>> but can
>>> only be a single bus of any given bus interface.
>>>
>>> Device names are completely independent of pathnames.  For devices that
>>> are no
>>> user created, device names should be treated as opaque blobs with
>>> absolutely no
>>> semantic meaning.
>>>
>>> All device relationships are identified as named properties.  A QOM path
>>> name
>>> consists of a named device,
>>
>> With a system root device called '/'. So '/' is another
>> character(-sequence) that is forbidden in device names.
> 
> Yes, but there is no system root device.

There is always a generic link to some root device. I think it would be
more regular to make that link an abstract device called '/' - maybe
even one that can hold a larger number of children. Keeps the door open
for crazy multi-root systems models.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/14/2011 03:37 PM, Blue Swirl wrote:
> On Wed, Sep 14, 2011 at 8:22 PM, Anthony Liguori<aliguori@us.ibm.com>  wrote:
>> On 09/14/2011 03:00 PM, Edgar E. Iglesias wrote:
>>>
>>> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>>>>
>>>> Hi,
>>>>
>>>> I spent a couple hours today writing up some comparisons and an
>>>> initial task list for converting qdev to QOM.  The main location of
>>>> this is the wiki[1] but I thought I would inline it here for easier
>>>> commenting.
>>>>
>>>> I decided to do this because I wanted to avoid a massively long 00
>>>> patch explaining the rationale for the first batch of changes that
>>>> I'm going to send out.
>>>>
>>>> There is so much complexity in qdev and the device model in general
>>>> that it's hard to come up with a concise document.  I'd really
>>>> appreciate suggestions for topics to write up more rationale as that
>>>> would help me avoid writing a book on the topic :-)
>>>
>>> Thanks Anthony,
>>>
>>> I'd appreciate if you could elaborate more on the backlinks. Also,
>>> tiny code examples would help. Maybe you've got a git repo with
>>> more code to link to?
>>
>> Message-Id:<1311558293-5855-1-git-send-email-aliguori@us.ibm.com>
>>
>> Or
>>
>> http://repo.or.cz/w/qemu/aliguori.git/shortlog/refs/heads/qom
>
> This set only contains the framework. Converting a few real life
> devices would be helpful to see the framework in action.

Unfortunately, I need to semantically change qdev first before I can convert 
real devices if the tree is going to keep working as expected.

>
> Also, I didn't find anything about backlinks, did I miss something?

I had been using the term "socket" to describe [back]links and "plug" to 
describe composition types.

Regards,

Anthony Liguori


>
>>>
>>> Regarding the composition, a problem I face (which I mostly just
>>> hack my way around) is that inter device connections may exist
>>> at various layers at the same time. For example, a device that is
>>> burried under a couple of busses/bridges may somehow be tighly
>>> connected to another device at a very different location in
>>> the bus hierarchy by means of another overlayed interconnect (e.g
>>> data channels, timing generation, IRQs etc). Maybe you could
>>> clarify a bit more on how QOM handles these topics.
>>
>> Links are meant to handle this.  You can have something like:
>>
>> + PlatformDevice1
>> + PlatformDevice2
>> + PciBus
>>   |
>>   +- PciDevice1
>>   +- PciDevice2
>>      |
>>      + I2CBus
>>        |
>>        +- I2CDevice1
>>           |- pd2: link<PlatformDevice2>
>>
>> So you have I2CDevice that sites behind multiple levels of hierarchy, but
>> has a direct relationship (via a link) to a platform device much higher in
>> the hierarchy.  This is an simple demonstration of how you can create a
>> graph in QOM.
>>
>> qdev doesn't allow graphs in the object model which is the problem that
>> you're having today.  That's because its a tree with alternating levels.
>>   The top level is always a bus, the next level is always devices, the next
>> level are always busses, etc.
>>
>> With QOM, there are only devices, and devices can link to 0 or more other
>> devices.
>>
>> Regards,
>>
>> Anthony Liguori
>>
>>
>>>
>>> Cheers
>>
>>
>>
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/14/2011 04:15 PM, Jan Kiszka wrote:
> On 2011-09-14 21:42, Anthony Liguori wrote:
>>> Such names can get fairly long I'm afraid...
>>
>> A user should never even see these names.  A user probably will always
>> interact with devices via paths.
>
> Right.
> <scratching head>
> But will those automatic names be used at all then?

Yes, because QEMU is not going to know anything about path names :-)

Path names should be a concept that exists entirely in the client.  That may be 
HMP or that may be a command line tool (like the proposed qemu script).

The only management interface exposed to the client is:

create_object(type, name)
value = get_object_property(name, property_name)
void set_object_property(name, property_name, value)
props = list_object_properties(name)
names = list_objects()

So names are very important from a QMP perspective, but not something users 
every really see.

>
>>
>> We can also look at doing things like user-defined aliases or something
>> like that.
>
> ...or a way to set the name of an auto-generated device via its pathname.

The auto-generated name is already basically generated from a pathname.  But 
that path isn't necessarily the obvious one from a user's point of view.

>>>> Since a bus is-a device in QOM, there is no notion of having multiple
>>>> busses
>>>> under the same device.  A device can implement multiple bus interfaces,
>>>> but can
>>>> only be a single bus of any given bus interface.
>>>>
>>>> Device names are completely independent of pathnames.  For devices that
>>>> are no
>>>> user created, device names should be treated as opaque blobs with
>>>> absolutely no
>>>> semantic meaning.
>>>>
>>>> All device relationships are identified as named properties.  A QOM path
>>>> name
>>>> consists of a named device,
>>>
>>> With a system root device called '/'. So '/' is another
>>> character(-sequence) that is forbidden in device names.
>>
>> Yes, but there is no system root device.
>
> There is always a generic link to some root device. I think it would be
> more regular to make that link an abstract device called '/' - maybe
> even one that can hold a larger number of children. Keeps the door open
> for crazy multi-root systems models.

 From a client perspective, yes.  The scripts I showed at KVM Forum had '/' 
return the list of all user-created devices as the contents.  It's a nice 
feature but I think it's something that lives in a client, not in the object model.

Regards,

Anthony Liguori

>
> Jan
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
> All device relationships are identified as named properties.  A QOM
> path name
> consists of a named device, followed by a series of properties which
> may or may
> not refer to other devices.  For instance, all of the following are
> valid paths:
> 
>  /i440fx/piix3/i8042/aux
>  /i440fx/slot[1.0]/i8042/aux
>  /i440fx/slot[1.0]/bus/piix3/i8042/aux
> 
Have you looked at device paths generated by get_fw_dev_path() in qdev?
This function generates Open Firmware device path. The difference
between OF device path and the examples above is that OF device path has
a meaning outside of QEMU and can be used by firmware to find a device
a path refers too. Will QOM be able to generate them?

The paths look like:
/pci@i0cf8/ide@1,1/drive@1/disk@0
/pci@i0cf8/isa@1/fdc@03f1/floppy@1
/pci@i0cf8/isa@1/fdc@03f1/floppy@0
/pci@i0cf8/ide@1,1/drive@1/disk@1
/pci@i0cf8/ide@1,1/drive@0/disk@0
/pci@i0cf8/scsi@3/disk@0,0
/pci@i0cf8/ethernet@4/ethernet-phy@0
/pci@i0cf8/ethernet@5/ethernet-phy@0
/pci@i0cf8/ide@1,1/drive@0/disk@1
/pci@i0cf8/isa@1/ide@01e8/drive@0/disk@0
/pci@i0cf8/usb@1,2/network@0/ethernet@0
/pci@i0cf8/usb@1,2/hub@1/network@0/ethernet@0
/rom@genroms/linuxboot.bin

For isa machines:
/isa/ide@0170/drive@0/disk@0
/isa/fdc@03f1/floppy@1
/isa/fdc@03f1/floppy@0
/isa/ide@0170/drive@0/disk@1

 
--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paolo Bonzini]

On 09/14/2011 08:04 PM, Anthony Liguori wrote:
> The concept of busses are implemented as an
> interface that a device implements.

I noticed that you haven't written in the document how to make devices 
reside on a particular bus (PCI, ISA, I2C, ...).

The three possibilities for this are:

* a device implements an interface.  I would rule this out because for 
most buses the devices will need to store some data (PCI: configuration 
data, pointer to the parent bus; ISA: pointer to the parent bus). 
Interfaces are implementation-only, so you have to place the data in 
each device and cause massive code duplication.

* a device inherits from an abstract class, e.g. PCIDevice.  It is 
useful to see how the inheritance tree would look like for two devices 
with a common chipset and multiple interfaces:

     Device
         NE2000
         PCIDevice
             PCI_NE2000  ------> includes a NE2000
             ISA_NE2000  ------> includes a NE2000

* a device is composed with a connector object.  There is no PCIDevice 
class anymore, so the bus has an array of socket<PCIConnector> instead. 
  The case above would look like this

     Device
         NE2000 (abstract)
             PCI_NE2000  ------> includes a PCIConnector
             ISA_NE2000  ------> includes an ISAConnector

Or, if you insist on a closer mapping of real hardware, where there are 
no abstract classes, it would look like this:

     Device
         NE2000
         PCI_NE2000  ------> includes an NE2000 and a PCIConnector
         ISA_NE2000  ------> includes an NE2000 and an ISAConnector


Advantages of abstract classes are pretty obvious, so I will just list 
them: it is more similar to what is done in QDev, and perhaps it is more 
intuitive.


Advantages of connectors include:

* it is more flexible: it lets you choose between a more abstract and a 
more low-level representation (the two hierarchies above);

* you have the option of showing a simpler device tree to the user, 
without the internal composition.  This is important because, unlike 
QDev, composition in QOM is explicit.  So QOM would place NIC properties 
in NE2000, not in *_NE2000 (right?).

* related to this, it keeps property names more stable.  If I understand 
correctly,  if the device starts as ISA-only or PCI-only, i.e.:

     Device
         PCIDevice
             PCI_NE2000

and later you change it to support multiple buses, suddenly properties 
would have to be addressed differently to account for the composition of 
NE2000 inside PCI_NE2000.  You could I guess implement "forwarder 
properties", but that would also lead to more boilerplate code.

Any other opinions?

Paolo

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Stefan Hajnoczi]

On Thu, Sep 15, 2011 at 7:31 AM, Gleb Natapov <gleb@redhat.com> wrote:
> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>> All device relationships are identified as named properties.  A QOM
>> path name
>> consists of a named device, followed by a series of properties which
>> may or may
>> not refer to other devices.  For instance, all of the following are
>> valid paths:
>>
>>  /i440fx/piix3/i8042/aux
>>  /i440fx/slot[1.0]/i8042/aux
>>  /i440fx/slot[1.0]/bus/piix3/i8042/aux
>>
> Have you looked at device paths generated by get_fw_dev_path() in qdev?
> This function generates Open Firmware device path. The difference
> between OF device path and the examples above is that OF device path has
> a meaning outside of QEMU and can be used by firmware to find a device
> a path refers too. Will QOM be able to generate them?
>
> The paths look like:
> /pci@i0cf8/ide@1,1/drive@1/disk@0
> /pci@i0cf8/isa@1/fdc@03f1/floppy@1
> /pci@i0cf8/isa@1/fdc@03f1/floppy@0
> /pci@i0cf8/ide@1,1/drive@1/disk@1
> /pci@i0cf8/ide@1,1/drive@0/disk@0
> /pci@i0cf8/scsi@3/disk@0,0
> /pci@i0cf8/ethernet@4/ethernet-phy@0
> /pci@i0cf8/ethernet@5/ethernet-phy@0
> /pci@i0cf8/ide@1,1/drive@0/disk@1
> /pci@i0cf8/isa@1/ide@01e8/drive@0/disk@0
> /pci@i0cf8/usb@1,2/network@0/ethernet@0
> /pci@i0cf8/usb@1,2/hub@1/network@0/ethernet@0
> /rom@genroms/linuxboot.bin
>
> For isa machines:
> /isa/ide@0170/drive@0/disk@0
> /isa/fdc@03f1/floppy@1
> /isa/fdc@03f1/floppy@0
> /isa/ide@0170/drive@0/disk@1

Yes, it's not clear to me whether the OF device path will be
orthogonal to QOM or not.

Another part where I am not sure yet:
"Device names are completely independent of pathnames.  For devices that are no
user created, device names should be treated as opaque blobs with absolutely no
semantic meaning."

What does this mean?  Perhaps you're saying that:
create_object('rtl8139', 'my-nic')
pci_bus_name = resolve_path('/i440fx')
set_object_property(pci_bus_name, 'slot[2.0]', 'my-nic')

The device name is a unique identifier that can be used to select a
specific device.  The path is a relative way of selecting devices
given a chain of properties - you only use property names in a path,
not device names.  Is this correct?

For convenience I imagine people would prefer this:
create_object('rtl8139', 'my-nic')
set_object_property('/i440fx', 'slot[2.0]', 'my-nic')

So here set_object_property() detects that there is a '/' in the
device name.  Therefore it treats the argument as a path instead of a
device name.  We'd need to add the constraints that paths always start
with '/' and device names may not contain '/'.

Anyway, did I understand the independence of paths and device names correctly?

Stefan

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 05:49 AM, Stefan Hajnoczi wrote:
> On Thu, Sep 15, 2011 at 7:31 AM, Gleb Natapov<gleb@redhat.com>  wrote:
>> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>> For isa machines:
>> /isa/ide@0170/drive@0/disk@0
>> /isa/fdc@03f1/floppy@1
>> /isa/fdc@03f1/floppy@0
>> /isa/ide@0170/drive@0/disk@1
>
> Yes, it's not clear to me whether the OF device path will be
> orthogonal to QOM or not.

It's orthogonal to QOM.

> Another part where I am not sure yet:
> "Device names are completely independent of pathnames.  For devices that are no
> user created, device names should be treated as opaque blobs with absolutely no
> semantic meaning."
>
> What does this mean?  Perhaps you're saying that:
> create_object('rtl8139', 'my-nic')
> pci_bus_name = resolve_path('/i440fx')
> set_object_property(pci_bus_name, 'slot[2.0]', 'my-nic')

More specifically:

def resolve_path(pathname):
     ...

qmp.create_object('rtl8139', 'my-nic')
pci_bus_name = resolve_path('/i440fx')
qmp.set_object_property(pci_bus_name, 'slot[2.0]', 'my-nic')

Paths are a concept of the client.  Maybe we add some path resolution logic to 
QMP as a convenience to help keep clients consistent with each other but from a 
strict perspective, a canonical path representation doesn't exist in QOM.

> The device name is a unique identifier that can be used to select a
> specific device.  The path is a relative way of selecting devices
> given a chain of properties - you only use property names in a path,
> not device names.  Is this correct?

With the exception of the very first component in the path, yes, in the path 
representation I proposed property names are used as the remaining components.

> For convenience I imagine people would prefer this:
> create_object('rtl8139', 'my-nic')
> set_object_property('/i440fx', 'slot[2.0]', 'my-nic')
>
> So here set_object_property() detects that there is a '/' in the
> device name.  Therefore it treats the argument as a path instead of a
> device name.  We'd need to add the constraints that paths always start
> with '/' and device names may not contain '/'.

Yes, this is exactly how my python client code works FWIW.  But 
set_object_property is a local function to the client, not a function of QMP.

Again, we don't need to be this strict in the QMP interface but from a QOM 
design perspective, I think the separation is important.

> Anyway, did I understand the independence of paths and device names correctly?

Yup.

Regards,

Anthony Liguori

> Stefan
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 01:31 AM, Gleb Natapov wrote:
> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>> All device relationships are identified as named properties.  A QOM
>> path name
>> consists of a named device, followed by a series of properties which
>> may or may
>> not refer to other devices.  For instance, all of the following are
>> valid paths:
>>
>>   /i440fx/piix3/i8042/aux
>>   /i440fx/slot[1.0]/i8042/aux
>>   /i440fx/slot[1.0]/bus/piix3/i8042/aux
>>
> Have you looked at device paths generated by get_fw_dev_path() in qdev?

get_fw_dev_path() won't exist in QOM.  The fact that it exists in qdev is a 
problem with qdev.

> This function generates Open Firmware device path.

The function generates *a* OF device path.  OF is not a canonical representation 
of arbitrary hardware.  It's a representation chosen (usually by a human) of 
what information about the hardware is needed by the OS-level software.

If you look at what other folks have done with OF integration in QEMU, you'll 
see a recurring theme of two OF trees, one used to create the hardware and the 
other that is actually exposed to the guest.  The reason you need two is because 
guests sometimes expect very specific things that you really can't generate 
programmatically in every circumstance.

> The difference
> between OF device path and the examples above is that OF device path has
> a meaning outside of QEMU and can be used by firmware to find a device
> a path refers too. Will QOM be able to generate them?

All of the information needed to generate an OF tree is available as device 
properties.  To the extent that you need to knowledge of each bus to generate a 
OF path component, you'll need some extra knowledge of each bus to do that (just 
like with qdev today).  But that knowledge will definitely not be part of QOM.

Paths are not part of QOM.  They're representations used by client software to 
navigate the QOM graph.  There is no real need to make paths part of QOM explicitly.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 01:47 AM, Paolo Bonzini wrote:
> On 09/14/2011 08:04 PM, Anthony Liguori wrote:
>> The concept of busses are implemented as an
>> interface that a device implements.
>
> I noticed that you haven't written in the document how to make devices reside on
> a particular bus (PCI, ISA, I2C, ...).
>
> The three possibilities for this are:
>
> * a device implements an interface. I would rule this out because for most buses
> the devices will need to store some data (PCI: configuration data, pointer to
> the parent bus; ISA: pointer to the parent bus). Interfaces are
> implementation-only, so you have to place the data in each device and cause
> massive code duplication.

I agree.

>
> * a device inherits from an abstract class, e.g. PCIDevice. It is useful to see
> how the inheritance tree would look like for two devices with a common chipset
> and multiple interfaces:
>
> Device
> NE2000
> PCIDevice
> PCI_NE2000 ------> includes a NE2000
> ISA_NE2000 ------> includes a NE2000

I think this model is the closest to what we have today and is the most obvious. 
  For something like ne2k, I would expect:

class NE2000 : public Device
{
   // ne2k public functions
};

class PCI_NE2000 : public PciDevice
{
   // implement PCI functions by calling ne2k public functions
   NE2000 ne2k;
};

class ISA_NE2000 : public IsaDevice
{
   // implement ISA functions by calling ne2k public functions
   NE2000 ne2k;
};

> * a device is composed with a connector object. There is no PCIDevice class
> anymore, so the bus has an array of socket<PCIConnector> instead. The case above
> would look like this
>
> Device
> NE2000 (abstract)
> PCI_NE2000 ------> includes a PCIConnector
> ISA_NE2000 ------> includes an ISAConnector
>
> Or, if you insist on a closer mapping of real hardware, where there are no
> abstract classes, it would look like this:
>
> Device
> NE2000
> PCI_NE2000 ------> includes an NE2000 and a PCIConnector
> ISA_NE2000 ------> includes an NE2000 and an ISAConnector

I think there are two ways to view this:

class PciDevice : public Device
{
    PciConnector connector;
    // init function registers closures with connector that dispatch
    // to abstract functions
};

Or:

class PciConnector : public PciDevice
{
    // provides interfaces to register closures which implement
    // PCI abstract functions
};

I personally lean toward the later as I don't think the PciConnector model 
really does map all that well to hardware (normally, at least).  I think this is 
much closer to how real hardware actually works.

>
> Advantages of abstract classes are pretty obvious, so I will just list them: it
> is more similar to what is done in QDev, and perhaps it is more intuitive.
>
>
> Advantages of connectors include:
>
> * it is more flexible: it lets you choose between a more abstract and a more
> low-level representation (the two hierarchies above);
>
> * you have the option of showing a simpler device tree to the user, without the
> internal composition. This is important because, unlike QDev, composition in QOM
> is explicit. So QOM would place NIC properties in NE2000, not in *_NE2000 (right?).
>
> * related to this, it keeps property names more stable. If I understand
> correctly, if the device starts as ISA-only or PCI-only, i.e.:
>
> Device
> PCIDevice
> PCI_NE2000
>
> and later you change it to support multiple buses, suddenly properties would
> have to be addressed differently to account for the composition of NE2000 inside
> PCI_NE2000. You could I guess implement "forwarder properties", but that would
> also lead to more boilerplate code.
>
> Any other opinions?

The properties thing is definitely an interesting point, but I'm not sure how 
far you can push it.  If you start out with a NE2000 device that is ISA and you 
decide to abstract it to a shared model, all you need to do is keep the ISA 
NE2000 device named NE2000 and call the common chip and PCI bridge something else.

I really think it's important to keep the simple cases simple.  I think any 
model where you don't do:

class E1000 : public PciDevice
{
};

Is unnecessarily complicated.  If it's too complicated, conversions will be much 
slower to do and will be more likely to be done wrong.

Regards,

Anthony Liguori

> Paolo
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paolo Bonzini]

On 09/15/2011 03:26 PM, Anthony Liguori wrote:
> class PciConnector : public PciDevice
> {
> // provides interfaces to register closures which implement
> // PCI abstract functions
> };

or actually even

class PciConnector : Plug
{
     PciBus *bus;
     uint8_t config_space[4096];
     ...
}

> The properties thing is definitely an interesting point, but I'm not
> sure how far you can push it. If you start out with a NE2000 device that
> is ISA and you decide to abstract it to a shared model, all you need to
> do is keep the ISA NE2000 device named NE2000 and call the common chip
> and PCI bridge something else.

It still holds, for two reasons.  1) The properties will be named 
differently for ISA and PCI versions, which is at the very least ugly; 
2) if you want to save the property names for the ISA version, you need 
to duplicate the logic in NE2000 and PCI_NE2000.

If we had C++ and templates, the problem would be much less pressing: 
just use inheritance for PCI and template for ISA vs. PCI.  The code 
duplication at least is invisible to the programmer, it's only in the 
object files.

> I really think it's important to keep the simple cases simple. I think
> any model where you don't do:
>
> class E1000 : public PciDevice
> {
> };
>
> Is unnecessarily complicated.

I don't think it's any more complicated.  It _is_ different from what is 
done now, which may indeed make conversions slower.  That's the main 
disadvantage of the connector model indeed.

Paolo

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Jan Kiszka]

On 2011-09-15 00:11, Anthony Liguori wrote:
> On 09/14/2011 04:15 PM, Jan Kiszka wrote:
>> On 2011-09-14 21:42, Anthony Liguori wrote:
>>>> Such names can get fairly long I'm afraid...
>>>
>>> A user should never even see these names.  A user probably will always
>>> interact with devices via paths.
>>
>> Right.
>> <scratching head>
>> But will those automatic names be used at all then?
> 
> Yes, because QEMU is not going to know anything about path names :-)

I bet that's a needless self-restriction. What prevents reusing the
introspection services that allow path resolutions on the client side
also QEMU internally? It would enable us to skip any traps and pitfalls
associated with unique device name construction. From a higher
perspective, they are completely redundant.

> 
> Path names should be a concept that exists entirely in the client.  That
> may be HMP or that may be a command line tool (like the proposed qemu
> script).
> 
> The only management interface exposed to the client is:
> 
> create_object(type, name)
> value = get_object_property(name, property_name)
> void set_object_property(name, property_name, value)
> props = list_object_properties(name)
> names = list_objects()
> 
> So names are very important from a QMP perspective, but not something
> users every really see.

I don't get the added value of something that looks almost like a path
but is still not as readable at it (e.g. when debugging the communication).

> 
>>
>>>
>>> We can also look at doing things like user-defined aliases or something
>>> like that.
>>
>> ...or a way to set the name of an auto-generated device via its pathname.
> 
> The auto-generated name is already basically generated from a pathname. 
> But that path isn't necessarily the obvious one from a user's point of
> view.
> 
>>>>> Since a bus is-a device in QOM, there is no notion of having multiple
>>>>> busses
>>>>> under the same device.  A device can implement multiple bus
>>>>> interfaces,
>>>>> but can
>>>>> only be a single bus of any given bus interface.
>>>>>
>>>>> Device names are completely independent of pathnames.  For devices
>>>>> that
>>>>> are no
>>>>> user created, device names should be treated as opaque blobs with
>>>>> absolutely no
>>>>> semantic meaning.
>>>>>
>>>>> All device relationships are identified as named properties.  A QOM
>>>>> path
>>>>> name
>>>>> consists of a named device,
>>>>
>>>> With a system root device called '/'. So '/' is another
>>>> character(-sequence) that is forbidden in device names.
>>>
>>> Yes, but there is no system root device.
>>
>> There is always a generic link to some root device. I think it would be
>> more regular to make that link an abstract device called '/' - maybe
>> even one that can hold a larger number of children. Keeps the door open
>> for crazy multi-root systems models.
> 
> From a client perspective, yes.  The scripts I showed at KVM Forum had
> '/' return the list of all user-created devices as the contents.  It's a
> nice feature but I think it's something that lives in a client, not in
> the object model.

This is part of the object model as it defines the tree construction and
if the root can only be a single target-specific device or a generic,
invariant container. This '/' exception as defined so far looks odd to me.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Peter Maydell]

On 15 September 2011 14:35, Paolo Bonzini <pbonzini@redhat.com> wrote:
> It still holds, for two reasons.  1) The properties will be named
> differently for ISA and PCI versions, which is at the very least ugly; 2) if
> you want to save the property names for the ISA version, you need to
> duplicate the logic in NE2000 and PCI_NE2000.
>
> If we had C++ and templates, the problem would be much less pressing: just
> use inheritance for PCI and template for ISA vs. PCI.  The code duplication
> at least is invisible to the programmer, it's only in the object files.

Personally I think the right way to approach this sort of thing is going
to be to have some sort of special purpose language for declaring
devices (and their composition, wiring, interfaces, properties) which is
used to generate all the tedious boilerplate code. Actual interesting code
(corresponding roughly to "method bodies") stays as C functions in
standard C source files.

It's already the case with existing qdev that just instantiating and
setting the properties for a qdev device is sufficiently tedious
that we have lots of utility foo_create() functions scattered around
the codebase. I don't think QOM is going to be any better there.

More generally, I don't think we should let our conceptual model of
how devices are composed and connected be warped by whether you can
implement it with C-and-a-pile-of-macros or not.

-- PMM

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 01:31 AM, Gleb Natapov wrote:
> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>> All device relationships are identified as named properties.  A QOM
>> path name
>> consists of a named device, followed by a series of properties which
>> may or may
>> not refer to other devices.  For instance, all of the following are
>> valid paths:
>>
>>   /i440fx/piix3/i8042/aux
>>   /i440fx/slot[1.0]/i8042/aux
>>   /i440fx/slot[1.0]/bus/piix3/i8042/aux
>>
> Have you looked at device paths generated by get_fw_dev_path() in qdev?
> This function generates Open Firmware device path. The difference
> between OF device path and the examples above is that OF device path has
> a meaning outside of QEMU and can be used by firmware to find a device
> a path refers too. Will QOM be able to generate them?
>
> The paths look like:
> /pci@i0cf8/ide@1,1/drive@1/disk@0
> /pci@i0cf8/isa@1/fdc@03f1/floppy@1
> /pci@i0cf8/isa@1/fdc@03f1/floppy@0
> /pci@i0cf8/ide@1,1/drive@1/disk@1
> /pci@i0cf8/ide@1,1/drive@0/disk@0
> /pci@i0cf8/scsi@3/disk@0,0
> /pci@i0cf8/ethernet@4/ethernet-phy@0
> /pci@i0cf8/ethernet@5/ethernet-phy@0
> /pci@i0cf8/ide@1,1/drive@0/disk@1
> /pci@i0cf8/isa@1/ide@01e8/drive@0/disk@0
> /pci@i0cf8/usb@1,2/network@0/ethernet@0
> /pci@i0cf8/usb@1,2/hub@1/network@0/ethernet@0
> /rom@genroms/linuxboot.bin
>
> For isa machines:
> /isa/ide@0170/drive@0/disk@0
> /isa/fdc@03f1/floppy@1
> /isa/fdc@03f1/floppy@0
> /isa/ide@0170/drive@0/disk@1

A critical point that I neglected to previously mention is that while qdev has a 
canonical path (only one path, really) to address a device, QOM does not.  That 
means a different approach is needed to make the graph look like a tree as OF 
requires.

You need something like this:

void generate_tree(Device *node)
{
    if (IS_PCI_BUS(node)) {
       for (i = 0; i < 32; i++) {
          generate_tree(lookup_device(get_property(node, "slot[%d]", i)));
       }
    } else if (IS_ISA_BUS(node)) {
       ....
    } else {
       // leaf node, generate path segment
    }
}

There are certainly ways to walk the graph generically (by coloring or following 
the composition paths) but that won't give you the desired ordering.

Regards,

Anthony Liguori
}

>
>
> --
> 			Gleb.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 08:43 AM, Jan Kiszka wrote:
> On 2011-09-15 00:11, Anthony Liguori wrote:
>> On 09/14/2011 04:15 PM, Jan Kiszka wrote:
>>> On 2011-09-14 21:42, Anthony Liguori wrote:
>>>>> Such names can get fairly long I'm afraid...
>>>>
>>>> A user should never even see these names.  A user probably will always
>>>> interact with devices via paths.
>>>
>>> Right.
>>> <scratching head>
>>> But will those automatic names be used at all then?
>>
>> Yes, because QEMU is not going to know anything about path names :-)
>
> I bet that's a needless self-restriction. What prevents reusing the
> introspection services that allow path resolutions on the client side
> also QEMU internally? It would enable us to skip any traps and pitfalls
> associated with unique device name construction. From a higher
> perspective, they are completely redundant.

I actually agree :-)

We should probably pick a path format and implement in QMP.  I think that 
discussion is orthogonal though.

>> Path names should be a concept that exists entirely in the client.  That
>> may be HMP or that may be a command line tool (like the proposed qemu
>> script).
>>
>> The only management interface exposed to the client is:
>>
>> create_object(type, name)
>> value = get_object_property(name, property_name)
>> void set_object_property(name, property_name, value)
>> props = list_object_properties(name)
>> names = list_objects()
>>
>> So names are very important from a QMP perspective, but not something
>> users every really see.
>
> I don't get the added value of something that looks almost like a path
> but is still not as readable at it (e.g. when debugging the communication).

It's two separate namespaces.  The name namespace is controlled by the user and 
we have to bend over backwards to avoid clashing with it.

The path namespace is controller by QEMU (more or less).

The name namespace also maps 1-1 to devices which means names can be used to 
represent devices.  They absolutely never change as long as the device never 
changes.

Paths maps N-1 to devices.  Paths may change but names never change.  I don't 
think there can ever be a fixed canonical path.

An example is a NIC with nvram that stores a mac address.  In QOM, the guest 
could change the mac address, then a user could hot unplug the device, and then 
hot plug the device into a different PCI slot.  The path is now different but 
the device name has not change.

>>  From a client perspective, yes.  The scripts I showed at KVM Forum had
>> '/' return the list of all user-created devices as the contents.  It's a
>> nice feature but I think it's something that lives in a client, not in
>> the object model.
>
> This is part of the object model as it defines the tree construction and
> if the root can only be a single target-specific device or a generic,
> invariant container. This '/' exception as defined so far looks odd to me.

I don't think I understand you're suggestion of having a special '/' device 
name.  I think we may be talking past each other.

In QOM, whenever a device is created, it's created with a name.  This basically is:

void type_init(void *memory, const char *name)
{
    global_object_hash_table[name] = memory;
}

Looking up a device by name is just a hash table lookup.  In the case of 
composition, you have:

struct MyDevice {
   struct MyComposedDevice foo;
   MyDevice *sibling;
};

void my_device_init(MyDevice *obj, const char *name)
{
     type_init(obj, name);
     type_init(&obj->foo, name + "::foo");
}

Which gives you two devices in the global hash table.  Links are created by 
basically having:

void my_device_set_sibling(MyDevice *obj, MyDevice *sibling)
{
    obj->sibling = sibling;
}

void my_device_init(MyDevice *obj, const char *name)
{
     ...
     type_add_property(&obj, "sibling", my_device_set_sibling, "link<MyDevice>");
}

type_add_property() has special sauce that lets you write the "sibling" property 
as a string, does the lookup and error checking, and ultimately calls 
my_device_set_sibling() with a Device * pointer.

There is no root.  You can create many completely separate graphs.

Regards,

Anthony Liguori

> Jan
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paolo Bonzini]

On 09/15/2011 03:57 PM, Anthony Liguori wrote:
>
> void generate_tree(Device *node)
> {
>     if (IS_PCI_BUS(node)) {
>        for (i = 0; i < 32; i++) {
>           generate_tree(lookup_device(get_property(node, "slot[%d]", i)));
>        }
>     } else if (IS_ISA_BUS(node)) {
>        ....
>     } else {
>        // leaf node, generate path segment
>     }
> }
>
> There are certainly ways to walk the graph generically (by coloring or
> following the composition paths) but that won't give you the desired
> ordering.

It seems easier to go backwards from the target device.

Each device most likely will have a canonical parent link, and together 
they will give the OF path.

Paolo

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 08:54 AM, Peter Maydell wrote:
> On 15 September 2011 14:35, Paolo Bonzini<pbonzini@redhat.com>  wrote:
>> It still holds, for two reasons.  1) The properties will be named
>> differently for ISA and PCI versions, which is at the very least ugly; 2) if
>> you want to save the property names for the ISA version, you need to
>> duplicate the logic in NE2000 and PCI_NE2000.
>>
>> If we had C++ and templates, the problem would be much less pressing: just
>> use inheritance for PCI and template for ISA vs. PCI.  The code duplication
>> at least is invisible to the programmer, it's only in the object files.
>
> Personally I think the right way to approach this sort of thing is going
> to be to have some sort of special purpose language for declaring
> devices (and their composition, wiring, interfaces, properties) which is
> used to generate all the tedious boilerplate code. Actual interesting code
> (corresponding roughly to "method bodies") stays as C functions in
> standard C source files.
>
> It's already the case with existing qdev that just instantiating and
> setting the properties for a qdev device is sufficiently tedious
> that we have lots of utility foo_create() functions scattered around
> the codebase. I don't think QOM is going to be any better there.
>
> More generally, I don't think we should let our conceptual model of
> how devices are composed and connected be warped by whether you can
> implement it with C-and-a-pile-of-macros or not.

I think there are two real discussions happening here.  One is how to model and 
one is what is the right representation of that modelling.

I think Paolo is strongly advocating a non-OO model that uses the command 
pattern to establish most of the relationships.  Maybe one level of inheritance 
is used just to provide the right infrastructure for the objects to have 
properties but that's inheritance for functionality verses modelling.

I'm advocating a strict OO model.  I'm not normally a fan of OO based design but 
I think it fits a device model pretty well because it's such a concrete 
hierarchy.  All of the objects we create really correspond to real world objects 
and the inheritance relationships are pretty darn obvious.  When you're holding 
a PCI NE2000 card in your hand, there's simply no doubt that it is-a PciDevice.

And you can look at the PCB and see that it has-a NE2K chip on it.

Regards,

Anthony Liguori

>
> -- PMM
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 08:17:13AM -0500, Anthony Liguori wrote:
> On 09/15/2011 01:31 AM, Gleb Natapov wrote:
> >On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
> >>All device relationships are identified as named properties.  A QOM
> >>path name
> >>consists of a named device, followed by a series of properties which
> >>may or may
> >>not refer to other devices.  For instance, all of the following are
> >>valid paths:
> >>
> >>  /i440fx/piix3/i8042/aux
> >>  /i440fx/slot[1.0]/i8042/aux
> >>  /i440fx/slot[1.0]/bus/piix3/i8042/aux
> >>
> >Have you looked at device paths generated by get_fw_dev_path() in qdev?
> 
> get_fw_dev_path() won't exist in QOM.  The fact that it exists in
> qdev is a problem with qdev.
> 
I do not need get_fw_dev_path() as such. I need the way to generate OF
device path for a device. I hope you are not saying that the fact that
we generate OF path is the problem of qdev. OF device path is an ABI
between QEMU and a firmware. Firmware can't do much with QOM device
paths.

> >This function generates Open Firmware device path.
> 
> The function generates *a* OF device path.  OF is not a canonical
> representation of arbitrary hardware.  It's a representation chosen
> (usually by a human) of what information about the hardware is
> needed by the OS-level software.
> 
Of course it is chosen by human, just like QOM is the representation
chosen by human :) But human made sure that it presents enough information
for OS level software to unambiguously determine a device a path points
too.

> If you look at what other folks have done with OF integration in
> QEMU, you'll see a recurring theme of two OF trees, one used to
> create the hardware and the other that is actually exposed to the
> guest.  The reason you need two is because guests sometimes expect
> very specific things that you really can't generate programmatically
> in every circumstance.
> 
> >The difference
> >between OF device path and the examples above is that OF device path has
> >a meaning outside of QEMU and can be used by firmware to find a device
> >a path refers too. Will QOM be able to generate them?
> 
> All of the information needed to generate an OF tree is available as
> device properties.  To the extent that you need to knowledge of each
> bus to generate a OF path component, you'll need some extra
> knowledge of each bus to do that (just like with qdev today).  But
> that knowledge will definitely not be part of QOM.
> 
With qdev buses are different from devices so it is very clear where to
put that extra knowledge (inside get_fw_dev_path callback of a bus). How
are you going to do that with QOM? As you said in your other email QOM
is a graph, so dumb recursion will not work like it did in qdev. At each
node you need to know which link to take.

> Paths are not part of QOM.  They're representations used by client
> software to navigate the QOM graph.  There is no real need to make
> paths part of QOM explicitly.
> 
I am not saying there is. I just what to make sure that we will not
regress by moving to QOM.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 04:14:45PM +0200, Paolo Bonzini wrote:
> On 09/15/2011 03:57 PM, Anthony Liguori wrote:
> >
> >void generate_tree(Device *node)
> >{
> >    if (IS_PCI_BUS(node)) {
> >       for (i = 0; i < 32; i++) {
> >          generate_tree(lookup_device(get_property(node, "slot[%d]", i)));
> >       }
> >    } else if (IS_ISA_BUS(node)) {
> >       ....
> >    } else {
> >       // leaf node, generate path segment
> >    }
> >}
> >
> >There are certainly ways to walk the graph generically (by coloring or
> >following the composition paths) but that won't give you the desired
> >ordering.
> 
> It seems easier to go backwards from the target device.
That what we do in qdev.

> 
> Each device most likely will have a canonical parent link, and
> together they will give the OF path.
> 
Yeah, this canonical parent link should be marked somehow.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paolo Bonzini]

On 09/15/2011 04:18 PM, Anthony Liguori wrote:
> I think there are two real discussions happening here.  One is how to
> model and one is what is the right representation of that modelling.
>
> I think Paolo is strongly advocating a non-OO model that uses the
> command pattern to establish most of the relationships.  Maybe one level
> of inheritance is used just to provide the right infrastructure for the
> objects to have properties but that's inheritance for functionality
> verses modelling.
>
> I'm advocating a strict OO model.  I'm not normally a fan of OO based
> design but I think it fits a device model pretty well because it's such
> a concrete hierarchy.  All of the objects we create really correspond to
> real world objects and the inheritance relationships are pretty darn
> obvious.  When you're holding a PCI NE2000 card in your hand, there's
> simply no doubt that it is-a PciDevice.

I think it's in the eye of a beholder.  Hold a PCI NE2000 and E1000, 
they're clearly both PciDevices, but also they clearly both have 
PciConnectors.

Write a driver for a PCI and ISA NE2000, and then they're clearly both 
NE2000, but also they clearly both have an NE2K chip.

So, I don't find either model to be more or less OO.  It just boils down 
to what you pick as your primary choice for inheritance.  It is true 
that my model will produce a more shallow class hierarchies, but only 
because IMO hardware is more about composition than inheritance; that's 
why I am indeed advocating that relationships be established mostly by 
composition.  (I didn't understand your reference to the command 
pattern, but it's probably not so important).

Secondarily, I'd hate to force a particular inheritance model, because 
class inheritance is a very scarce resource---you can only inherit from 
one class at a time.

Paolo

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Peter Maydell]

On 15 September 2011 15:33, Paolo Bonzini <pbonzini@redhat.com> wrote:
> I think it's in the eye of a beholder.  Hold a PCI NE2000 and E1000, they're
> clearly both PciDevices, but also they clearly both have PciConnectors.
>
> Write a driver for a PCI and ISA NE2000, and then they're clearly both
> NE2000, but also they clearly both have an NE2K chip.
>
> So, I don't find either model to be more or less OO.  It just boils down to
> what you pick as your primary choice for inheritance.  It is true that my
> model will produce a more shallow class hierarchies, but only because IMO
> hardware is more about composition than inheritance; that's why I am indeed
> advocating that relationships be established mostly by composition.

Yes. In particular if you're trying to connect one bit of hardware
to another bit of hardware, you don't care at all about whether
the hardware is-a-Foo or is-a-Bar, you care only whether it has-a
set of connections which semantically and logically match up to
the other end. This suggests to me that the class hierarchy
should be whatever is convenient for internal implementation but
shouldn't be relevant or visible to between-device relationships.

-- PMM

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> On Thu, Sep 15, 2011 at 04:14:45PM +0200, Paolo Bonzini wrote:
>> On 09/15/2011 03:57 PM, Anthony Liguori wrote:
>>>
>>> void generate_tree(Device *node)
>>> {
>>>     if (IS_PCI_BUS(node)) {
>>>        for (i = 0; i<  32; i++) {
>>>           generate_tree(lookup_device(get_property(node, "slot[%d]", i)));
>>>        }
>>>     } else if (IS_ISA_BUS(node)) {
>>>        ....
>>>     } else {
>>>        // leaf node, generate path segment
>>>     }
>>> }
>>>
>>> There are certainly ways to walk the graph generically (by coloring or
>>> following the composition paths) but that won't give you the desired
>>> ordering.
>>
>> It seems easier to go backwards from the target device.
> That what we do in qdev.
>
>>
>> Each device most likely will have a canonical parent link, and
>> together they will give the OF path.
>>
> Yeah, this canonical parent link should be marked somehow.

There is no canonical parent link.  A device may have multiple (more or less 
equivalent) parents.

What should be treated as the "canonical" link depends on what you're trying to 
do.  In the case of OF, you want to treat the bus as a parent.  If a device 
happens to sit on multiple buses, I'm not really sure what you do.

Regards,

Anthony Liguori

>
> --
> 			Gleb.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 09:33 AM, Paolo Bonzini wrote:
> On 09/15/2011 04:18 PM, Anthony Liguori wrote:
>> I think there are two real discussions happening here. One is how to
>> model and one is what is the right representation of that modelling.
>>
>> I think Paolo is strongly advocating a non-OO model that uses the
>> command pattern to establish most of the relationships. Maybe one level
>> of inheritance is used just to provide the right infrastructure for the
>> objects to have properties but that's inheritance for functionality
>> verses modelling.
>>
>> I'm advocating a strict OO model. I'm not normally a fan of OO based
>> design but I think it fits a device model pretty well because it's such
>> a concrete hierarchy. All of the objects we create really correspond to
>> real world objects and the inheritance relationships are pretty darn
>> obvious. When you're holding a PCI NE2000 card in your hand, there's
>> simply no doubt that it is-a PciDevice.
>
> I think it's in the eye of a beholder. Hold a PCI NE2000 and E1000, they're
> clearly both PciDevices, but also they clearly both have PciConnectors.
>
> Write a driver for a PCI and ISA NE2000, and then they're clearly both NE2000,
> but also they clearly both have an NE2K chip.

Are we just bike shedding then?  Since both models can be expressed by the same 
infrastructure, does it really matter which model is the One True Right Model?

I think this might be a better discussion to have during conversions.  IOW, what 
makes sense for PCI, what makes sense for ISA, etc.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
> On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> >On Thu, Sep 15, 2011 at 04:14:45PM +0200, Paolo Bonzini wrote:
> >>On 09/15/2011 03:57 PM, Anthony Liguori wrote:
> >>>
> >>>void generate_tree(Device *node)
> >>>{
> >>>    if (IS_PCI_BUS(node)) {
> >>>       for (i = 0; i<  32; i++) {
> >>>          generate_tree(lookup_device(get_property(node, "slot[%d]", i)));
> >>>       }
> >>>    } else if (IS_ISA_BUS(node)) {
> >>>       ....
> >>>    } else {
> >>>       // leaf node, generate path segment
> >>>    }
> >>>}
> >>>
> >>>There are certainly ways to walk the graph generically (by coloring or
> >>>following the composition paths) but that won't give you the desired
> >>>ordering.
> >>
> >>It seems easier to go backwards from the target device.
> >That what we do in qdev.
> >
> >>
> >>Each device most likely will have a canonical parent link, and
> >>together they will give the OF path.
> >>
> >Yeah, this canonical parent link should be marked somehow.
> 
> There is no canonical parent link.  A device may have multiple (more
> or less equivalent) parents.
> 
> What should be treated as the "canonical" link depends on what
> you're trying to do.  In the case of OF, you want to treat the bus
> as a parent.  If a device happens to sit on multiple buses, I'm not
> really sure what you do.
> 
Yes, "canonical" is a link to a bus. Can you give an example of a device
that sits on multiple buses?

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paolo Bonzini]

On 09/15/2011 05:31 PM, Anthony Liguori wrote:
>>
>> I think it's in the eye of a beholder. Hold a PCI NE2000 and E1000,
>> they're clearly both PciDevices, but also they clearly both have PciConnectors.
>>
>> Write a driver for a PCI and ISA NE2000, and then they're clearly both
>> NE2000, but also they clearly both have an NE2K chip.
>
> Are we just bike shedding then?  Since both models can be expressed by
> the same infrastructure, does it really matter which model is the One
> True Right Model?

Well, it matters when "how do we model PCI" covers 90% of the devices 
around. :)  In general, early conversions tend to become models for 
later ones.

It's certainly good that QOM can be applied to either model.  If it 
couldn't, something would be very wrong in it.

> I think this might be a better discussion to have during conversions.
> IOW, what makes sense for PCI, what makes sense for ISA, etc.

More pragmatically, what I am suggesting is a refactoring of how PCI is 
abstracted, and that obviously makes it less appealing than a straight 
conversion.

So, it would have been nice to have it as the one true right model, and 
as an example of how to do buses in QOM, but it is indeed safer to do it 
in two steps.  Connectors can be done later when somebody will really 
need to inherit from something else.

Paolo

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 10:38 AM, Gleb Natapov wrote:
> On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
>> On 09/15/2011 09:25 AM, Gleb Natapov wrote:
>>
>> There is no canonical parent link.  A device may have multiple (more
>> or less equivalent) parents.
>>
>> What should be treated as the "canonical" link depends on what
>> you're trying to do.  In the case of OF, you want to treat the bus
>> as a parent.  If a device happens to sit on multiple buses, I'm not
>> really sure what you do.
>>
> Yes, "canonical" is a link to a bus. Can you give an example of a device
> that sits on multiple buses?

Not all devices buses that they sit on.

A good example is our favorite one to debate--the PIIX3.  Devices like the UART 
don't sit on a bus.  They don't have any links at all.

Instead, the PIIX3 itself bridges the public interface of the UART via its PC 
interface.  So it looks something like this:

class Serial : public Device
{
    uint8_t read(uint8_t addr);
};

class PIIX3 : public PciDevice, implements PciBus
{
    Serial *tty[4];
};

uint32_t PIIX3::pio_access(uint16_t addr, int size)
{
    if (addr == 0x3f8 && this->tty[0]) {
        return this->tty[0]->read(addr - 0x3f8);
    } else {
        ...
    }
}

There is no backlink in the Serial device so there's no way of walking up the 
graph from the Serial device itself.  You have to transverse to it to build a path.

Regards,

Anthony Liguori

>
> --
> 			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Jan Kiszka]

On 2011-09-15 16:11, Anthony Liguori wrote:
> An example is a NIC with nvram that stores a mac address.  In QOM, the guest 
> could change the mac address, then a user could hot unplug the device, and then 
> hot plug the device into a different PCI slot.  The path is now different but 
> the device name has not change.

One of the many paths is still "<unique-nic-name>/nvram", so nothing
changed.

Auto-generated names are not needed, just use paths. In the end a device
name is a path as well, one with only a single element.

>>>  From a client perspective, yes.  The scripts I showed at KVM Forum had
>>> '/' return the list of all user-created devices as the contents.  It's a
>>> nice feature but I think it's something that lives in a client, not in
>>> the object model.
>>
>> This is part of the object model as it defines the tree construction and
>> if the root can only be a single target-specific device or a generic,
>> invariant container. This '/' exception as defined so far looks odd to me.
> 
> I don't think I understand you're suggestion of having a special '/' device 
> name.  I think we may be talking past each other.
> 
> In QOM, whenever a device is created, it's created with a name.  This basically is:
> 
> void type_init(void *memory, const char *name)
> {
>     global_object_hash_table[name] = memory;
> }
> 
> Looking up a device by name is just a hash table lookup.  In the case of 
> composition, you have:
> 
> struct MyDevice {
>    struct MyComposedDevice foo;
>    MyDevice *sibling;
> };
> 
> void my_device_init(MyDevice *obj, const char *name)
> {
>      type_init(obj, name);
>      type_init(&obj->foo, name + "::foo");
> }
> 
> Which gives you two devices in the global hash table.  Links are created by 
> basically having:
> 
> void my_device_set_sibling(MyDevice *obj, MyDevice *sibling)
> {
>     obj->sibling = sibling;
> }
> 
> void my_device_init(MyDevice *obj, const char *name)
> {
>      ...
>      type_add_property(&obj, "sibling", my_device_set_sibling, "link<MyDevice>");
> }
> 
> type_add_property() has special sauce that lets you write the "sibling" property 
> as a string, does the lookup and error checking, and ultimately calls 
> my_device_set_sibling() with a Device * pointer.
> 
> There is no root.  You can create many completely separate graphs.

You need a "default" root to make the graphs reasonably discoverable. If
I ask QEMU "How does my system look like?" I don't want to get a flat
list of all devices in it. Roots are where creation started, the
chipsets, the SOCs etc. It's an organizational matter at least, but
that's one thing what paths are about...

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
> On 09/15/2011 10:38 AM, Gleb Natapov wrote:
> >On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
> >>On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> >>
> >>There is no canonical parent link.  A device may have multiple (more
> >>or less equivalent) parents.
> >>
> >>What should be treated as the "canonical" link depends on what
> >>you're trying to do.  In the case of OF, you want to treat the bus
> >>as a parent.  If a device happens to sit on multiple buses, I'm not
> >>really sure what you do.
> >>
> >Yes, "canonical" is a link to a bus. Can you give an example of a device
> >that sits on multiple buses?
> 
> Not all devices buses that they sit on.
> 
Missing "have"? If device has no bus how do you talk to it? Who carries
the signal from a cpu to a device?

> A good example is our favorite one to debate--the PIIX3.  Devices
PIIX3 is a collection of devices, not a device.

> like the UART don't sit on a bus.  They don't have any links at all.
In PC UART sits on isa bus. How device can have no links at all? It just
glued to a motherboard not touching any wires?

> 
> Instead, the PIIX3 itself bridges the public interface of the UART
> via its PC interface.  So it looks something like this:
Again you are talking about particular HW implantation, not SW interface
that the package implements and we need to emulate the later not the
former.

> 
> class Serial : public Device
> {
>    uint8_t read(uint8_t addr);
> };
> 
> class PIIX3 : public PciDevice, implements PciBus
> {
>    Serial *tty[4];
> };
> 
Make this isa slots, encapsulate UART into isa device and plug one
into another. You just cut corners here. You wouldn't do the same for
PCI device, but theoretically you can. You will be able to compose new
chipset with config file too!

> uint32_t PIIX3::pio_access(uint16_t addr, int size)
> {
>    if (addr == 0x3f8 && this->tty[0]) {
>        return this->tty[0]->read(addr - 0x3f8);
>    } else {
>        ...
>    }
> }
Wow, hard coded that way? So another chipset implementation will
have to duplicate the same exact code?

> 
> There is no backlink in the Serial device so there's no way of
> walking up the graph from the Serial device itself.  You have to
> transverse to it to build a path.
> 
That because implementation cut corners (read "incorrect"). Actually
ability to walk the graph all the way up is a good test that we didn't
cheated anywhere by pretending that devices are connected by magic.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 11:59 AM, Gleb Natapov wrote:
> On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
>> On 09/15/2011 10:38 AM, Gleb Natapov wrote:
>>> On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
>>>> On 09/15/2011 09:25 AM, Gleb Natapov wrote:
>>>>
>>>> There is no canonical parent link.  A device may have multiple (more
>>>> or less equivalent) parents.
>>>>
>>>> What should be treated as the "canonical" link depends on what
>>>> you're trying to do.  In the case of OF, you want to treat the bus
>>>> as a parent.  If a device happens to sit on multiple buses, I'm not
>>>> really sure what you do.
>>>>
>>> Yes, "canonical" is a link to a bus. Can you give an example of a device
>>> that sits on multiple buses?
>>
>> Not all devices buses that they sit on.
>>
> Missing "have"? If device has no bus how do you talk to it? Who carries
> the signal from a cpu to a device?
>
>> A good example is our favorite one to debate--the PIIX3.  Devices
> PIIX3 is a collection of devices, not a device.
>
>> like the UART don't sit on a bus.  They don't have any links at all.
> In PC UART sits on isa bus. How device can have no links at all? It just
> glued to a motherboard not touching any wires?

A bus implies a bidirectional relationship.  IOW, the device has to know that it 
sits on a ISA bus to be an ISA device.

The UART has no knowledge of the fact that is mapped behind ISA.  The UART 
exposes a public interface (through it's pins) that's orthogonal to any buses.

That public interface may be consumed by 1 or more consumers.  Some pins may be 
used by one device while other pins are used by another device.

Some other logic maps the UART to the rest of the system.  My view of modelling 
the PIIX3 that I illustrated in this thread is that the PIIX3 encompasses this 
logic.

Another way to view it, the PIIX3 has direct knowledge of a UART's public 
interface but a UART has no knowledge of the PIIX3's public interface.

>> Instead, the PIIX3 itself bridges the public interface of the UART
>> via its PC interface.  So it looks something like this:
> Again you are talking about particular HW implantation, not SW interface
> that the package implements and we need to emulate the later not the
> former.

I'm not.  There is no way for a UART to talk to the PIIX3.  You can't query it 
to figure out where it lives in the device graph.  All it can do is generate an 
interrupt and hope something is listening.  There is no hardware equivalent of 
'cpu_register_physical_memory'.  Some other piece of hardware has to route 
requests to it in a form that it understands.

>
>>
>> class Serial : public Device
>> {
>>     uint8_t read(uint8_t addr);
>> };
>>
>> class PIIX3 : public PciDevice, implements PciBus
>> {
>>     Serial *tty[4];
>> };
>>
> Make this isa slots, encapsulate UART into isa device and plug one
> into another. You just cut corners here. You wouldn't do the same for
> PCI device, but theoretically you can. You will be able to compose new
> chipset with config file too!

But then you need to have:

class IsaSerial : public IsaDevice
{
    Serial uart;
};

All you've done is take the dispatch logic and moved it from PIIX3 to IsaSerial 
for the only purpose of creating an artificial abstraction (of having a bus that 
doesn't exist).

All devices don't sit on buses.  That's the main design problem of qdev.

>> uint32_t PIIX3::pio_access(uint16_t addr, int size)
>> {
>>     if (addr == 0x3f8&&  this->tty[0]) {
>>         return this->tty[0]->read(addr - 0x3f8);
>>     } else {
>>         ...
>>     }
>> }
> Wow, hard coded that way? So another chipset implementation will
> have to duplicate the same exact code?

Yup.  We're moving to this model anyway with the memory API.  The Serial device 
will export a MemoryRegion where its registers start at address 0 and then a 
higher layer (IsaSerial or PIIX3) will add that MemoryRegion to it's 
MemoryRegion starting at a given offset.

>> There is no backlink in the Serial device so there's no way of
>> walking up the graph from the Serial device itself.  You have to
>> transverse to it to build a path.
>>
> That because implementation cut corners (read "incorrect"). Actually
> ability to walk the graph all the way up is a good test that we didn't
> cheated anywhere by pretending that devices are connected by magic.

How do you "walk up the device graph" from a 16650A?  What signals are you going 
to send out of the pins to do that?

If a device can always do self->parent->parent->parent->send_io(foo) then the 
design is fundamentally broken and you will end up with devices that do things 
that they shouldn't do.

Regards,

Anthony Liguori

>
> --
> 			Gleb.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 11:38 AM, Jan Kiszka wrote:
> On 2011-09-15 16:11, Anthony Liguori wrote:
>> An example is a NIC with nvram that stores a mac address.  In QOM, the guest
>> could change the mac address, then a user could hot unplug the device, and then
>> hot plug the device into a different PCI slot.  The path is now different but
>> the device name has not change.
>
> One of the many paths is still "<unique-nic-name>/nvram", so nothing
> changed.
>
> Auto-generated names are not needed, just use paths. In the end a device
> name is a path as well, one with only a single element.

The auto-generated name right now would be:

"<unique-nic-name>::nvram"

You are right that that is also a path.  And I now think you are right that we 
could just s|::|/|g and declare that names are paths.

The only problem with this is it introduces a canonical path format.  That may 
or may not be a bad thing..

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Avi Kivity]

On 09/15/2011 04:26 PM, Anthony Liguori wrote:
>
> I think this model is the closest to what we have today and is the 
> most obvious.  For something like ne2k, I would expect:
>
> class NE2000 : public Device
> {
>   // ne2k public functions
> };
>
> class PCI_NE2000 : public PciDevice
> {
>   // implement PCI functions by calling ne2k public functions
>   NE2000 ne2k;
> };
>
> class ISA_NE2000 : public IsaDevice
> {
>   // implement ISA functions by calling ne2k public functions
>   NE2000 ne2k;
> };

Also, NE2000 methods have to call ISA_NE2000 and PCI_NE2000 methods, 
yes?  That's going to be more difficult.  Not impossible, just hard.  
It's probably going to involve device specific code that models what 
type of glue was used for that particular device/bus combo.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 12:51:23PM -0500, Anthony Liguori wrote:
> On 09/15/2011 11:59 AM, Gleb Natapov wrote:
> >On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
> >>On 09/15/2011 10:38 AM, Gleb Natapov wrote:
> >>>On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
> >>>>On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> >>>>
> >>>>There is no canonical parent link.  A device may have multiple (more
> >>>>or less equivalent) parents.
> >>>>
> >>>>What should be treated as the "canonical" link depends on what
> >>>>you're trying to do.  In the case of OF, you want to treat the bus
> >>>>as a parent.  If a device happens to sit on multiple buses, I'm not
> >>>>really sure what you do.
> >>>>
> >>>Yes, "canonical" is a link to a bus. Can you give an example of a device
> >>>that sits on multiple buses?
> >>
> >>Not all devices buses that they sit on.
> >>
> >Missing "have"? If device has no bus how do you talk to it? Who carries
> >the signal from a cpu to a device?
> >
> >>A good example is our favorite one to debate--the PIIX3.  Devices
> >PIIX3 is a collection of devices, not a device.
> >
> >>like the UART don't sit on a bus.  They don't have any links at all.
> >In PC UART sits on isa bus. How device can have no links at all? It just
> >glued to a motherboard not touching any wires?
> 
> A bus implies a bidirectional relationship.  IOW, the device has to
> know that it sits on a ISA bus to be an ISA device.
> 
And ISA device with UART on it definitely knows that.

> The UART has no knowledge of the fact that is mapped behind ISA.
> The UART exposes a public interface (through it's pins) that's
> orthogonal to any buses.
> 
The UART itself has no knowledge, yes. But UART does not exists in
vacuum. It is always a part of other device that provides bus logic.
Original PC provided 2 or 4 ISA devices with UART on them. That is how
we need to model them on a PC. You can (or could) easily buy PCI card
with many more additional UARTs. You wouldn't claim that those UARTs are
not on the PCI bus, would you?

> That public interface may be consumed by 1 or more consumers.  Some
> pins may be used by one device while other pins are used by another
> device.
> 
> Some other logic maps the UART to the rest of the system.  My view
> of modelling the PIIX3 that I illustrated in this thread is that the
> PIIX3 encompasses this logic.
> 
That is where we differ. My view that there is no such device as PIIX3
(really what is the functionality of this device?).  PIIX3 is only a
collection of devices. It provides ISA bus logic and it also has 4 ISA
UART cards and each card has the logic to decode access to it.

> Another way to view it, the PIIX3 has direct knowledge of a UART's
> public interface but a UART has no knowledge of the PIIX3's public
> interface.
> 
> >>Instead, the PIIX3 itself bridges the public interface of the UART
> >>via its PC interface.  So it looks something like this:
> >Again you are talking about particular HW implantation, not SW interface
> >that the package implements and we need to emulate the later not the
> >former.
> 
> I'm not.  There is no way for a UART to talk to the PIIX3.  You
Why UART should talk to PIIX3 (what is this PIIX3 anyway)?

> can't query it to figure out where it lives in the device graph.
The way you model it - PIIX3 will have this knowledge. The way I propose
to model it ISA UART card will have it. But really, thinking more about
it there is no much difference. The part that decodes access to UART can
provide its OF path name node. So in you model PIIX3 will have to be
able to provide OF path name node for each device it includes. Not so elegant.

> All it can do is generate an interrupt and hope something is
> listening.  There is no hardware equivalent of
> 'cpu_register_physical_memory'.  Some other piece of hardware has to
> route requests to it in a form that it understands.
Correct. I am not saying UART is an ISA device, I am saying UART is a
part of ISA device. It may be a part of PCI device or something other.
 
> 
> >
> >>
> >>class Serial : public Device
> >>{
> >>    uint8_t read(uint8_t addr);
> >>};
> >>
> >>class PIIX3 : public PciDevice, implements PciBus
> >>{
> >>    Serial *tty[4];
> >>};
> >>
> >Make this isa slots, encapsulate UART into isa device and plug one
> >into another. You just cut corners here. You wouldn't do the same for
> >PCI device, but theoretically you can. You will be able to compose new
> >chipset with config file too!
> 
> But then you need to have:
> 
> class IsaSerial : public IsaDevice
> {
>    Serial uart;
> };
> 
OK.

> All you've done is take the dispatch logic and moved it from PIIX3
> to IsaSerial for the only purpose of creating an artificial
> abstraction (of having a bus that doesn't exist).
It is exists. It decodes access from CPU (in/out) and pass it to UART.
You are saying that since this logic physically resides on PIIX3 chip it
is not ISA bus logic, but PIIX3 logic.

> 
> All devices don't sit on buses.  That's the main design problem of qdev.
> 
Why not? Bus is something that carries signals. Each device needs to
get/set signals to do something useful. One line can be a valid bus.
I thought that qdev problem was that it mandates tree topology.
I can see how device can be on multiple buses (I do not know one),
but not on a bus at all...?

> >>uint32_t PIIX3::pio_access(uint16_t addr, int size)
> >>{
> >>    if (addr == 0x3f8&&  this->tty[0]) {
> >>        return this->tty[0]->read(addr - 0x3f8);
> >>    } else {
> >>        ...
> >>    }
> >>}
> >Wow, hard coded that way? So another chipset implementation will
> >have to duplicate the same exact code?
> 
> Yup.  We're moving to this model anyway with the memory API.  The
> Serial device will export a MemoryRegion where its registers start
> at address 0 and then a higher layer (IsaSerial or PIIX3) will add
> that MemoryRegion to it's MemoryRegion starting at a given offset.
> 
> >>There is no backlink in the Serial device so there's no way of
> >>walking up the graph from the Serial device itself.  You have to
> >>transverse to it to build a path.
> >>
> >That because implementation cut corners (read "incorrect"). Actually
> >ability to walk the graph all the way up is a good test that we didn't
> >cheated anywhere by pretending that devices are connected by magic.
> 
> How do you "walk up the device graph" from a 16650A?  What signals
> are you going to send out of the pins to do that?
16650A is not a device. ISA card it resides on is a device.
 
> 
> If a device can always do self->parent->parent->parent->send_io(foo)
> then the design is fundamentally broken and you will end up with
> devices that do things that they shouldn't do.
> 
Why?

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Peter Maydell]

On 15 September 2011 21:29, Gleb Natapov <gleb@redhat.com> wrote:
> 16650A is not a device. ISA card it resides on is a device.

The 16550A is an encapsulated set of functionality with some
well defined interfaces ("I provide a set of memory mapped
registers", "I have an output gpio line (irq)"), which we
need to be able to compose into other things (lots of models
use a 16550A one way or another, not just the ISA serial card),
connect up (ie connect that irq to an appropriate interrupt
controller, map the registers in system memory or under ISA
or whatever), and configure (eg specify the backend chardev).

I don't think there's any difference at all between that
and (say) the NE2000 PCI model, which also is encapsulated
functionality with well defined interfaces that we need to
be able to compose and connect and configure. We should be
using the same implementation and abstractions for both
cases.

(Note the analogy to hardware: a 16550A chip is a well
specified encapsulated set of functionality with some
electrical, timing, etc restrictions on its use. The
only difference between that and an ISA card is that the
ISA card happens to be physically manufactured so that an
end user can plug it, unplug it and wave it around.)

-- PMM

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 03:29 PM, Gleb Natapov wrote:
> On Thu, Sep 15, 2011 at 12:51:23PM -0500, Anthony Liguori wrote:
>> On 09/15/2011 11:59 AM, Gleb Natapov wrote:
>>> On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
>>>> On 09/15/2011 10:38 AM, Gleb Natapov wrote:
>>>>> On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
>>>>>> On 09/15/2011 09:25 AM, Gleb Natapov wrote:
>>>>>>
>>>>>> There is no canonical parent link.  A device may have multiple (more
>>>>>> or less equivalent) parents.
>>>>>>
>>>>>> What should be treated as the "canonical" link depends on what
>>>>>> you're trying to do.  In the case of OF, you want to treat the bus
>>>>>> as a parent.  If a device happens to sit on multiple buses, I'm not
>>>>>> really sure what you do.
>>>>>>
>>>>> Yes, "canonical" is a link to a bus. Can you give an example of a device
>>>>> that sits on multiple buses?
>>>>
>>>> Not all devices buses that they sit on.
>>>>
>>> Missing "have"? If device has no bus how do you talk to it? Who carries
>>> the signal from a cpu to a device?
>>>
>>>> A good example is our favorite one to debate--the PIIX3.  Devices
>>> PIIX3 is a collection of devices, not a device.
>>>
>>>> like the UART don't sit on a bus.  They don't have any links at all.
>>> In PC UART sits on isa bus. How device can have no links at all? It just
>>> glued to a motherboard not touching any wires?
>>
>> A bus implies a bidirectional relationship.  IOW, the device has to
>> know that it sits on a ISA bus to be an ISA device.
>>
> And ISA device with UART on it definitely knows that.
>
>> The UART has no knowledge of the fact that is mapped behind ISA.
>> The UART exposes a public interface (through it's pins) that's
>> orthogonal to any buses.
>>
> The UART itself has no knowledge, yes. But UART does not exists in
> vacuum. It is always a part of other device that provides bus logic.
> Original PC provided 2 or 4 ISA devices with UART on them. That is how
> we need to model them on a PC. You can (or could) easily buy PCI card
> with many more additional UARTs. You wouldn't claim that those UARTs are
> not on the PCI bus, would you?

Let's consider the following.

Let's say that we emulated a simpler micro controller that exposes a GPIO 
interface.  Something like an Amtel or maybe even a simple ARM chip.

IRL, you would wire the UART pins directly to the GPIO pins and call it a day. 
There is no bus and there is no intermediate layer.

In an everything has a bus world, how does something like this get modelled?

>> How do you "walk up the device graph" from a 16650A?  What signals
>> are you going to send out of the pins to do that?
> 16650A is not a device. ISA card it resides on is a device.
>
>>
>> If a device can always do self->parent->parent->parent->send_io(foo)
>> then the design is fundamentally broken and you will end up with
>> devices that do things that they shouldn't do.
>>
> Why?

Because a serial device has no business calling functions in the i440fx device. 
  It's a layering violation.

Regards,

Anthony Liguori

>
> --
> 			Gleb.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 03:23 PM, Avi Kivity wrote:
> On 09/15/2011 04:26 PM, Anthony Liguori wrote:
>>
>> I think this model is the closest to what we have today and is the most
>> obvious. For something like ne2k, I would expect:
>>
>> class NE2000 : public Device
>> {
>> // ne2k public functions
>> };
>>
>> class PCI_NE2000 : public PciDevice
>> {
>> // implement PCI functions by calling ne2k public functions
>> NE2000 ne2k;
>> };
>>
>> class ISA_NE2000 : public IsaDevice
>> {
>> // implement ISA functions by calling ne2k public functions
>> NE2000 ne2k;
>> };
>
> Also, NE2000 methods have to call ISA_NE2000 and PCI_NE2000 methods, yes?

I don't think so.  The NE2k would export an IRQ and the ISA_NE2K and PCI_NE2k 
would have to route that IRQ.  But I think that's the extent of the 
communication in that direction.

Am I missing something?

Regards,

Anthony Liguori

  That's
> going to be more difficult. Not impossible, just hard. It's probably going to
> involve device specific code that models what type of glue was used for that
> particular device/bus combo.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/15/2011 03:45 PM, Peter Maydell wrote:
> On 15 September 2011 21:29, Gleb Natapov<gleb@redhat.com>  wrote:
>> 16650A is not a device. ISA card it resides on is a device.
>
> The 16550A is an encapsulated set of functionality with some
> well defined interfaces ("I provide a set of memory mapped
> registers", "I have an output gpio line (irq)"), which we
> need to be able to compose into other things (lots of models
> use a 16550A one way or another, not just the ISA serial card),
> connect up (ie connect that irq to an appropriate interrupt
> controller, map the registers in system memory or under ISA
> or whatever), and configure (eg specify the backend chardev).

Absolutely.

My view of a "bus" has always been that it's just a well defined set of 
connectors.  Inheritance becomes a useful way of obtaining that well defined set 
of connectors.

So:

/i440fx/slot[3] = my-e1000

Is really just short-hand for:

/i440fx/slot[3]-bar[0] = my-e1000.bar[0]
/i440fx/slot[3]-bar[1] = my-e1000.bar[1]
...

And more specifically, my thinking was that you could basically omit the 
explicit use of connectors when using a Bus for the sake of convenience.  So 
there's no real need for my-e1000 to actively publish bar[0], bar[1].

I think that later point is where you object most and it's also the area that 
I'm the least convinced myself so I'm pretty open there.

Regards,

Anthony Liguori

> I don't think there's any difference at all between that
> and (say) the NE2000 PCI model, which also is encapsulated
> functionality with well defined interfaces that we need to
> be able to compose and connect and configure. We should be
> using the same implementation and abstractions for both
> cases.
>
> (Note the analogy to hardware: a 16550A chip is a well
> specified encapsulated set of functionality with some
> electrical, timing, etc restrictions on its use. The
> only difference between that and an ISA card is that the
> ISA card happens to be physically manufactured so that an
> end user can plug it, unplug it and wave it around.)
>
> -- PMM
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gerd Hoffmann]

   Hi,

> Also, NE2000 methods have to call ISA_NE2000 and PCI_NE2000 methods,
> yes? That's going to be more difficult. Not impossible, just hard. It's
> probably going to involve device specific code that models what type of
> glue was used for that particular device/bus combo.

I'd expect our new, shiny memory api should help here.  I think the 
difference between pci and isa ne2k is just how the registers are mapped 
and how the IRQ is linked up.  At least the registers are easy to 
handle: ne2k core just creates a mmio memory region, then isa-ne2k and 
pci-ne2k just map the memory region in different ways and any register 
access goes directly to the ne2k core code.

cheers,
   Gerd

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Kevin Wolf]

Am 15.09.2011 16:11, schrieb Anthony Liguori:
> On 09/15/2011 08:43 AM, Jan Kiszka wrote:
>> On 2011-09-15 00:11, Anthony Liguori wrote:
>>> On 09/14/2011 04:15 PM, Jan Kiszka wrote:
>>>> On 2011-09-14 21:42, Anthony Liguori wrote:
>>>>>> Such names can get fairly long I'm afraid...
>>>>>
>>>>> A user should never even see these names.  A user probably will always
>>>>> interact with devices via paths.
>>>>
>>>> Right.
>>>> <scratching head>
>>>> But will those automatic names be used at all then?
>>>
>>> Yes, because QEMU is not going to know anything about path names :-)
>>
>> I bet that's a needless self-restriction. What prevents reusing the
>> introspection services that allow path resolutions on the client side
>> also QEMU internally? It would enable us to skip any traps and pitfalls
>> associated with unique device name construction. From a higher
>> perspective, they are completely redundant.
> 
> I actually agree :-)
> 
> We should probably pick a path format and implement in QMP.  I think that 
> discussion is orthogonal though.
> 
>>> Path names should be a concept that exists entirely in the client.  That
>>> may be HMP or that may be a command line tool (like the proposed qemu
>>> script).
>>>
>>> The only management interface exposed to the client is:
>>>
>>> create_object(type, name)
>>> value = get_object_property(name, property_name)
>>> void set_object_property(name, property_name, value)
>>> props = list_object_properties(name)
>>> names = list_objects()
>>>
>>> So names are very important from a QMP perspective, but not something
>>> users every really see.
>>
>> I don't get the added value of something that looks almost like a path
>> but is still not as readable at it (e.g. when debugging the communication).
> 
> It's two separate namespaces.  The name namespace is controlled by the user and 
> we have to bend over backwards to avoid clashing with it.
> 
> The path namespace is controller by QEMU (more or less).
> 
> The name namespace also maps 1-1 to devices which means names can be used to 
> represent devices.  They absolutely never change as long as the device never 
> changes.
> 
> Paths maps N-1 to devices.  Paths may change but names never change.  I don't 
> think there can ever be a fixed canonical path.
> 
> An example is a NIC with nvram that stores a mac address.  In QOM, the guest 
> could change the mac address, then a user could hot unplug the device, and then 
> hot plug the device into a different PCI slot.  The path is now different but 
> the device name has not change.

Maybe then the device name shouldn't default to something that looks
like a path but rather something like "#foo-1" where "foo" is a device
name and "1" a counter for devices of the same type (I'm reusing Jan's #
notation in order to avoid clashes with user specified names, but that's
a detail). I guess a name like that would actually be relatively
convenient to use from a user interface like HMP.

Kevin

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 05:12 AM, Kevin Wolf wrote:
> Am 15.09.2011 16:11, schrieb Anthony Liguori:
>> An example is a NIC with nvram that stores a mac address.  In QOM, the guest
>> could change the mac address, then a user could hot unplug the device, and then
>> hot plug the device into a different PCI slot.  The path is now different but
>> the device name has not change.
>
> Maybe then the device name shouldn't default to something that looks
> like a path but rather something like "#foo-1" where "foo" is a device
> name and "1" a counter for devices of the same type (I'm reusing Jan's #
> notation in order to avoid clashes with user specified names, but that's
> a detail). I guess a name like that would actually be relatively
> convenient to use from a user interface like HMP.

Yeah, I was thinking about this myself yesterday.  To avoid confusion, we could 
just make the names more or less random.  That would ensure that no one used 
them directly.

I would expect people to use paths in HMP.  A nice advantage of paths is that 
you can autocomplete them in HMP.

Regards,

Anthony Liguori

>
> Kevin
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[John Williams]

On Thu, Sep 15, 2011 at 11:17 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 09/15/2011 01:31 AM, Gleb Natapov wrote:
>>
>> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>>>
>>> All device relationships are identified as named properties.  A QOM
>>> path name
>>> consists of a named device, followed by a series of properties which
>>> may or may
>>> not refer to other devices.  For instance, all of the following are
>>> valid paths:
>>>
>>>  /i440fx/piix3/i8042/aux
>>>  /i440fx/slot[1.0]/i8042/aux
>>>  /i440fx/slot[1.0]/bus/piix3/i8042/aux
>>>
>> Have you looked at device paths generated by get_fw_dev_path() in qdev?
>
> get_fw_dev_path() won't exist in QOM.  The fact that it exists in qdev is a
> problem with qdev.
>
>> This function generates Open Firmware device path.
>
> The function generates *a* OF device path.  OF is not a canonical
> representation of arbitrary hardware.  It's a representation chosen (usually
> by a human) of what information about the hardware is needed by the OS-level
> software.

That need not be the case - with the

 link=<&target>

syntax, device trees can be topologically accurate descriptions - this
is part of our still-unreviewed patchset, the ability to have
arbitrary links between devices, beyond the hierarchical bus topology.

Another counter-example - our device trees are autogenerated out of a
high level system synthesis tool.  One path is a device tree for QEMU
and kernel configuration, the other is to actually create the system
based on a high level design specification.


>
> If you look at what other folks have done with OF integration in QEMU,
> you'll see a recurring theme of two OF trees, one used to create the
> hardware and the other that is actually exposed to the guest.  The reason
> you need two is because guests sometimes expect very specific things that
> you really can't generate programmatically in every circumstance.

Again this is contrary to our experience - the predominant reason we
have differing OF trees is because we routinely encounter machine
models that contain devices that QEMU knows nothing about.  So, we
invalidate them in the device tree before passing it through to the
guest kernel, to avoid the problem of drives trying to probe hardware
that isn't there.

John
-- 
John Williams, PhD, B. Eng, B. IT
PetaLogix - Linux Solutions for a Reconfigurable World
w: www.petalogix.com  p: +61-7-30090663  f: +61-7-30090663

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 09:46 AM, John Williams wrote:
> On Thu, Sep 15, 2011 at 11:17 PM, Anthony Liguori<anthony@codemonkey.ws>  wrote:
>> On 09/15/2011 01:31 AM, Gleb Natapov wrote:
>>>
>>> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>>>>
>>>> All device relationships are identified as named properties.  A QOM
>>>> path name
>>>> consists of a named device, followed by a series of properties which
>>>> may or may
>>>> not refer to other devices.  For instance, all of the following are
>>>> valid paths:
>>>>
>>>>   /i440fx/piix3/i8042/aux
>>>>   /i440fx/slot[1.0]/i8042/aux
>>>>   /i440fx/slot[1.0]/bus/piix3/i8042/aux
>>>>
>>> Have you looked at device paths generated by get_fw_dev_path() in qdev?
>>
>> get_fw_dev_path() won't exist in QOM.  The fact that it exists in qdev is a
>> problem with qdev.
>>
>>> This function generates Open Firmware device path.
>>
>> The function generates *a* OF device path.  OF is not a canonical
>> representation of arbitrary hardware.  It's a representation chosen (usually
>> by a human) of what information about the hardware is needed by the OS-level
>> software.
>
> That need not be the case - with the
>
>   link=<&target>
>
> syntax, device trees can be topologically accurate descriptions - this
> is part of our still-unreviewed patchset,

It's not unreviewed.  Any type of machine configuration needs to be done using 
qdev/qom factory interfaces, not implementing custom logic tied to a config format.

Can you construct OF paths based on link attributes?  What would that look like 
in practice?

> Another counter-example - our device trees are autogenerated out of a
> high level system synthesis tool.  One path is a device tree for QEMU
> and kernel configuration, the other is to actually create the system
> based on a high level design specification.

That's all well and good, but the mechanism that I think is important to have in 
QEMU is a programmatic interface for constructing and manipulating the guest 
devices.  A config file is not a programmatic interface.  You can implement 
config file support in terms of a programmatic interface but implementing the 
later in terms of the former is extremely painful.

Regards,

Anthony Liguori

>
>>
>> If you look at what other folks have done with OF integration in QEMU,
>> you'll see a recurring theme of two OF trees, one used to create the
>> hardware and the other that is actually exposed to the guest.  The reason
>> you need two is because guests sometimes expect very specific things that
>> you really can't generate programmatically in every circumstance.
>
> Again this is contrary to our experience - the predominant reason we
> have differing OF trees is because we routinely encounter machine
> models that contain devices that QEMU knows nothing about.  So, we
> invalidate them in the device tree before passing it through to the
> guest kernel, to avoid the problem of drives trying to probe hardware
> that isn't there.
>
> John

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 09:45:33PM +0100, Peter Maydell wrote:
> On 15 September 2011 21:29, Gleb Natapov <gleb@redhat.com> wrote:
> > 16650A is not a device. ISA card it resides on is a device.
> 
> The 16550A is an encapsulated set of functionality with some
> well defined interfaces ("I provide a set of memory mapped
> registers", "I have an output gpio line (irq)"), which we
> need to be able to compose into other things (lots of models
> use a 16550A one way or another, not just the ISA serial card),
> connect up (ie connect that irq to an appropriate interrupt
> controller, map the registers in system memory or under ISA
> or whatever), and configure (eg specify the backend chardev).
> 
> I don't think there's any difference at all between that
> and (say) the NE2000 PCI model, which also is encapsulated
> functionality with well defined interfaces that we need to
> be able to compose and connect and configure. We should be
> using the same implementation and abstractions for both
> cases.
> 
IDE is another such device (it was ISA later converted to PCI).
As far as I understand your view of UART is the same as mine.
It is not a whole device, but only a part of it.

> (Note the analogy to hardware: a 16550A chip is a well
> specified encapsulated set of functionality with some
> electrical, timing, etc restrictions on its use. The
> only difference between that and an ISA card is that the
> ISA card happens to be physically manufactured so that an
> end user can plug it, unplug it and wave it around.)
> 
> -- PMM

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Thu, Sep 15, 2011 at 03:50:28PM -0500, Anthony Liguori wrote:
> On 09/15/2011 03:29 PM, Gleb Natapov wrote:
> >On Thu, Sep 15, 2011 at 12:51:23PM -0500, Anthony Liguori wrote:
> >>On 09/15/2011 11:59 AM, Gleb Natapov wrote:
> >>>On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
> >>>>On 09/15/2011 10:38 AM, Gleb Natapov wrote:
> >>>>>On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
> >>>>>>On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> >>>>>>
> >>>>>>There is no canonical parent link.  A device may have multiple (more
> >>>>>>or less equivalent) parents.
> >>>>>>
> >>>>>>What should be treated as the "canonical" link depends on what
> >>>>>>you're trying to do.  In the case of OF, you want to treat the bus
> >>>>>>as a parent.  If a device happens to sit on multiple buses, I'm not
> >>>>>>really sure what you do.
> >>>>>>
> >>>>>Yes, "canonical" is a link to a bus. Can you give an example of a device
> >>>>>that sits on multiple buses?
> >>>>
> >>>>Not all devices buses that they sit on.
> >>>>
> >>>Missing "have"? If device has no bus how do you talk to it? Who carries
> >>>the signal from a cpu to a device?
> >>>
> >>>>A good example is our favorite one to debate--the PIIX3.  Devices
> >>>PIIX3 is a collection of devices, not a device.
> >>>
> >>>>like the UART don't sit on a bus.  They don't have any links at all.
> >>>In PC UART sits on isa bus. How device can have no links at all? It just
> >>>glued to a motherboard not touching any wires?
> >>
> >>A bus implies a bidirectional relationship.  IOW, the device has to
> >>know that it sits on a ISA bus to be an ISA device.
> >>
> >And ISA device with UART on it definitely knows that.
> >
> >>The UART has no knowledge of the fact that is mapped behind ISA.
> >>The UART exposes a public interface (through it's pins) that's
> >>orthogonal to any buses.
> >>
> >The UART itself has no knowledge, yes. But UART does not exists in
> >vacuum. It is always a part of other device that provides bus logic.
> >Original PC provided 2 or 4 ISA devices with UART on them. That is how
> >we need to model them on a PC. You can (or could) easily buy PCI card
> >with many more additional UARTs. You wouldn't claim that those UARTs are
> >not on the PCI bus, would you?
> 
> Let's consider the following.
> 
> Let's say that we emulated a simpler micro controller that exposes a
> GPIO interface.  Something like an Amtel or maybe even a simple ARM
> chip.
> 
> IRL, you would wire the UART pins directly to the GPIO pins and call
> it a day. There is no bus and there is no intermediate layer.
> 
I am not familiar enough with UART schematic and timings to tell if it
possible, but lets say it is.

> In an everything has a bus world, how does something like this get modelled?
> 
It is a bus. Ad-Hoc one, but still. GPIO pins connected to one or
several devices in order for CPU to communicate with them comprise a
bus.

Is i2c a bus? It calls itself this way. You can use two gpio pins to
implement it. Does it stops to be a bus only because you used gpio pins
to implement it and not a HW controller? Heck, there is 1-wire bus too.

Wikipedia says: "In computer architecture, a bus is a subsystem that
transfers data between components inside a computer, or between
computers". According to his definition what you described above is a
bus.


> >>How do you "walk up the device graph" from a 16650A?  What signals
> >>are you going to send out of the pins to do that?
> >16650A is not a device. ISA card it resides on is a device.
> >
> >>
> >>If a device can always do self->parent->parent->parent->send_io(foo)
> >>then the design is fundamentally broken and you will end up with
> >>devices that do things that they shouldn't do.
> >>
> >Why?
> 
> Because a serial device has no business calling functions in the
> i440fx device.  It's a layering violation.
> 
Ah, yes. I agree. I misunderstood what you were saying. I thought you
meant that having parent link is fundamentally broken and somehow will
cause devices to do things that they shouldn't do.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Peter Maydell]

On 16 September 2011 17:33, Gleb Natapov <gleb@redhat.com> wrote:
> On Thu, Sep 15, 2011 at 09:45:33PM +0100, Peter Maydell wrote:
>> On 15 September 2011 21:29, Gleb Natapov <gleb@redhat.com> wrote:
>> > 16650A is not a device. ISA card it resides on is a device.
>>
>> The 16550A is an encapsulated set of functionality with some
>> well defined interfaces ("I provide a set of memory mapped
>> registers", "I have an output gpio line (irq)"), which we
>> need to be able to compose into other things (lots of models
>> use a 16550A one way or another, not just the ISA serial card),
>> connect up (ie connect that irq to an appropriate interrupt
>> controller, map the registers in system memory or under ISA
>> or whatever), and configure (eg specify the backend chardev).
>>
>> I don't think there's any difference at all between that
>> and (say) the NE2000 PCI model, which also is encapsulated
>> functionality with well defined interfaces that we need to
>> be able to compose and connect and configure. We should be
>> using the same implementation and abstractions for both
>> cases.
>>
> IDE is another such device (it was ISA later converted to PCI).
> As far as I understand your view of UART is the same as mine.
> It is not a whole device, but only a part of it.

If we have the same view of the UART then one of us is rather
misunderstanding the other (could be me).

I don't care whether you apply the label "device" to the
UART, but I definitely want it to be exactly the same kind
of QEMU "object", in terms of how you implement it and use it,
as a PCI NE2000 model or an ISA serial card or an ARM-Cortex-A8
CPU model or a "vexpress-a9" board model.

As it happens, personally I think "device" is a pretty
reasonable label to use for all these things, since we're going
to use it anyway for the QEMU objects which happen to correspond
to ISA cards or PCI cards or whatever.

-- PMM

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 12:47 PM, Peter Maydell wrote:
> On 16 September 2011 17:33, Gleb Natapov<gleb@redhat.com>  wrote:
>> On Thu, Sep 15, 2011 at 09:45:33PM +0100, Peter Maydell wrote:
>>> On 15 September 2011 21:29, Gleb Natapov<gleb@redhat.com>  wrote:
>>>> 16650A is not a device. ISA card it resides on is a device.
>>>
>>> The 16550A is an encapsulated set of functionality with some
>>> well defined interfaces ("I provide a set of memory mapped
>>> registers", "I have an output gpio line (irq)"), which we
>>> need to be able to compose into other things (lots of models
>>> use a 16550A one way or another, not just the ISA serial card),
>>> connect up (ie connect that irq to an appropriate interrupt
>>> controller, map the registers in system memory or under ISA
>>> or whatever), and configure (eg specify the backend chardev).
>>>
>>> I don't think there's any difference at all between that
>>> and (say) the NE2000 PCI model, which also is encapsulated
>>> functionality with well defined interfaces that we need to
>>> be able to compose and connect and configure. We should be
>>> using the same implementation and abstractions for both
>>> cases.
>>>
>> IDE is another such device (it was ISA later converted to PCI).
>> As far as I understand your view of UART is the same as mine.
>> It is not a whole device, but only a part of it.
>
> If we have the same view of the UART then one of us is rather
> misunderstanding the other (could be me).
>
> I don't care whether you apply the label "device" to the
> UART, but I definitely want it to be exactly the same kind
> of QEMU "object", in terms of how you implement it and use it,
> as a PCI NE2000 model or an ISA serial card or an ARM-Cortex-A8
> CPU model or a "vexpress-a9" board model.

Exactly.  This is what I dislike about qdev today.  The UART (SerialState) is 
not a DeviceState.  That's a major problem for me.

There should be no different between IsaSerialState and SerialState in terms of 
what they are and how you interact with them.  Whether IsaSerialState should 
even exist is a separate discussion that's really just a minor detail.

Regards,

Anthony Liguori

>
> As it happens, personally I think "device" is a pretty
> reasonable label to use for all these things, since we're going
> to use it anyway for the QEMU objects which happen to correspond
> to ISA cards or PCI cards or whatever.
>
> -- PMM
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Fri, Sep 16, 2011 at 06:47:56PM +0100, Peter Maydell wrote:
> On 16 September 2011 17:33, Gleb Natapov <gleb@redhat.com> wrote:
> > On Thu, Sep 15, 2011 at 09:45:33PM +0100, Peter Maydell wrote:
> >> On 15 September 2011 21:29, Gleb Natapov <gleb@redhat.com> wrote:
> >> > 16650A is not a device. ISA card it resides on is a device.
> >>
> >> The 16550A is an encapsulated set of functionality with some
> >> well defined interfaces ("I provide a set of memory mapped
> >> registers", "I have an output gpio line (irq)"), which we
> >> need to be able to compose into other things (lots of models
> >> use a 16550A one way or another, not just the ISA serial card),
> >> connect up (ie connect that irq to an appropriate interrupt
> >> controller, map the registers in system memory or under ISA
> >> or whatever), and configure (eg specify the backend chardev).
> >>
> >> I don't think there's any difference at all between that
> >> and (say) the NE2000 PCI model, which also is encapsulated
> >> functionality with well defined interfaces that we need to
> >> be able to compose and connect and configure. We should be
> >> using the same implementation and abstractions for both
> >> cases.
> >>
> > IDE is another such device (it was ISA later converted to PCI).
> > As far as I understand your view of UART is the same as mine.
> > It is not a whole device, but only a part of it.
> 
> If we have the same view of the UART then one of us is rather
> misunderstanding the other (could be me).
> 
May be.

> I don't care whether you apply the label "device" to the
> UART, but I definitely want it to be exactly the same kind
> of QEMU "object", in terms of how you implement it and use it,
> as a PCI NE2000 model or an ISA serial card or an ARM-Cortex-A8
> CPU model or a "vexpress-a9" board model.
> 
> As it happens, personally I think "device" is a pretty
> reasonable label to use for all these things, since we're going
> to use it anyway for the QEMU objects which happen to correspond
> to ISA cards or PCI cards or whatever.
> 
I am not arguing with that. The argument is about what devices hierarchy
in QMEU should be, not what to call "device" and what not to.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Fri, Sep 16, 2011 at 01:08:27PM -0500, Anthony Liguori wrote:
> On 09/16/2011 12:47 PM, Peter Maydell wrote:
> >On 16 September 2011 17:33, Gleb Natapov<gleb@redhat.com>  wrote:
> >>On Thu, Sep 15, 2011 at 09:45:33PM +0100, Peter Maydell wrote:
> >>>On 15 September 2011 21:29, Gleb Natapov<gleb@redhat.com>  wrote:
> >>>>16650A is not a device. ISA card it resides on is a device.
> >>>
> >>>The 16550A is an encapsulated set of functionality with some
> >>>well defined interfaces ("I provide a set of memory mapped
> >>>registers", "I have an output gpio line (irq)"), which we
> >>>need to be able to compose into other things (lots of models
> >>>use a 16550A one way or another, not just the ISA serial card),
> >>>connect up (ie connect that irq to an appropriate interrupt
> >>>controller, map the registers in system memory or under ISA
> >>>or whatever), and configure (eg specify the backend chardev).
> >>>
> >>>I don't think there's any difference at all between that
> >>>and (say) the NE2000 PCI model, which also is encapsulated
> >>>functionality with well defined interfaces that we need to
> >>>be able to compose and connect and configure. We should be
> >>>using the same implementation and abstractions for both
> >>>cases.
> >>>
> >>IDE is another such device (it was ISA later converted to PCI).
> >>As far as I understand your view of UART is the same as mine.
> >>It is not a whole device, but only a part of it.
> >
> >If we have the same view of the UART then one of us is rather
> >misunderstanding the other (could be me).
> >
> >I don't care whether you apply the label "device" to the
> >UART, but I definitely want it to be exactly the same kind
> >of QEMU "object", in terms of how you implement it and use it,
> >as a PCI NE2000 model or an ISA serial card or an ARM-Cortex-A8
> >CPU model or a "vexpress-a9" board model.
> 
> Exactly.  This is what I dislike about qdev today.  The UART
> (SerialState) is not a DeviceState.  That's a major problem for me.
> 
> There should be no different between IsaSerialState and SerialState
> in terms of what they are and how you interact with them.  Whether
> IsaSerialState should even exist is a separate discussion that's
> really just a minor detail.
> 
Then we are arguing about minor detail. But according to you this minor
detail will prevent us from walking device tree up to the root, so it is
not so minor for me.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 01:22 PM, Gleb Natapov wrote:
> Then we are arguing about minor detail. But according to you this minor
> detail will prevent us from walking device tree up to the root, so it is
> not so minor for me.

There is no root.  It's not a tree.  The composition tree (which we've been 
talking about using for canonical pathnames) has nothing to do with the buses.

Moreover, the composition tree doesn't have a single root.  It's got multiple 
root nodes (any user created device is a root node of a composition tree).

Regards,

Anthony Liguori

>
> --
> 			Gleb.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Fri, Sep 16, 2011 at 01:42:02PM -0500, Anthony Liguori wrote:
> On 09/16/2011 01:22 PM, Gleb Natapov wrote:
> >Then we are arguing about minor detail. But according to you this minor
> >detail will prevent us from walking device tree up to the root, so it is
> >not so minor for me.
> 
> There is no root.  It's not a tree.  The composition tree (which
There is "virtual root". System bus in qdev speak. You can create one
explicitly like qdev did or you can say that if a device has no parent
it is on a system bus.

> we've been talking about using for canonical pathnames) has nothing
> to do with the buses.
> 

I do not care about canonical pathnames you are talking about too
much. The reason is that they are useless outside of QEMU. The problem
we need to solve is to name a device in such a way that it can be found
without knowing any QEMU implementation details. This is not for internal
QEMU use (for that you can use canonical pathnames you are talking about),
but for communicating device location outside of QEMU. Currently we pass
OF device paths to firmware and this is ABI QEMU expose to a guest, so
QOM needs to preserve it. What I asked is how it can be done with QOM and
you are saying that it can't be done and this is not a very good answer.
ABI requires OF path to be built not for all QEMU devices, but only for
those that support bootindex property, so this may make our task more
simple, although I think the correct solution should be generic. So
lets think about solutions to the very real problem instead of arguing.

> Moreover, the composition tree doesn't have a single root.  It's got
> multiple root nodes (any user created device is a root node of a
> composition tree).
We are interested in a device tree (or graph) as see from a CPU node.

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 02:13 PM, Gleb Natapov wrote:
> On Fri, Sep 16, 2011 at 01:42:02PM -0500, Anthony Liguori wrote:
>> On 09/16/2011 01:22 PM, Gleb Natapov wrote:
>>> Then we are arguing about minor detail. But according to you this minor
>>> detail will prevent us from walking device tree up to the root, so it is
>>> not so minor for me.
>>
>> There is no root.  It's not a tree.  The composition tree (which
> There is "virtual root". System bus in qdev speak. You can create one
> explicitly like qdev did or you can say that if a device has no parent
> it is on a system bus.

Graphs don't have roots.

>> we've been talking about using for canonical pathnames) has nothing
>> to do with the buses.
>>
>
> I do not care about canonical pathnames you are talking about too
> much. The reason is that they are useless outside of QEMU. The problem
> we need to solve is to name a device in such a way that it can be found
> without knowing any QEMU implementation details. This is not for internal
> QEMU use (for that you can use canonical pathnames you are talking about),
> but for communicating device location outside of QEMU. Currently we pass
> OF device paths to firmware and this is ABI QEMU expose to a guest, so
> QOM needs to preserve it. What I asked is how it can be done with QOM and
> you are saying that it can't be done and this is not a very good answer.

No, it's very easy.  Something just has to decide where to start the 
transversal, and then walk the graph starting at that node until they find the 
node.  They need to map each node to whatever the OF representation is that 
makes sense.

A tree is just a degenerate graph so going from QOM to OF is easy.  It just 
requires looking at a subset.

> ABI requires OF path to be built not for all QEMU devices, but only for
> those that support bootindex property, so this may make our task more
> simple, although I think the correct solution should be generic.

To be fair, it's not an ABI that is supported.  We only need to support a single 
BIOS version that we provide.

But that's just splitting hairs.  You can still generate these paths.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Gleb Natapov]

On Fri, Sep 16, 2011 at 02:29:51PM -0500, Anthony Liguori wrote:
> On 09/16/2011 02:13 PM, Gleb Natapov wrote:
> >On Fri, Sep 16, 2011 at 01:42:02PM -0500, Anthony Liguori wrote:
> >>On 09/16/2011 01:22 PM, Gleb Natapov wrote:
> >>>Then we are arguing about minor detail. But according to you this minor
> >>>detail will prevent us from walking device tree up to the root, so it is
> >>>not so minor for me.
> >>
> >>There is no root.  It's not a tree.  The composition tree (which
> >There is "virtual root". System bus in qdev speak. You can create one
> >explicitly like qdev did or you can say that if a device has no parent
> >it is on a system bus.
> 
> Graphs don't have roots.
> 
Lets not pretend that this device graph is absolutely arbitrary.

> >>we've been talking about using for canonical pathnames) has nothing
> >>to do with the buses.
> >>
> >
> >I do not care about canonical pathnames you are talking about too
> >much. The reason is that they are useless outside of QEMU. The problem
> >we need to solve is to name a device in such a way that it can be found
> >without knowing any QEMU implementation details. This is not for internal
> >QEMU use (for that you can use canonical pathnames you are talking about),
> >but for communicating device location outside of QEMU. Currently we pass
> >OF device paths to firmware and this is ABI QEMU expose to a guest, so
> >QOM needs to preserve it. What I asked is how it can be done with QOM and
> >you are saying that it can't be done and this is not a very good answer.
> 
> No, it's very easy.  Something just has to decide where to start the
> transversal, and then walk the graph starting at that node until
> they find the node.  They need to map each node to whatever the OF
> representation is that makes sense.
> 
That is obvious thing to do, but you said not all devices have a link to
a parent node which will make such traversal impossible. Another thing
is that graph needs to contain enough information (nodes) to create
correct path. If a bus is omitted resulting path will not be accurate.

> A tree is just a degenerate graph so going from QOM to OF is easy.
> It just requires looking at a subset.
> 
> >ABI requires OF path to be built not for all QEMU devices, but only for
> >those that support bootindex property, so this may make our task more
> >simple, although I think the correct solution should be generic.
> 
> To be fair, it's not an ABI that is supported.  We only need to
> support a single BIOS version that we provide.
Yes, but how is this help us here? We will still have to provide some
device identification, that is independent from QEMU internals, to
a firmware.  We chose to use OF because it is established standard and
using something different will be NIH. Seabios is not the only QEMU's
firmware and Seabios is used not only with QEMU (and coreboot is/will be
able to pass bootorder to Seabios the same way QEMU does). So inventing
QEMU specific format, that will have to contain all the same information
about device as OF path anyway, is just spreading NIH around.

> 
> But that's just splitting hairs.  You can still generate these paths.
> 
> Regards,
> 
> Anthony Liguori

--
			Gleb.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 03:48 PM, Gleb Natapov wrote:
> On Fri, Sep 16, 2011 at 02:29:51PM -0500, Anthony Liguori wrote:
>> On 09/16/2011 02:13 PM, Gleb Natapov wrote:
>>> On Fri, Sep 16, 2011 at 01:42:02PM -0500, Anthony Liguori wrote:
>>>> On 09/16/2011 01:22 PM, Gleb Natapov wrote:
>>>>> Then we are arguing about minor detail. But according to you this minor
>>>>> detail will prevent us from walking device tree up to the root, so it is
>>>>> not so minor for me.
>>>>
>>>> There is no root.  It's not a tree.  The composition tree (which
>>> There is "virtual root". System bus in qdev speak. You can create one
>>> explicitly like qdev did or you can say that if a device has no parent
>>> it is on a system bus.
>>
>> Graphs don't have roots.
>>
> Lets not pretend that this device graph is absolutely arbitrary.
>
>>>> we've been talking about using for canonical pathnames) has nothing
>>>> to do with the buses.
>>>>
>>>
>>> I do not care about canonical pathnames you are talking about too
>>> much. The reason is that they are useless outside of QEMU. The problem
>>> we need to solve is to name a device in such a way that it can be found
>>> without knowing any QEMU implementation details. This is not for internal
>>> QEMU use (for that you can use canonical pathnames you are talking about),
>>> but for communicating device location outside of QEMU. Currently we pass
>>> OF device paths to firmware and this is ABI QEMU expose to a guest, so
>>> QOM needs to preserve it. What I asked is how it can be done with QOM and
>>> you are saying that it can't be done and this is not a very good answer.
>>
>> No, it's very easy.  Something just has to decide where to start the
>> transversal, and then walk the graph starting at that node until
>> they find the node.  They need to map each node to whatever the OF
>> representation is that makes sense.
>>
> That is obvious thing to do, but you said not all devices have a link to
> a parent node which will make such traversal impossible.

No, it's really as simple as this:

bool build_of_path(Device *node, Device *target, char *path)
{
  if (IS_PCI_BUS(node)) {
    for (int i = 0; i < 32; i++) {
      PciBus *bus = PCI_BUS(node);
      if (build_of_path(bus->slot[i], target, path)) {
         strappend(path, "/pci@%d", i);
         return true;
      }
    }
  } else if (IS_ISA_BUS(node)) {
    IsaBus *bus = ISA_BUS(node);
    for (int i = 0; i < bus->nb_slots; i++) {
       if (build_of_path(bus->slots[i], target, path)) {
          strappend(path, "/isa@" TARGET_FMT_plx, bus->slots[i]->mem.base);
          return true;
        }
     }
  } else if (node == target) {
    return true;
  }

  return false;
}

This uses centralized logic which I think is the right thing to do.  You could 
also have an OpenFirmwareCompatBus interface which implemented this logic within 
each bus and then a bus could choose to implement this interface.  I think 
centralized is better though.

> Another thing
> is that graph needs to contain enough information (nodes) to create
> correct path. If a bus is omitted resulting path will not be accurate.

It's really not that different from what we have today except that you can 
interact better with the devices in C.  Whereas there is no way today to read 
the "iobase" property of the IsaSerial device, there would be an 
isa_serial_get_iobase() in QOM that the property would be automatically 
implemented in terms of.


>> A tree is just a degenerate graph so going from QOM to OF is easy.
>> It just requires looking at a subset.
>>
>>> ABI requires OF path to be built not for all QEMU devices, but only for
>>> those that support bootindex property, so this may make our task more
>>> simple, although I think the correct solution should be generic.
>>
>> To be fair, it's not an ABI that is supported.  We only need to
>> support a single BIOS version that we provide.
> Yes, but how is this help us here? We will still have to provide some
> device identification, that is independent from QEMU internals, to
> a firmware.  We chose to use OF because it is established standard and
> using something different will be NIH. Seabios is not the only QEMU's
> firmware and Seabios is used not only with QEMU (and coreboot is/will be
> able to pass bootorder to Seabios the same way QEMU does). So inventing
> QEMU specific format, that will have to contain all the same information
> about device as OF path anyway, is just spreading NIH around.

No one is suggesting getting rid of OF paths.  You're confusing constructing 
paths with device naming.  They are not the same thing.

Regards,

Anthony Liguori

>
>>
>> But that's just splitting hairs.  You can still generate these paths.
>>
>> Regards,
>>
>> Anthony Liguori
>
> --
> 			Gleb.
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Fri, Sep 16, 2011 at 01:42:02PM -0500, Anthony Liguori wrote:
> On 09/16/2011 01:22 PM, Gleb Natapov wrote:
> >Then we are arguing about minor detail. But according to you this minor
> >detail will prevent us from walking device tree up to the root, so it is
> >not so minor for me.
> 
> There is no root.  It's not a tree.  The composition tree (which
> we've been talking about using for canonical pathnames) has nothing
> to do with the buses.

Yep, I completely agree.

The bus used by a CPU (and how do you decide which CPU cause different
CPU's might have differnt access paths) to accesse devices may just be
one out of multiple other access paths to the device.

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Fri, Sep 16, 2011 at 07:47:57PM +0300, Gleb Natapov wrote:
> On Thu, Sep 15, 2011 at 03:50:28PM -0500, Anthony Liguori wrote:
> > On 09/15/2011 03:29 PM, Gleb Natapov wrote:
> > >On Thu, Sep 15, 2011 at 12:51:23PM -0500, Anthony Liguori wrote:
> > >>On 09/15/2011 11:59 AM, Gleb Natapov wrote:
> > >>>On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
> > >>>>On 09/15/2011 10:38 AM, Gleb Natapov wrote:
> > >>>>>On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
> > >>>>>>On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> > >>>>>>
> > >>>>>>There is no canonical parent link.  A device may have multiple (more
> > >>>>>>or less equivalent) parents.
> > >>>>>>
> > >>>>>>What should be treated as the "canonical" link depends on what
> > >>>>>>you're trying to do.  In the case of OF, you want to treat the bus
> > >>>>>>as a parent.  If a device happens to sit on multiple buses, I'm not
> > >>>>>>really sure what you do.
> > >>>>>>
> > >>>>>Yes, "canonical" is a link to a bus. Can you give an example of a device
> > >>>>>that sits on multiple buses?
> > >>>>
> > >>>>Not all devices buses that they sit on.
> > >>>>
> > >>>Missing "have"? If device has no bus how do you talk to it? Who carries
> > >>>the signal from a cpu to a device?
> > >>>
> > >>>>A good example is our favorite one to debate--the PIIX3.  Devices
> > >>>PIIX3 is a collection of devices, not a device.
> > >>>
> > >>>>like the UART don't sit on a bus.  They don't have any links at all.
> > >>>In PC UART sits on isa bus. How device can have no links at all? It just
> > >>>glued to a motherboard not touching any wires?
> > >>
> > >>A bus implies a bidirectional relationship.  IOW, the device has to
> > >>know that it sits on a ISA bus to be an ISA device.
> > >>
> > >And ISA device with UART on it definitely knows that.

IMO, this discussion is going nowhere, Partly because assummptions are
beeing made about how hardware "works".

Hardware works the way it gets designed, and it can be desinged in pretty
much anyway you want.

When it comes to devives, you can design them in a way so they become very
dependent on a specific bus. But you can also design them in a more genric
way so that they become bus agnostic. Then you just need to connect a bus
adaptor that hooks things up to the particular bus the device needs to be
hooked up to.

Sometimes, the bus adaptor becomes more like like wrapper that is part
of the logic, other times, the bus adaptor is just a passthru unit.

QEMU should allow us to model devices in a a bus agnostic way.

IMO a device shouldnt neceserally need to know what kind of bus it's
connected to. Those kind of details should be decided by the board
instantiation (of course they might be exceptions).

Another thing that I find very wrong is th endianness model we've got
now, where devices specify their endianness. That is complety bogus.
If you've got the RTL for a device it is trivial to switch endianness,
why shouldnt QEMU boards be able to do that? Endianness should be up to
the board, not to the device itself.

Sometimes, it get the impressino we think more about synthesized
devices into phsyical chips than to the actual options that exist
when you've got the logic at hand and  can place things the way you
(read board) desires.

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Fri, Sep 16, 2011 at 11:10:19AM -0500, Anthony Liguori wrote:
> On 09/16/2011 09:46 AM, John Williams wrote:
> >On Thu, Sep 15, 2011 at 11:17 PM, Anthony Liguori<anthony@codemonkey.ws>  wrote:
> >>On 09/15/2011 01:31 AM, Gleb Natapov wrote:
> >>>
> >>>On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
> >>>>
> >>>>All device relationships are identified as named properties.  A QOM
> >>>>path name
> >>>>consists of a named device, followed by a series of properties which
> >>>>may or may
> >>>>not refer to other devices.  For instance, all of the following are
> >>>>valid paths:
> >>>>
> >>>>  /i440fx/piix3/i8042/aux
> >>>>  /i440fx/slot[1.0]/i8042/aux
> >>>>  /i440fx/slot[1.0]/bus/piix3/i8042/aux
> >>>>
> >>>Have you looked at device paths generated by get_fw_dev_path() in qdev?
> >>
> >>get_fw_dev_path() won't exist in QOM.  The fact that it exists in qdev is a
> >>problem with qdev.
> >>
> >>>This function generates Open Firmware device path.
> >>
> >>The function generates *a* OF device path.  OF is not a canonical
> >>representation of arbitrary hardware.  It's a representation chosen (usually
> >>by a human) of what information about the hardware is needed by the OS-level
> >>software.
> >
> >That need not be the case - with the
> >
> >  link=<&target>
> >
> >syntax, device trees can be topologically accurate descriptions - this
> >is part of our still-unreviewed patchset,
> 
> It's not unreviewed.  Any type of machine configuration needs to be
> done using qdev/qom factory interfaces, not implementing custom
> logic tied to a config format.
> 
> Can you construct OF paths based on link attributes?  What would
> that look like in practice?
> 
> >Another counter-example - our device trees are autogenerated out of a
> >high level system synthesis tool.  One path is a device tree for QEMU
> >and kernel configuration, the other is to actually create the system
> >based on a high level design specification.
> 
> That's all well and good, but the mechanism that I think is
> important to have in QEMU is a programmatic interface for
> constructing and manipulating the guest devices.  A config file is
> not a programmatic interface.  You can implement config file support
> in terms of a programmatic interface but implementing the later in
> terms of the former is extremely painful.

I agree, but I also thinik that we have to be a bit pragmatic in the
sense that if the external interfaces won't be available until years
from now, we should take iternmediate steps.

We've all got imaginary interfaces in our minds, but as long as
these are nowhere near reality, IMO we need to review approaches
towards the current model with a more current perspective.

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 08:11 PM, Edgar E. Iglesias wrote:
> On Fri, Sep 16, 2011 at 11:10:19AM -0500, Anthony Liguori wrote:
>> On 09/16/2011 09:46 AM, John Williams wrote:
>>> On Thu, Sep 15, 2011 at 11:17 PM, Anthony Liguori<anthony@codemonkey.ws>   wrote:
>>>> On 09/15/2011 01:31 AM, Gleb Natapov wrote:
>>>>>
>>>>> On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
>>>>>>
>>>>>> All device relationships are identified as named properties.  A QOM
>>>>>> path name
>>>>>> consists of a named device, followed by a series of properties which
>>>>>> may or may
>>>>>> not refer to other devices.  For instance, all of the following are
>>>>>> valid paths:
>>>>>>
>>>>>>   /i440fx/piix3/i8042/aux
>>>>>>   /i440fx/slot[1.0]/i8042/aux
>>>>>>   /i440fx/slot[1.0]/bus/piix3/i8042/aux
>>>>>>
>>>>> Have you looked at device paths generated by get_fw_dev_path() in qdev?
>>>>
>>>> get_fw_dev_path() won't exist in QOM.  The fact that it exists in qdev is a
>>>> problem with qdev.
>>>>
>>>>> This function generates Open Firmware device path.
>>>>
>>>> The function generates *a* OF device path.  OF is not a canonical
>>>> representation of arbitrary hardware.  It's a representation chosen (usually
>>>> by a human) of what information about the hardware is needed by the OS-level
>>>> software.
>>>
>>> That need not be the case - with the
>>>
>>>   link=<&target>
>>>
>>> syntax, device trees can be topologically accurate descriptions - this
>>> is part of our still-unreviewed patchset,
>>
>> It's not unreviewed.  Any type of machine configuration needs to be
>> done using qdev/qom factory interfaces, not implementing custom
>> logic tied to a config format.
>>
>> Can you construct OF paths based on link attributes?  What would
>> that look like in practice?
>>
>>> Another counter-example - our device trees are autogenerated out of a
>>> high level system synthesis tool.  One path is a device tree for QEMU
>>> and kernel configuration, the other is to actually create the system
>>> based on a high level design specification.
>>
>> That's all well and good, but the mechanism that I think is
>> important to have in QEMU is a programmatic interface for
>> constructing and manipulating the guest devices.  A config file is
>> not a programmatic interface.  You can implement config file support
>> in terms of a programmatic interface but implementing the later in
>> terms of the former is extremely painful.
>
> I agree, but I also thinik that we have to be a bit pragmatic

It's not pragmatic to leave something that's mostly broken alone and tack on 
something new that replicates the function to gain a new feature.

It just results in more cruft and makes it harder to ever fix the real problem.

> in the
> sense that if the external interfaces won't be available until years
> from now, we should take iternmediate steps.

These are the intermediate steps:

http://wiki.qemu.org/Features/QOM#TODO

We really aren't that far away from fixing qdev and making all of these problems 
go away.

> We've all got imaginary interfaces in our minds, but as long as
> these are nowhere near reality,

They are very much reality and they aren't imaginary.  Code exists, we just need 
to move in a common direction here and spend some time fixing past mistakes.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 07:48 PM, Edgar E. Iglesias wrote:
> On Fri, Sep 16, 2011 at 07:47:57PM +0300, Gleb Natapov wrote:
>> On Thu, Sep 15, 2011 at 03:50:28PM -0500, Anthony Liguori wrote:
>>> On 09/15/2011 03:29 PM, Gleb Natapov wrote:
>>>> On Thu, Sep 15, 2011 at 12:51:23PM -0500, Anthony Liguori wrote:
>>>>> On 09/15/2011 11:59 AM, Gleb Natapov wrote:
>>>>>> On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
>>>>>>> On 09/15/2011 10:38 AM, Gleb Natapov wrote:
>>>>>>>> On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
>>>>>>>>> On 09/15/2011 09:25 AM, Gleb Natapov wrote:
>>>>>>>>>
>>>>>>>>> There is no canonical parent link.  A device may have multiple (more
>>>>>>>>> or less equivalent) parents.
>>>>>>>>>
>>>>>>>>> What should be treated as the "canonical" link depends on what
>>>>>>>>> you're trying to do.  In the case of OF, you want to treat the bus
>>>>>>>>> as a parent.  If a device happens to sit on multiple buses, I'm not
>>>>>>>>> really sure what you do.
>>>>>>>>>
>>>>>>>> Yes, "canonical" is a link to a bus. Can you give an example of a device
>>>>>>>> that sits on multiple buses?
>>>>>>>
>>>>>>> Not all devices buses that they sit on.
>>>>>>>
>>>>>> Missing "have"? If device has no bus how do you talk to it? Who carries
>>>>>> the signal from a cpu to a device?
>>>>>>
>>>>>>> A good example is our favorite one to debate--the PIIX3.  Devices
>>>>>> PIIX3 is a collection of devices, not a device.
>>>>>>
>>>>>>> like the UART don't sit on a bus.  They don't have any links at all.
>>>>>> In PC UART sits on isa bus. How device can have no links at all? It just
>>>>>> glued to a motherboard not touching any wires?
>>>>>
>>>>> A bus implies a bidirectional relationship.  IOW, the device has to
>>>>> know that it sits on a ISA bus to be an ISA device.
>>>>>
>>>> And ISA device with UART on it definitely knows that.
>
> IMO, this discussion is going nowhere, Partly because assummptions are
> beeing made about how hardware "works".
>
> Hardware works the way it gets designed, and it can be desinged in pretty
> much anyway you want.
>
> When it comes to devives, you can design them in a way so they become very
> dependent on a specific bus. But you can also design them in a more genric
> way so that they become bus agnostic. Then you just need to connect a bus
> adaptor that hooks things up to the particular bus the device needs to be
> hooked up to.
>
> Sometimes, the bus adaptor becomes more like like wrapper that is part
> of the logic, other times, the bus adaptor is just a passthru unit.
>
> QEMU should allow us to model devices in a a bus agnostic way.

And this is the problem to fix in qdev.  We need to kill buses in qdev.  The 
approach really boils down to:

1) Add unique names to devices (this is needed because buses must have names 
because they must be addressable).

2) Eliminate any device that has two bus instances in it.  I'm pretty sure IDE 
is the only example of this.

3) Move all BusState functionality to DeviceState and eliminate buses.  This is 
a compatibility breaker but a critical change.

Along the way, we also need to refactor properties to support getters/setters 
and add composition properties and link properties.  At some point, we need to 
introduce the QOM type system to get the full benefits of all of this but that 
can probably be mechanical once we've completed the above.

I think this is all well within the scope of 1.1 and can be done without a 
tremendous amount of churn in the tree.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 09/16/2011 09:17 PM, Anthony Liguori wrote:
> On 09/16/2011 07:48 PM, Edgar E. Iglesias wrote:
>> QEMU should allow us to model devices in a a bus agnostic way.
>
> And this is the problem to fix in qdev. We need to kill buses in qdev. The
> approach really boils down to:
>
> 1) Add unique names to devices (this is needed because buses must have names
> because they must be addressable).
>
> 2) Eliminate any device that has two bus instances in it. I'm pretty sure IDE is
> the only example of this.
>
> 3) Move all BusState functionality to DeviceState and eliminate buses. This is a
> compatibility breaker but a critical change.
>
> Along the way, we also need to refactor properties to support getters/setters

If someone wants to jump in and help, this is a good spot to start as its pretty 
much orthogonal to the bits I'm working on.  Today we define properties like:

    .props = (Property[]){
       DEFINE_PROP_UINT8("port", PCIESlot, port.port, 0),
       ..
    }.

Property looks like:

struct PropertyInfo {
     const char *name;
     size_t size;
     enum PropertyType type;
     int (*parse)(DeviceState *dev, Property *prop, const char *str);
     int (*print)(DeviceState *dev, Property *prop, char *dest, size_t len);
     void (*free)(DeviceState *dev, Property *prop);
};

struct Property {
     const char   *name;
     PropertyInfo *info;
     int          offset;
     int          bitnr;
     void         *defval;
};

This forces a model where all properties are members of the state structure with 
simple copy semantics.  We want Property to look like this:

struct NewProperty {
     const char *name;
     const char *type;
     void (*set)(DeviceState *dev, const char *prop, Visitor *v,
                 void *opaque, Error **errp);
     void (*get)(DeviceState *dev, const char *prop, Visitor *v,
                 void *opaque, Error **errp);
     void *opaque;
};

Which lets the device model implement arbitrarily complex properties and 
implement behavior both on setting and getting of properties.  That means we can 
have truly synthetic properties.

The trick we can do is redefine DEFINE_PROP as:

-#define DEFINE_PROP(_name, _state, _field, _prop, _type) { \
-        .name      = (_name),                                    \
-        .info      = &(_prop),                                   \
-        .offset    = offsetof(_state, _field)                    \
-            + type_check(_type,typeof_field(_state, _field)),    \
-        }
+#define DEFINE_PROP(_name, _state, _field, _prop, _type) {        \
+        .name      = (_name),                                     \
+        .set       = qdev_prop_set_legacy,                        \
+        .get       = qdev_prop_get_legacy,                        \
+        .opaque    = (void *)&(Property){                         \
+            .name      = (_name),                                 \
+            .info      = &(_prop),                                \
+            .offset    = offsetof(_state, _field)                 \
+                + type_check(_type,typeof_field(_state, _field)), \
+        },                                                        \
+}

And then implement qdev_prop_set_legacy/qdev_prop_get_legacy to simply call the 
old style property parse/print functions.  That means that all of the existing 
qdev properties will be treated as strings as far as the Visitor is concerned 
but since that's how it works today, it's not all that bad.

Once this is in place, a link<> and child<> property type could be added to 
automate support of composition and linking which would let us start 
constructing graphs (solving a number of practical problems).

Regards,

Anthony Liguori

> and add composition properties and link properties. At some point, we need to
> introduce the QOM type system to get the full benefits of all of this but that
> can probably be mechanical once we've completed the above.

>
> I think this is all well within the scope of 1.1 and can be done without a
> tremendous amount of churn in the tree.
>
> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Fri, Sep 16, 2011 at 09:12:19PM -0500, Anthony Liguori wrote:
> On 09/16/2011 08:11 PM, Edgar E. Iglesias wrote:
> >On Fri, Sep 16, 2011 at 11:10:19AM -0500, Anthony Liguori wrote:
> >>On 09/16/2011 09:46 AM, John Williams wrote:
> >>>On Thu, Sep 15, 2011 at 11:17 PM, Anthony Liguori<anthony@codemonkey.ws>   wrote:
> >>>>On 09/15/2011 01:31 AM, Gleb Natapov wrote:
> >>>>>
> >>>>>On Wed, Sep 14, 2011 at 01:04:00PM -0500, Anthony Liguori wrote:
> >>>>>>
> >>>>>>All device relationships are identified as named properties.  A QOM
> >>>>>>path name
> >>>>>>consists of a named device, followed by a series of properties which
> >>>>>>may or may
> >>>>>>not refer to other devices.  For instance, all of the following are
> >>>>>>valid paths:
> >>>>>>
> >>>>>>  /i440fx/piix3/i8042/aux
> >>>>>>  /i440fx/slot[1.0]/i8042/aux
> >>>>>>  /i440fx/slot[1.0]/bus/piix3/i8042/aux
> >>>>>>
> >>>>>Have you looked at device paths generated by get_fw_dev_path() in qdev?
> >>>>
> >>>>get_fw_dev_path() won't exist in QOM.  The fact that it exists in qdev is a
> >>>>problem with qdev.
> >>>>
> >>>>>This function generates Open Firmware device path.
> >>>>
> >>>>The function generates *a* OF device path.  OF is not a canonical
> >>>>representation of arbitrary hardware.  It's a representation chosen (usually
> >>>>by a human) of what information about the hardware is needed by the OS-level
> >>>>software.
> >>>
> >>>That need not be the case - with the
> >>>
> >>>  link=<&target>
> >>>
> >>>syntax, device trees can be topologically accurate descriptions - this
> >>>is part of our still-unreviewed patchset,
> >>
> >>It's not unreviewed.  Any type of machine configuration needs to be
> >>done using qdev/qom factory interfaces, not implementing custom
> >>logic tied to a config format.
> >>
> >>Can you construct OF paths based on link attributes?  What would
> >>that look like in practice?
> >>
> >>>Another counter-example - our device trees are autogenerated out of a
> >>>high level system synthesis tool.  One path is a device tree for QEMU
> >>>and kernel configuration, the other is to actually create the system
> >>>based on a high level design specification.
> >>
> >>That's all well and good, but the mechanism that I think is
> >>important to have in QEMU is a programmatic interface for
> >>constructing and manipulating the guest devices.  A config file is
> >>not a programmatic interface.  You can implement config file support
> >>in terms of a programmatic interface but implementing the later in
> >>terms of the former is extremely painful.
> >
> >I agree, but I also thinik that we have to be a bit pragmatic
> 
> It's not pragmatic to leave something that's mostly broken alone and
> tack on something new that replicates the function to gain a new
> feature.
> 
> It just results in more cruft and makes it harder to ever fix the real problem.

Yes, but that thinking can be applied recurseivly. Forever. There's always
going to be new and better interfaces in the future.


> >in the
> >sense that if the external interfaces won't be available until years
> >from now, we should take iternmediate steps.
> 
> These are the intermediate steps:
> 
> http://wiki.qemu.org/Features/QOM#TODO
> 
> We really aren't that far away from fixing qdev and making all of
> these problems go away.
> 
> >We've all got imaginary interfaces in our minds, but as long as
> >these are nowhere near reality,
> 
> They are very much reality and they aren't imaginary.  Code exists,
> we just need to move in a common direction here and spend some time
> fixing past mistakes.

Yes. Don't missunderstand me, I like the way QOM is heading. I think
it is reasonable to wait for QOM and an RPC interface to come along.
But it also depends on the time it takes. These topics have, and can be
discussed for a long time while we have users waiting for features.

Last time this was discussed, qdev was the answer. Now, QOM is the
answer. In the future QOM2 will be the answer.

Let's give QOM a shot, but if it doesn't happen within reasonable
time, we'll have to consider reviweing qdev approches., IMHO.
I don't wan't to hurry QOM, but I will ping in 6 months or a year
or so.

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Edgar E. Iglesias]

On Fri, Sep 16, 2011 at 09:17:54PM -0500, Anthony Liguori wrote:
> On 09/16/2011 07:48 PM, Edgar E. Iglesias wrote:
> >On Fri, Sep 16, 2011 at 07:47:57PM +0300, Gleb Natapov wrote:
> >>On Thu, Sep 15, 2011 at 03:50:28PM -0500, Anthony Liguori wrote:
> >>>On 09/15/2011 03:29 PM, Gleb Natapov wrote:
> >>>>On Thu, Sep 15, 2011 at 12:51:23PM -0500, Anthony Liguori wrote:
> >>>>>On 09/15/2011 11:59 AM, Gleb Natapov wrote:
> >>>>>>On Thu, Sep 15, 2011 at 11:33:00AM -0500, Anthony Liguori wrote:
> >>>>>>>On 09/15/2011 10:38 AM, Gleb Natapov wrote:
> >>>>>>>>On Thu, Sep 15, 2011 at 10:28:52AM -0500, Anthony Liguori wrote:
> >>>>>>>>>On 09/15/2011 09:25 AM, Gleb Natapov wrote:
> >>>>>>>>>
> >>>>>>>>>There is no canonical parent link.  A device may have multiple (more
> >>>>>>>>>or less equivalent) parents.
> >>>>>>>>>
> >>>>>>>>>What should be treated as the "canonical" link depends on what
> >>>>>>>>>you're trying to do.  In the case of OF, you want to treat the bus
> >>>>>>>>>as a parent.  If a device happens to sit on multiple buses, I'm not
> >>>>>>>>>really sure what you do.
> >>>>>>>>>
> >>>>>>>>Yes, "canonical" is a link to a bus. Can you give an example of a device
> >>>>>>>>that sits on multiple buses?
> >>>>>>>
> >>>>>>>Not all devices buses that they sit on.
> >>>>>>>
> >>>>>>Missing "have"? If device has no bus how do you talk to it? Who carries
> >>>>>>the signal from a cpu to a device?
> >>>>>>
> >>>>>>>A good example is our favorite one to debate--the PIIX3.  Devices
> >>>>>>PIIX3 is a collection of devices, not a device.
> >>>>>>
> >>>>>>>like the UART don't sit on a bus.  They don't have any links at all.
> >>>>>>In PC UART sits on isa bus. How device can have no links at all? It just
> >>>>>>glued to a motherboard not touching any wires?
> >>>>>
> >>>>>A bus implies a bidirectional relationship.  IOW, the device has to
> >>>>>know that it sits on a ISA bus to be an ISA device.
> >>>>>
> >>>>And ISA device with UART on it definitely knows that.
> >
> >IMO, this discussion is going nowhere, Partly because assummptions are
> >beeing made about how hardware "works".
> >
> >Hardware works the way it gets designed, and it can be desinged in pretty
> >much anyway you want.
> >
> >When it comes to devives, you can design them in a way so they become very
> >dependent on a specific bus. But you can also design them in a more genric
> >way so that they become bus agnostic. Then you just need to connect a bus
> >adaptor that hooks things up to the particular bus the device needs to be
> >hooked up to.
> >
> >Sometimes, the bus adaptor becomes more like like wrapper that is part
> >of the logic, other times, the bus adaptor is just a passthru unit.
> >
> >QEMU should allow us to model devices in a a bus agnostic way.
> 
> And this is the problem to fix in qdev.  We need to kill buses in
> qdev.  The approach really boils down to:
> 
> 1) Add unique names to devices (this is needed because buses must
> have names because they must be addressable).
> 
> 2) Eliminate any device that has two bus instances in it.  I'm
> pretty sure IDE is the only example of this.
> 
> 3) Move all BusState functionality to DeviceState and eliminate
> buses.  This is a compatibility breaker but a critical change.
> Anthony Liguori

I agree. Because busses are in the eye of the beholder. For
some devices a clock connection is it's main relationship with
the system. For others, maybe a "back" channel or link is the
only connectino. For the common device maybe the "main" bus
on which CPU's is are connected is the master bus.

Cheers

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Avi Kivity]

On 09/15/2011 11:52 PM, Anthony Liguori wrote:
>> Also, NE2000 methods have to call ISA_NE2000 and PCI_NE2000 methods, 
>> yes?
>
>
> I don't think so.  The NE2k would export an IRQ and the ISA_NE2K and 
> PCI_NE2k would have to route that IRQ.  But I think that's the extent 
> of the communication in that direction.
>
> Am I missing something?

I guess for simple devices it's unlikely that anything further is needed.

The most complicated device with a dual interface I can think of is vga, 
it needs to register memory regions dynamically.  But even that can be 
done by registering a container on startup, and doing the dynamic stuff 
within the container.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Avi Kivity]

On 09/18/2011 10:56 AM, Avi Kivity wrote:
> On 09/15/2011 11:52 PM, Anthony Liguori wrote:
>>> Also, NE2000 methods have to call ISA_NE2000 and PCI_NE2000 methods, 
>>> yes?
>>
>>
>> I don't think so.  The NE2k would export an IRQ and the ISA_NE2K and 
>> PCI_NE2k would have to route that IRQ.  But I think that's the extent 
>> of the communication in that direction.
>>
>> Am I missing something?
>
> I guess for simple devices it's unlikely that anything further is needed.
>
> The most complicated device with a dual interface I can think of is 
> vga, it needs to register memory regions dynamically.  But even that 
> can be done by registering a container on startup, and doing the 
> dynamic stuff within the container.
>

And in fact this exact problem has just hit us with the vga portio_list 
conversion.  And note we can't just use a container, because 
isa_register_portio_list() does more than just register the memory regions.

(problems are in memory/queue, not upstream).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> The full set of devices names and properties used in this example are
> below:
> 
>   Type: I440FX
>   Is-a: Device
>   Implements: PciBus
>   Name: i440fx
>   Properties:
>    piix3: Composition<PIIX3>
>    slot[1.0]: Backlink<PciDevice>
> 
>   Type: PIIX3
>   Isa-a: PciDevice
>   Implements: IsaBus
>   Name: i440fx::piix3
>   Properties:
>    i8042: Composition<I8042>
>    bus: Backlink<PciDevice> (inherited from PciDevice)
>
>   Type: I8042
>   Isa-a: Device
>   Implements: Ps2Controller
>   Name: i440fx::piix3::i8042
>   Properties:
>    aux: Backlink<Ps2Mouse>

I haven't looks at the patches in detail, but your description looks fairly 
dubious in its current form.

Is the type of piix3::bus a typo? I think this should be Backlink<PciBus>.

What is the difference between "Is-a" and "Implements"? It sounds like some 
form of multiple inheritance, but it's unclear to me which should be the 
primary type.  For PCI host bridges like the i440fx we currently have to split 
them into two, one fot the host bus and the other for the PCI bus.  It feels 
like if we design the model properly this should no longer be necessary.

I'm not so sure we should have Is-a at all.  Isn't the point of replacing the 
bus/device hierachy with a set of links that devices don't follow a simple 
tree structure?  It's fairly common for the tree branches to merge back 
together when a single device communicates via multiple busses.

It's somewhat unlear what the purpose of composition is.  Using composition 
for piix4::piiix and piix3::i8042 seems wrong.  In fact piix3 shouldn't have 
any knowledge of i8042 at all, it's just annother random ISA device.

Speaking of which, presumably i8042 should be an IsaDevice, which inherits 
(amongst other things) a bus: Backlink<IsaBus> property.

What happens when my device has multiple parts that need to be referred to by 
other devices?  An interrupt controller is the most obvious example, but 
similar issues occur for any device that has multiple connections of the same 
type.  Am I expected to create a whole bunch of dummy devices and a private 
interface to bounce everything back to the main device, somethig like:

Type: MagicInterruptController
  Is-a: SystemBusDevice
  Implements: MagicInterruptInterface
  Properties:
    in_pin0: Composition<MagicInterruptPin, pin_number=0, controller=this>
    in_pin1: Composition<MagicInterruptPin, pin_number=1, controller=this>
    out_pin: link<IRQ>

Type: MagicInterruptPin
  Implements: IRQ
  Properties:
    pin_number: int
    controller: BackLink<MagicInterruptInterface>

It seems like rather than having a device implement a set of interfaces, it 
might be better to have these be properties.  Call them a Target<>.  The above 
then becomes something like:

Type: MagicInterruptController
  Is-a: SystemBusDevice
  Properties:
    in_pin0: Target<IRQ>
    in_pin1: Target<IRQ>
    out_pin: Link<IRQ>


Where the Target<> properties are objects/methods provided by the device, and 
the Link<> properties are pointed to them as the machine is constructed.

Paul

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/12/2011 10:47 PM, Paul Brook wrote:
>> The full set of devices names and properties used in this example are
>> below:
>>
>>    Type: I440FX
>>    Is-a: Device
>>    Implements: PciBus
>>    Name: i440fx
>>    Properties:
>>     piix3: Composition<PIIX3>
>>     slot[1.0]: Backlink<PciDevice>
>>
>>    Type: PIIX3
>>    Isa-a: PciDevice
>>    Implements: IsaBus
>>    Name: i440fx::piix3
>>    Properties:
>>     i8042: Composition<I8042>
>>     bus: Backlink<PciDevice>  (inherited from PciDevice)
>>
>>    Type: I8042
>>    Isa-a: Device
>>    Implements: Ps2Controller
>>    Name: i440fx::piix3::i8042
>>    Properties:
>>     aux: Backlink<Ps2Mouse>
>
> I haven't looks at the patches in detail, but your description looks fairly
> dubious in its current form.

This is an old thread.  A few things have evolved since then.

>
> Is the type of piix3::bus a typo? I think this should be Backlink<PciBus>.

Yup.  It's now link<PciBus>.

>
> What is the difference between "Is-a" and "Implements"?

It's multiple inheritance but interfaces (implements) cannot carry state which 
avoids an inheritance diamond problem.  This is the inheritance model used by 
Java, C#, and glib.

> It sounds like some
> form of multiple inheritance, but it's unclear to me which should be the
> primary type.  For PCI host bridges like the i440fx we currently have to split
> them into two, one fot the host bus and the other for the PCI bus.  It feels
> like if we design the model properly this should no longer be necessary.

Yes, that's exactly what this solves.  I think we could probably make i440fx 
is-a PCIDevice that implements a PciBus.  It's the same thing we would do with a 
PCI Bridge.

> I'm not so sure we should have Is-a at all.

A point that has been repeatedly brought up is that you can avoid primary 
inheritance and just rely on composition using composed interfaces as your 
communication vehicle.

I think in some cases, that will make sense, and the object model allows that.

> Isn't the point of replacing the
> bus/device hierachy with a set of links that devices don't follow a simple
> tree structure?  It's fairly common for the tree branches to merge back
> together when a single device communicates via multiple busses.

Yes, but don't confuse the inheritance tree with the device graph.

> It's somewhat unlear what the purpose of composition is.  Using composition
> for piix4::piiix and piix3::i8042 seems wrong.  In fact piix3 shouldn't have
> any knowledge of i8042 at all, it's just annother random ISA device.

i8042 is not a random ISA device.  It lives in the piix3 and the piix3 has 
intimate knowledge of it.  When model a20 line propagation, this is actually an 
important detail.

> Speaking of which, presumably i8042 should be an IsaDevice, which inherits
> (amongst other things) a bus: Backlink<IsaBus>  property.
>
> What happens when my device has multiple parts that need to be referred to by
> other devices?  An interrupt controller is the most obvious example, but
> similar issues occur for any device that has multiple connections of the same
> type.  Am I expected to create a whole bunch of dummy devices and a private
> interface to bounce everything back to the main device, somethig like:
>
> Type: MagicInterruptController
>    Is-a: SystemBusDevice
>    Implements: MagicInterruptInterface
>    Properties:
>      in_pin0: Composition<MagicInterruptPin, pin_number=0, controller=this>
>      in_pin1: Composition<MagicInterruptPin, pin_number=1, controller=this>
>      out_pin: link<IRQ>


No, there would be a single 'Pin' class that would be used for irqs.  There is 
no need to subclass it.

>
> Type: MagicInterruptPin
>    Implements: IRQ
>    Properties:
>      pin_number: int
>      controller: BackLink<MagicInterruptInterface>
>
> It seems like rather than having a device implement a set of interfaces, it
> might be better to have these be properties.  Call them a Target<>.  The above
> then becomes something like:
>
> Type: MagicInterruptController
>    Is-a: SystemBusDevice
>    Properties:
>      in_pin0: Target<IRQ>
>      in_pin1: Target<IRQ>
>      out_pin: Link<IRQ>

So this is more like how it works today except s/Link/child/ and s/Target/link/g.

Regards,

Anthony Liguori

>
> Where the Target<>  properties are objects/methods provided by the device, and
> the Link<>  properties are pointed to them as the machine is constructed.
>
> Paul
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> It's multiple inheritance but interfaces (implements) cannot carry state
> which avoids an inheritance diamond problem.

> > It sounds like some
> > form of multiple inheritance, but it's unclear to me which should be the
> > primary type.  For PCI host bridges like the i440fx we currently have to
> > split them into two, one fot the host bus and the other for the PCI bus.
> >  It feels like if we design the model properly this should no longer be
> > necessary.
> 
> Yes, that's exactly what this solves.  I think we could probably make
> i440fx is-a PCIDevice that implements a PciBus.  It's the same thing we
> would do with a PCI Bridge.

Hmm, so how does that work given all of SysbusDevice, PCIDevice and PciBus 
have state?  Making PciBus a separate (composition/shild) object is IMHO not a 
bad thing anyway - I dislike saying that the device and the bus are the same 
thing.  That still leaves both PCIDevice and SysBusDevice state.  The i440fx 
is a fairly simple device, but more complex bridges have additional state that 
needs to be accessible from both the PCI and SysBus interfaces.

Similar examples occur when we have an audio codec with both an i2c command 
interface and an SSI data port.

> > I'm not so sure we should have Is-a at all.
> 
> A point that has been repeatedly brought up is that you can avoid primary
> inheritance and just rely on composition using composed interfaces as your
> communication vehicle.
> 
> I think in some cases, that will make sense, and the object model allows
> that.

I'm not sure choice is a good thing here!  One of the mistakes I made with 
qdev was to insufficiently document how the device model was supposed to fit 
together.  This resulted in several bastardised qdev "conversions" involving 
gratuitous layering violations.  The end result was that these devices are 
effectively useless as qdev objects, and propagate all the problems that qdev 
was designed to fix.

It would be good to at least have guidelines on how and why the different 
approaches should be used.  Especially for those of us who maybe aren't 
convesant with the more theoretical/academic principles/terminology behind OO 
design.

If it's only a good idea in some cases then you can pretty much guarantee a 
large number of developers will try and use it in the cases where it isn't, 
and vice-versa ;-)

> > It's somewhat unlear what the purpose of composition is.  Using
> > composition for piix4::piiix and piix3::i8042 seems wrong.  In fact
> > piix3 shouldn't have any knowledge of i8042 at all, it's just annother
> > random ISA device.
> 
> i8042 is not a random ISA device.  It lives in the piix3 and the piix3 has
> intimate knowledge of it.  When model a20 line propagation, this is
> actually an important detail.

Huh? I see absoutely no relationship between piix3 and i8042.
It happens that on the PC machine they are etched into the same piece of 
silicon and one of the i8042 GPIO pins it connected to the A20 gate input, but 
the devices have no actual knowledge of each other.

If you're wanting to provide a convenient way of instantiating common clumps 
of devices then I think that should be separate.  Likewise defining convenient 
machine specific aliases for users.

> > What happens when my device has multiple parts that need to be referred
> > to by other devices?  An interrupt controller is the most obvious
> > example, but similar issues occur for any device that has multiple
> > connections of the same type.  Am I expected to create a whole bunch of
> > dummy devices and a private interface to bounce everything back to the
> > main device, somethig like:
> > 
> > Type: MagicInterruptController
> > 
> >    Is-a: SystemBusDevice
> >    Implements: MagicInterruptInterface
> >    
> >    Properties:
> >      in_pin0: Composition<MagicInterruptPin, pin_number=0,
> >        controller=this>
> >      in_pin1: Composition<MagicInterruptPin, pin_number=1,
> >        controller=this>
> >      out_pin: link<IRQ>
> 
> No, there would be a single 'Pin' class that would be used for irqs.  There
> is no need to subclass it.
>
> > 
> > Type: MagicInterruptController
> > 
> >    Is-a: SystemBusDevice
> >    
> >    Properties:
> >      in_pin0: Target<IRQ>
> >      in_pin1: Target<IRQ>
> >      out_pin: Link<IRQ>
> 
> So this is more like how it works today except s/Link/child/ and
> s/Target/link/g.

Isn't that the other way around?  The input pin owns the IRQ object, and the 
output pin is the board/machine/user configured reference to that object.

So the link is to an Interface, not a device?  To support multiple instances 
of the same interface we use a closure-like proxy object?
I guess it we assume the number of interface classes is relatively small that 
should work.

Paul

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/13/2011 11:40 AM, Paul Brook wrote:
>> It's multiple inheritance but interfaces (implements) cannot carry state
>> which avoids an inheritance diamond problem.
>
>>> It sounds like some
>>> form of multiple inheritance, but it's unclear to me which should be the
>>> primary type.  For PCI host bridges like the i440fx we currently have to
>>> split them into two, one fot the host bus and the other for the PCI bus.
>>>   It feels like if we design the model properly this should no longer be
>>> necessary.
>>
>> Yes, that's exactly what this solves.  I think we could probably make
>> i440fx is-a PCIDevice that implements a PciBus.  It's the same thing we
>> would do with a PCI Bridge.
>
> Hmm, so how does that work given all of SysbusDevice, PCIDevice and PciBus
> have state?

Depends.  If you look at PCIBus as an example:

struct PCIBus {
     BusState qbus;
     uint8_t devfn_min;

This is a characteristic that isn't common to all PCI busses.

     pci_set_irq_fn set_irq;
     pci_map_irq_fn map_irq;
     pci_hotplug_fn hotplug;

These are essentially overload points and would be more natural if PCIBus was an 
interface.

     DeviceState *hotplug_qdev;

This is a trick to support hotplug that can be refactored more nicely.

     void *irq_opaque;

This is part of the set_irq interface.

     PCIDevice *devices[PCI_SLOT_MAX * PCI_FUNC_MAX];
     PCIDevice *parent_dev;

These are naturally links.

     MemoryRegion *address_space_mem;
     MemoryRegion *address_space_io;

This would naturally be replaced with a pure virtual function overloaded by the 
child.  A virtual function may be even better that simply used 
get_system_mem()/get_system_io().

     QLIST_HEAD(, PCIBus) child; /* this will be replaced by qdev later */
     QLIST_ENTRY(PCIBus) sibling;/* this will be replaced by qdev later */

Relics of the fact that PCI hasn't been full converted to an object model.

     /* The bus IRQ state is the logical OR of the connected devices.
        Keep a count of the number of devices with raised IRQs.  */
     int nirq;
     int *irq_count;

Probably more naturally expressed as an AND gate.  With a stateful pin model, 
this would end up modeled quite nicely.

So at least for PCI, I think PCIBus as an interface works very well.

> Making PciBus a separate (composition/shild) object is IMHO not a
> bad thing anyway - I dislike saying that the device and the bus are the same
> thing.

Yeah, in some cases, this will almost certainly be the right thing to do.

> That still leaves both PCIDevice and SysBusDevice state.  The i440fx
> is a fairly simple device, but more complex bridges have additional state that
> needs to be accessible from both the PCI and SysBus interfaces.

SysBus is so badly abused today I think we would have to reevaluate what to do 
with it once we fix up some of the devices that make use of it.

> Similar examples occur when we have an audio codec with both an i2c command
> interface and an SSI data port.
>
>>> I'm not so sure we should have Is-a at all.
>>
>> A point that has been repeatedly brought up is that you can avoid primary
>> inheritance and just rely on composition using composed interfaces as your
>> communication vehicle.
>>
>> I think in some cases, that will make sense, and the object model allows
>> that.
>
> I'm not sure choice is a good thing here!  One of the mistakes I made with
> qdev was to insufficiently document how the device model was supposed to fit
> together.

FWIW, my next TODO is to write up a document on QOM.

I will say, I learned a lot about the design of qdev from rewriting it :-)

> This resulted in several bastardised qdev "conversions" involving
> gratuitous layering violations.

Yes, I never understood your comments at first but now I do understand where we 
went wrong and am very eager to avoid duplicating that.

> The end result was that these devices are
> effectively useless as qdev objects, and propagate all the problems that qdev
> was designed to fix.
>
> It would be good to at least have guidelines on how and why the different
> approaches should be used.  Especially for those of us who maybe aren't
> convesant with the more theoretical/academic principles/terminology behind OO
> design.

Yeah, that's the bit I need help with.  I'll be putting together some slides for 
the next KVM call in January and will also post them on qemu-devel.  I'm going 
to try very hard to explain what the design points are in reasonable terms.

> If it's only a good idea in some cases then you can pretty much guarantee a
> large number of developers will try and use it in the cases where it isn't,
> and vice-versa ;-)
>
>>> It's somewhat unlear what the purpose of composition is.  Using
>>> composition for piix4::piiix and piix3::i8042 seems wrong.  In fact
>>> piix3 shouldn't have any knowledge of i8042 at all, it's just annother
>>> random ISA device.
>>
>> i8042 is not a random ISA device.  It lives in the piix3 and the piix3 has
>> intimate knowledge of it.  When model a20 line propagation, this is
>> actually an important detail.
>
> Huh? I see absoutely no relationship between piix3 and i8042.
> It happens that on the PC machine they are etched into the same piece of
> silicon and one of the i8042 GPIO pins it connected to the A20 gate input, but
> the devices have no actual knowledge of each other.

Composition == "etching to the same piece of silicon".  Nothing more, nothing less.

A device has no knowledge of the fact that it is a child of another device.

> If you're wanting to provide a convenient way of instantiating common clumps
> of devices then I think that should be separate.  Likewise defining convenient
> machine specific aliases for users.

There is a separate container device that lets you create arbitrary collections 
of devices, but in many cases, we can achieve better code reuse by using 
composition.

Consider the serial device.  In QOM, I've got it modelled as:

MMSerial has-a UART
ISASerial has-a UART

And there are lots of cases where a design is compatible with a UART16650A but 
adds additional logic/features.  In those cases, we would do:

MCFSerial is-a UART

Now you could certainly remove the has-a logic and make it so something had to 
instantiate a UART separately from the ISASerial device and connect things up 
but that's overly complicated IMHO.

>>> What happens when my device has multiple parts that need to be referred
>>> to by other devices?  An interrupt controller is the most obvious
>>> example, but similar issues occur for any device that has multiple
>>> connections of the same type.  Am I expected to create a whole bunch of
>>> dummy devices and a private interface to bounce everything back to the
>>> main device, somethig like:
>>>
>>> Type: MagicInterruptController
>>>
>>>     Is-a: SystemBusDevice
>>>     Implements: MagicInterruptInterface
>>>
>>>     Properties:
>>>       in_pin0: Composition<MagicInterruptPin, pin_number=0,
>>>         controller=this>
>>>       in_pin1: Composition<MagicInterruptPin, pin_number=1,
>>>         controller=this>
>>>       out_pin: link<IRQ>
>>
>> No, there would be a single 'Pin' class that would be used for irqs.  There
>> is no need to subclass it.
>>
>>>
>>> Type: MagicInterruptController
>>>
>>>     Is-a: SystemBusDevice
>>>
>>>     Properties:
>>>       in_pin0: Target<IRQ>
>>>       in_pin1: Target<IRQ>
>>>       out_pin: Link<IRQ>
>>
>> So this is more like how it works today except s/Link/child/ and
>> s/Target/link/g.
>
> Isn't that the other way around?  The input pin owns the IRQ object, and the
> output pin is the board/machine/user configured reference to that object.

It just depends on your view of input vs. output.  Either way works out just as 
well.

>
> So the link is to an Interface, not a device?

Devices are Objects

All Objects have Types.  An interface is just another Type.  Types can inherit 
from other types.

All devices inherit from the DeviceState type.

A link has a Type associated with it.  The Type can be an interface, or it can 
be a stateful base class, or it can be a Type that isn't derived at all from 
Device (like a CharDriverState).

>  To support multiple instances
> of the same interface we use a closure-like proxy object?

If you needed to, but I think most of the time, that's not necessary.

Regards,

Anthony Liguori

> I guess it we assume the number of interface classes is relatively small that
> should work.
>
> Paul
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> > That still leaves both PCIDevice and SysBusDevice state.  The i440fx
> > is a fairly simple device, but more complex bridges have additional state
> > that needs to be accessible from both the PCI and SysBus interfaces.
> 
> SysBus is so badly abused today I think we would have to reevaluate what to
> do with it once we fix up some of the devices that make use of it.

Ok, so i440fx might not need any state on the SysBus side (or whatever the PC 
mutation of SysBus ends up being).  However I'm pretty sure at some of our 
devices will hit this issue.

It also means the cpu-to-PCI bridge being primarily a PCI device.  Not 
necessarily a problem per-se, but certainly seems a strange outcome.
 
> >>> It's somewhat unlear what the purpose of composition is.  Using
> >>> composition for piix4::piiix and piix3::i8042 seems wrong.  In fact
> >>> piix3 shouldn't have any knowledge of i8042 at all, it's just annother
> >>> random ISA device.
> >> 
> >> i8042 is not a random ISA device.  It lives in the piix3 and the piix3
> >> has intimate knowledge of it.  When model a20 line propagation, this is
> >> actually an important detail.
> > 
> > Huh? I see absoutely no relationship between piix3 and i8042.
> > It happens that on the PC machine they are etched into the same piece of
> > silicon and one of the i8042 GPIO pins it connected to the A20 gate
> > input, but the devices have no actual knowledge of each other.
> 
> Composition == "etching to the same piece of silicon".  Nothing more,
> nothing less.

That's a really crappy definition.  Most ARM systems are comprised of 
approximately one piece of silicon.  There may be dozens of variants of a 
particular chip, with the same device models stamped into the silicon slightly 
different ways.

IMO it would be much better to split on logical device boundaries.  Especially 
when the functionality provided (pci to ISA bridge) works perfectly well 
without the child device.

> A device has no knowledge of the fact that it is a child of another device.
>
> > If you're wanting to provide a convenient way of instantiating common
> > clumps of devices then I think that should be separate.  Likewise
> > defining convenient machine specific aliases for users.
> 
> There is a separate container device that lets you create arbitrary
> collections of devices, but in many cases, we can achieve better code
> reuse by using composition.
>
> Consider the serial device.  In QOM, I've got it modelled as:
> 
> MMSerial has-a UART
> ISASerial has-a UART
> 
> And there are lots of cases where a design is compatible with a UART16650A
> but adds additional logic/features.  In those cases, we would do:
> 
> MCFSerial is-a UART

Are you using UART as shorthand for UART16550A? Or is UART a base object that 
implements pure virtual methods for register access and properties for 
CharDevice and IRQ output. UART16550 inherits from and provides the 
implementation? So a UART8250 or UART6551 could act as drop-in replacements 
(replacing has-a UART with links-to-a UART).

> Now you could certainly remove the has-a logic and make it so something had
> to instantiate a UART separately from the ISASerial device and connect
> things up but that's overly complicated IMHO.

Why not ISASerial is-a UART? I guess because both are stateful, so not allowed 
by our inheritance model?
We could presumably do:
  ISASerial is-a UART has-a ISADevice
It seems like a map-isa-io-to-registers device has at least as may reuse 
opportunities as register-based-uart.

How is MCFSerial different?  Are we expecting MCFSerial to poke around in 
private UART16550 rather than just feeding it cooked data?  If so isn't it 
mode likely to end up as:

UARTMCF is-a UART16550 // with knobs on.
MCFSerial is-a ColdfireBusDevice has-a UART

> > So the link is to an Interface, not a device?
> 
> Devices are Objects
> 
> All Objects have Types.  An interface is just another Type.  Types can
> inherit from other types.

Ah, I think I see.  The trick is that this effectively gives multiple 
inheritance as long as no more than one base type has state (i.e. data 
members).
 
> All devices inherit from the DeviceState type.
> 
> A link has a Type associated with it.  The Type can be an interface, or it
> can be a stateful base class, or it can be a Type that isn't derived at
> all from Device (like a CharDriverState).

Ok. I suspect we may have slightly different ideas what constitutes a 
"Device", but I'll leave that for now.
 
> >  To support multiple instances
> > of the same interface we use a closure-like proxy object?
> 
> If you needed to, but I think most of the time, that's not necessary.

I don't see how this can work without a closure object.  We need a central 
device that is capable of recieving signals from many client devices.  Those 
client devices don't know where they are, they just shout down a point-point 
link. I'd say this is a fairly common topology.

If the central device implements that point-point interface directly then it 
has no idea which device is talking to it.  We need to create child objects 
with a port ID property and implementing the p-p interface, then bind each 
client device to one of these child objects.  The child objects can't do 
anything useful on their own, so need to proxy the signal to an interface on 
the main object, adding in their port id.

Paul

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/13/2011 02:36 PM, Paul Brook wrote:
>> Composition == "etching to the same piece of silicon".  Nothing more,
>> nothing less.
>
> That's a really crappy definition.  Most ARM systems are comprised of
> approximately one piece of silicon.  There may be dozens of variants of a
> particular chip, with the same device models stamped into the silicon slightly
> different ways.
>
> IMO it would be much better to split on logical device boundaries.  Especially
> when the functionality provided (pci to ISA bridge) works perfectly well
> without the child device.

What I'm really after is to have a way to do "-device piix3" and get all of the 
devices that would normally live on the piix3.  If it's desirable to split out 
piix3 into piix3-isa-bridge which is part of piix3 and can be used independently 
such that piix3 is just a thin container of piix3-isa-bridge and everything 
behind it, that's fine by me.

We should do whatever makes sense for the machine/target.  For pc, I've seen 
absolutely zero interest in someone creating a pc machine without an i8042 or an 
RTC.  I don't think it's the use-case to optimize for.

>> A device has no knowledge of the fact that it is a child of another device.
>>
>>> If you're wanting to provide a convenient way of instantiating common
>>> clumps of devices then I think that should be separate.  Likewise
>>> defining convenient machine specific aliases for users.
>>
>> There is a separate container device that lets you create arbitrary
>> collections of devices, but in many cases, we can achieve better code
>> reuse by using composition.
>>
>> Consider the serial device.  In QOM, I've got it modelled as:
>>
>> MMSerial has-a UART
>> ISASerial has-a UART
>>
>> And there are lots of cases where a design is compatible with a UART16650A
>> but adds additional logic/features.  In those cases, we would do:
>>
>> MCFSerial is-a UART
>
> Are you using UART as shorthand for UART16550A?

Yes.

> Or is UART a base object that
> implements pure virtual methods for register access and properties for
> CharDevice and IRQ output. UART16550 inherits from and provides the
> implementation? So a UART8250 or UART6551 could act as drop-in replacements
> (replacing has-a UART with links-to-a UART).
>
>> Now you could certainly remove the has-a logic and make it so something had
>> to instantiate a UART separately from the ISASerial device and connect
>> things up but that's overly complicated IMHO.
>
> Why not ISASerial is-a UART?

That's actually how I implemented MMSerial just to show that it's possible.

I prefer has-a because if you had a real life ISA card with a UART16550A on it, 
the ISA card would have an identifiable UART16650A on it.  I think it's unlikely 
that you would have a single chip with an ISA-compatible pin-out that happened 
to be compatible with a UART16550A.

> I guess because both are stateful, so not allowed
> by our inheritance model?

Right, but ISADevice could very easily become an Interface as it has no state today.

> We could presumably do:
>    ISASerial is-a UART has-a ISADevice
> It seems like a map-isa-io-to-registers device has at least as may reuse
> opportunities as register-based-uart.

We could also do that, but we still need to work out how to expose MemoryRegions 
and AddressSpaces in the new device model.

>
> How is MCFSerial different?  Are we expecting MCFSerial to poke around in
> private UART16550 rather than just feeding it cooked data?  If so isn't it
> mode likely to end up as:

Yes, it could go either way really but I would expect that MCFSerial would 
override the ioport_read() and ioport_write functions and implement custom 
functionality that way.

>
> UARTMCF is-a UART16550 // with knobs on.
> MCFSerial is-a ColdfireBusDevice has-a UART
>
>>> So the link is to an Interface, not a device?
>>
>> Devices are Objects
>>
>> All Objects have Types.  An interface is just another Type.  Types can
>> inherit from other types.
>
> Ah, I think I see.  The trick is that this effectively gives multiple
> inheritance as long as no more than one base type has state (i.e. data
> members).

Exactly.

>
>> All devices inherit from the DeviceState type.
>>
>> A link has a Type associated with it.  The Type can be an interface, or it
>> can be a stateful base class, or it can be a Type that isn't derived at
>> all from Device (like a CharDriverState).
>
> Ok. I suspect we may have slightly different ideas what constitutes a
> "Device", but I'll leave that for now.
>
>>>   To support multiple instances
>>> of the same interface we use a closure-like proxy object?
>>
>> If you needed to, but I think most of the time, that's not necessary.
>
> I don't see how this can work without a closure object.  We need a central
> device that is capable of recieving signals from many client devices.  Those
> client devices don't know where they are, they just shout down a point-point
> link. I'd say this is a fairly common topology.
>
> If the central device implements that point-point interface directly then it
> has no idea which device is talking to it.  We need to create child objects
> with a port ID property and implementing the p-p interface, then bind each
> client device to one of these child objects.  The child objects can't do
> anything useful on their own, so need to proxy the signal to an interface on
> the main object, adding in their port id.

If you aren't using inheritance, yes, you need to pass closures to the child 
objects.  I dislike that kind of proxy modeling.

I wonder how common is it though to have this kind of scenario where the bus 
protocol doesn't have any kind of identifier that identifies the target bus.  I 
presume in most cases you're thinking about, there's a single transport that can 
multiplex communications, no?

If there is a bus identifier in the various interface commands, there is no need 
to implement the interface multiple times.  You can just inherit from it once 
and use the bus identifier to determine which bus it's intended for.

Regards,

Anthony Liguori

>
> Paul
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> >> Composition == "etching to the same piece of silicon".  Nothing more,
> >> nothing less.
> > 
> > That's a really crappy definition.  Most ARM systems are comprised of
> > approximately one piece of silicon.  There may be dozens of variants of a
> > particular chip, with the same device models stamped into the silicon
> > slightly different ways.
> > 
> > IMO it would be much better to split on logical device boundaries. 
> > Especially when the functionality provided (pci to ISA bridge) works
> > perfectly well without the child device.
> 
> What I'm really after is to have a way to do "-device piix3" and get all of
> the devices that would normally live on the piix3.  If it's desirable to
> split out piix3 into piix3-isa-bridge which is part of piix3 and can be
> used independently such that piix3 is just a thin container of
> piix3-isa-bridge and everything behind it, that's fine by me.

Yes, that's what I was imagining.

> We should do whatever makes sense for the machine/target.  For pc, I've
> seen absolutely zero interest in someone creating a pc machine without an
> i8042 or an RTC.

For a PC that actually is an interesting config - chipset manufacturers have 
been threatening to do it for years, and I believe a good fraction is already 
emulated via SMM.

For the PIIX3 specifically I'm not sure how many different variants there are 
in terms of number of peripherals, USB ports, etc.  I'm pretty sure there are 
several models, but I don't know whether the defferences are purely 
electrical/packaging.

> >> Now you could certainly remove the has-a logic and make it so something
> >> had to instantiate a UART separately from the ISASerial device and
> >> connect things up but that's overly complicated IMHO.
> > 
> > Why not ISASerial is-a UART?
> 
> That's actually how I implemented MMSerial just to show that it's possible.
> 
> I prefer has-a because if you had a real life ISA card with a UART16550A on
> it, the ISA card would have an identifiable UART16650A on it.  I think
> it's unlikely that you would have a single chip with an ISA-compatible
> pin-out that happened to be compatible with a UART16550A.

Hmm, maybe. OTOH it's exactly the sort of corner that might beworth cutting if 
you're knocking out ISA serial card clones by the million :-)

> > I guess because both are stateful, so not allowed
> > by our inheritance model?
> 
> Right, but ISADevice could very easily become an Interface as it has no
> state today.

Maybe. I'm not entirely convinced that is viable solution going forward.

Do we have a user interface issue here?  If at some later point we discover 
that both bus and device need some state then we'll need to split the device.  
I didn't follow the device naming thread in detail, so maybe this has already 
been covered.

> > How is MCFSerial different?  Are we expecting MCFSerial to poke around in
> > private UART16550 rather than just feeding it cooked data?  If so isn't
> > it
> 
> > mode likely to end up as:
> Yes, it could go either way really but I would expect that MCFSerial would
> override the ioport_read() and ioport_write functions and implement custom
> functionality that way.

The fact that serial.c (which really should be called 16550.c) contains ISA 
specific IO callback routines feels like ISASerial isn't doing its job 
proberly to start with :-)
 
> > I don't see how this can work without a closure object.  We need a
> > central device that is capable of recieving signals from many client
> > devices.  Those client devices don't know where they are, they just
> > shout down a point-point link. I'd say this is a fairly common topology.
> > 
> > If the central device implements that point-point interface directly then
> > it has no idea which device is talking to it.  We need to create child
> > objects with a port ID property and implementing the p-p interface, then
> > bind each client device to one of these child objects.  The child
> > objects can't do anything useful on their own, so need to proxy the
> > signal to an interface on the main object, adding in their port id.
> 
> If you aren't using inheritance, yes, you need to pass closures to the
> child objects.  I dislike that kind of proxy modeling.

How would you solve this using inheritance?

I can see how it works when the device knows its address, but it seems kinda 
lame to tell a device "You have a dedicated communication channel.  But I'm 
lazy and will smush them all together.  Please add this additional token to 
every signal you send".
 
> I wonder how common is it though to have this kind of scenario where the
> bus protocol doesn't have any kind of identifier that identifies the
> target bus.  I presume in most cases you're thinking about, there's a
> single transport that can multiplex communications, no?

Not really.  Either individual transports, or multiplexing that is not visible 
to client devices.

After further thought IRQs are the only concrete example I can come up with.  
Others tend to have one [logical] host device per bus (various forms of 
serial), be host driven (USB, mostly), or deliberately make the link-level 
details software transparent (PCIe).

I suspect SMBus (sun4m) may be a candidate, though I could be wrong.

Paul

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Andreas Färber]

Am 13.12.2011 22:53, schrieb Anthony Liguori:
> ISADevice could very easily become an Interface as it has no
> state today.

Not in upstream, but Hervé and me had been working on that for PReP, as
you may recall.

There seemed to be consensus that some ISA devices can choose which I/O
ports and IRQs to listen on (EEPROM state).

Further, since we disallow hotplug on ISA, enabled/disabled became
another ISADevice-internal state (ugly IMO and that's where we stalled).

The latter might be avoided by using pc87312 Super I/O has-a UART and by
not pretending they are ISADevices on a global ISABus but something
custom, private to Super I/O (which has enabled/disabled state in its
registers), that happen to look-and-feel just like ISA devices?

Andreas

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/13/2011 06:39 PM, Paul Brook wrote:
>>>> Composition == "etching to the same piece of silicon".  Nothing more,
>>>> nothing less.
>>>
>>> That's a really crappy definition.  Most ARM systems are comprised of
>>> approximately one piece of silicon.  There may be dozens of variants of a
>>> particular chip, with the same device models stamped into the silicon
>>> slightly different ways.
>>>
>>> IMO it would be much better to split on logical device boundaries.
>>> Especially when the functionality provided (pci to ISA bridge) works
>>> perfectly well without the child device.
>>
>> What I'm really after is to have a way to do "-device piix3" and get all of
>> the devices that would normally live on the piix3.  If it's desirable to
>> split out piix3 into piix3-isa-bridge which is part of piix3 and can be
>> used independently such that piix3 is just a thin container of
>> piix3-isa-bridge and everything behind it, that's fine by me.
>
> Yes, that's what I was imagining.
>
>> We should do whatever makes sense for the machine/target.  For pc, I've
>> seen absolutely zero interest in someone creating a pc machine without an
>> i8042 or an RTC.
>
> For a PC that actually is an interesting config - chipset manufacturers have
> been threatening to do it for years, and I believe a good fraction is already
> emulated via SMM.
>
> For the PIIX3 specifically I'm not sure how many different variants there are
> in terms of number of peripherals, USB ports, etc.  I'm pretty sure there are
> several models, but I don't know whether the defferences are purely
> electrical/packaging.
>
>>>> Now you could certainly remove the has-a logic and make it so something
>>>> had to instantiate a UART separately from the ISASerial device and
>>>> connect things up but that's overly complicated IMHO.
>>>
>>> Why not ISASerial is-a UART?
>>
>> That's actually how I implemented MMSerial just to show that it's possible.
>>
>> I prefer has-a because if you had a real life ISA card with a UART16550A on
>> it, the ISA card would have an identifiable UART16650A on it.  I think
>> it's unlikely that you would have a single chip with an ISA-compatible
>> pin-out that happened to be compatible with a UART16550A.
>
> Hmm, maybe. OTOH it's exactly the sort of corner that might beworth cutting if
> you're knocking out ISA serial card clones by the million :-)
>
>>> I guess because both are stateful, so not allowed
>>> by our inheritance model?
>>
>> Right, but ISADevice could very easily become an Interface as it has no
>> state today.
>
> Maybe. I'm not entirely convinced that is viable solution going forward.
>
> Do we have a user interface issue here?

I want to separate backwards compatibility from ABI compatibility.  We should 
provide nice high level interfaces that are forever backwards compatible.  But 
when it comes to hooking up IRQs between devices, that interface should just be 
ABI compatible, not necessarily backwards compatible.

To achieve ABI compatibility, we just have to be strict about renaming types if 
they change significantly and introducing new field names instead of changing 
the semantics of existing fields.

> If at some later point we discover
> that both bus and device need some state then we'll need to split the device.
> I didn't follow the device naming thread in detail, so maybe this has already
> been covered.
>
>>> How is MCFSerial different?  Are we expecting MCFSerial to poke around in
>>> private UART16550 rather than just feeding it cooked data?  If so isn't
>>> it
>>
>>> mode likely to end up as:
>> Yes, it could go either way really but I would expect that MCFSerial would
>> override the ioport_read() and ioport_write functions and implement custom
>> functionality that way.
>
> The fact that serial.c (which really should be called 16550.c) contains ISA
> specific IO callback routines feels like ISASerial isn't doing its job
> proberly to start with :-)

It's not really ISA specific callbacks but I need to think through interaction 
with the memory API.

In my qom-next branch, serial.c exposes a serial_ioport_read/serial_ioport_write 
function.  isa-serial.c and mm-serial.c create MemoryRegions based on thin 
wrappers around that callback.

Ideally, serial.c would export a MemoryRegion directly for the registers it 
supported and isa-serial.c and mm-serial.c could bridge that to the appropriate 
address space.

I think the problem here ends up being itl_shift.  I think Avi's trying to avoid 
having a non-flat dispatch space so you can't really create a MemoryRegion that 
dispatches to another MemoryRegion.  We discussed it before and I believe he was 
planning on adding itl_shift as a MemoryRegion mutator.

Avi, did I understand that all correctly?

>>> I don't see how this can work without a closure object.  We need a
>>> central device that is capable of recieving signals from many client
>>> devices.  Those client devices don't know where they are, they just
>>> shout down a point-point link. I'd say this is a fairly common topology.
>>>
>>> If the central device implements that point-point interface directly then
>>> it has no idea which device is talking to it.  We need to create child
>>> objects with a port ID property and implementing the p-p interface, then
>>> bind each client device to one of these child objects.  The child
>>> objects can't do anything useful on their own, so need to proxy the
>>> signal to an interface on the main object, adding in their port id.
>>
>> If you aren't using inheritance, yes, you need to pass closures to the
>> child objects.  I dislike that kind of proxy modeling.
>
> How would you solve this using inheritance?
>
> I can see how it works when the device knows its address, but it seems kinda
> lame to tell a device "You have a dedicated communication channel.  But I'm
> lazy and will smush them all together.  Please add this additional token to
> every signal you send".

Yes, adding a token is how you would have to do it.

>> I wonder how common is it though to have this kind of scenario where the
>> bus protocol doesn't have any kind of identifier that identifies the
>> target bus.  I presume in most cases you're thinking about, there's a
>> single transport that can multiplex communications, no?
>
> Not really.  Either individual transports, or multiplexing that is not visible
> to client devices.
>
> After further thought IRQs are the only concrete example I can come up with.
> Others tend to have one [logical] host device per bus (various forms of
> serial), be host driven (USB, mostly), or deliberately make the link-level
> details software transparent (PCIe).
>
> I suspect SMBus (sun4m) may be a candidate, though I could be wrong.

Yeah, I can't think of any real examples either so I'd prefer to avoid thinking 
too much about it until we run into something that actually requires it :-)

Regards,

Anthony Liguori

> Paul
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Avi Kivity]

On 12/14/2011 03:53 PM, Anthony Liguori wrote:
>> The fact that serial.c (which really should be called 16550.c)
>> contains ISA
>> specific IO callback routines feels like ISASerial isn't doing its job
>> proberly to start with :-)
>
>
> It's not really ISA specific callbacks but I need to think through
> interaction with the memory API.
>
> In my qom-next branch, serial.c exposes a
> serial_ioport_read/serial_ioport_write function.  isa-serial.c and
> mm-serial.c create MemoryRegions based on thin wrappers around that
> callback.
>
> Ideally, serial.c would export a MemoryRegion directly for the
> registers it supported and isa-serial.c and mm-serial.c could bridge
> that to the appropriate address space.
>
> I think the problem here ends up being itl_shift.  I think Avi's
> trying to avoid having a non-flat dispatch space so you can't really
> create a MemoryRegion that dispatches to another MemoryRegion.

In fact I want the ability to create new address spaces.  For example
index/data style interfaces, as found in RTC, IOAPIC, and PCI 0xcf8
style config space, can be hooked to drive an RTC MemoryRegion, an
IOAPIC MemoryRegion, and the PCI config space address space.

>   We discussed it before and I believe he was planning on adding
> itl_shift as a MemoryRegion mutator.

I don't think it makes sense as a mutator, can it_shift change
dynamically?  But as part of setup, yes.

> Avi, did I understand that all correctly?

If you mean that the interface between serial.c and isa-serial.c (or
however they're renamed) should be MemoryRegions exposed by one and
mapped by the other, then yes, I think that's the right interface.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/14/2011 08:01 AM, Avi Kivity wrote:
> On 12/14/2011 03:53 PM, Anthony Liguori wrote:
>>> The fact that serial.c (which really should be called 16550.c)
>>> contains ISA
>>> specific IO callback routines feels like ISASerial isn't doing its job
>>> proberly to start with :-)
>>
>>
>> It's not really ISA specific callbacks but I need to think through
>> interaction with the memory API.
>>
>> In my qom-next branch, serial.c exposes a
>> serial_ioport_read/serial_ioport_write function.  isa-serial.c and
>> mm-serial.c create MemoryRegions based on thin wrappers around that
>> callback.
>>
>> Ideally, serial.c would export a MemoryRegion directly for the
>> registers it supported and isa-serial.c and mm-serial.c could bridge
>> that to the appropriate address space.
>>
>> I think the problem here ends up being itl_shift.  I think Avi's
>> trying to avoid having a non-flat dispatch space so you can't really
>> create a MemoryRegion that dispatches to another MemoryRegion.
>
> In fact I want the ability to create new address spaces.  For example
> index/data style interfaces, as found in RTC, IOAPIC, and PCI 0xcf8
> style config space, can be hooked to drive an RTC MemoryRegion, an
> IOAPIC MemoryRegion, and the PCI config space address space.

I think that might be overgeneralizing a bit.  There tends to be a lot of 
weirdness in how register demultiplexing works.

A UART uses a bit to determine whether address 0/1 reads the divisor or the 
thr/ier register.  It also dispatches reads/writes to different registers.

Even PCI config space is a bit weird.  The address register always needs to have 
the high bit set, otherwise ~0U is returned.  You could do something weird like 
either offset the address space, or have some special logic to handle it, but I 
think this tends to vary so much that you'll end up with a lot of special cases 
instead of having the logic live in the device where it more correctly belongs.

>
>>    We discussed it before and I believe he was planning on adding
>> itl_shift as a MemoryRegion mutator.
>
> I don't think it makes sense as a mutator, can it_shift change
> dynamically?  But as part of setup, yes.
>
>> Avi, did I understand that all correctly?
>
> If you mean that the interface between serial.c and isa-serial.c (or
> however they're renamed) should be MemoryRegions exposed by one and
> mapped by the other, then yes, I think that's the right interface.

So I guess it's just a question of how to make it work.  I actually think the 
most natural way is to have a MemoryRegion not attached to an AddressSpace and 
have the MemoryRegion bounce requests to another MemoryRegion.

I think that violates the basic design of the memory API though.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Avi Kivity]

On 12/14/2011 04:11 PM, Anthony Liguori wrote:
>> In fact I want the ability to create new address spaces.  For example
>> index/data style interfaces, as found in RTC, IOAPIC, and PCI 0xcf8
>> style config space, can be hooked to drive an RTC MemoryRegion, an
>> IOAPIC MemoryRegion, and the PCI config space address space.
>
>
> I think that might be overgeneralizing a bit.  There tends to be a lot
> of weirdness in how register demultiplexing works.
>
> A UART uses a bit to determine whether address 0/1 reads the divisor
> or the thr/ier register.  It also dispatches reads/writes to different
> registers.
>
> Even PCI config space is a bit weird.  The address register always
> needs to have the high bit set, otherwise ~0U is returned.  You could
> do something weird like either offset the address space, or have some
> special logic to handle it, but I think this tends to vary so much
> that you'll end up with a lot of special cases instead of having the
> logic live in the device where it more correctly belongs.

I don't want to hook it directly, instead issue an address_space_read()
or address_space_write() that will go into a separate MemoryRegion
hierarchy.  So the PCI 0xcfc callback,for example, can look at the high
bit, and if set, call address_space_read() on the PCI config address space.

This lets devices expose their config spaces via MemoryRegions, that can
then be directly mapped into the mmio space via mmconfig on newer host
bridges.  If we get a Register class, they can use that too.

>
>>
>>>    We discussed it before and I believe he was planning on adding
>>> itl_shift as a MemoryRegion mutator.
>>
>> I don't think it makes sense as a mutator, can it_shift change
>> dynamically?  But as part of setup, yes.
>>
>>> Avi, did I understand that all correctly?
>>
>> If you mean that the interface between serial.c and isa-serial.c (or
>> however they're renamed) should be MemoryRegions exposed by one and
>> mapped by the other, then yes, I think that's the right interface.
>
> So I guess it's just a question of how to make it work.  I actually
> think the most natural way is to have a MemoryRegion not attached to
> an AddressSpace and have the MemoryRegion bounce requests to another
> MemoryRegion.
>
> I think that violates the basic design of the memory API though.

Well, that's exactly what I had in mind (except that you'd bounce the
I/O to an AddressSpace, which takes care of preparing the dispatch
tables etc. instead of searching dynamically through the hierarchy).

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/14/2011 08:35 AM, Avi Kivity wrote:
> On 12/14/2011 04:11 PM, Anthony Liguori wrote:
>>>>     We discussed it before and I believe he was planning on adding
>>>> itl_shift as a MemoryRegion mutator.
>>>
>>> I don't think it makes sense as a mutator, can it_shift change
>>> dynamically?  But as part of setup, yes.
>>>
>>>> Avi, did I understand that all correctly?
>>>
>>> If you mean that the interface between serial.c and isa-serial.c (or
>>> however they're renamed) should be MemoryRegions exposed by one and
>>> mapped by the other, then yes, I think that's the right interface.
>>
>> So I guess it's just a question of how to make it work.  I actually
>> think the most natural way is to have a MemoryRegion not attached to
>> an AddressSpace and have the MemoryRegion bounce requests to another
>> MemoryRegion.
>>
>> I think that violates the basic design of the memory API though.
>
> Well, that's exactly what I had in mind (except that you'd bounce the
> I/O to an AddressSpace, which takes care of preparing the dispatch
> tables etc. instead of searching dynamically through the hierarchy).

But the I/O table for the AddressSpace would be independent of the system/io 
address space tables, correct?

For these sort of virtual AddressSpaces, do we care about flattening things?

For the serial device, you would:

1) Create a memory region using some form of addressing that makes sense for 
your device.  For serial, it would be a 0-8 single byte addressing.

2) Create a virtual AddressSpace, add the MemoryRegion from (1) to the AddressSpace

3) Export the AddressSpace from your device.  Serial would then expose an 
AddressSpace, not a MemoryRegion

4) MMSerial would create a MemoryRegion that implemented itl_shift by calling 
into the Serial device's AddressSpace

Is this right?

Honestly, it seems a bit complex to me without an obvious benefit.  The use of 
AddressSpace seems to be overkill since we really don't care about creating a 
linear table for the Serial device.

I actually think relaxing the requirement for MemoryRegions to be associated 
with an AddressSpace and having the ability to bounce from one MR to another is 
a bit more straight forward (with perhaps a loss in elegance).

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Avi Kivity]

On 12/14/2011 04:46 PM, Anthony Liguori wrote:
>> Well, that's exactly what I had in mind (except that you'd bounce the
>> I/O to an AddressSpace, which takes care of preparing the dispatch
>> tables etc. instead of searching dynamically through the hierarchy).
>
>
> But the I/O table for the AddressSpace would be independent of the
> system/io address space tables, correct?

Correct.  We'd have N address spaces; mmio would dispatch into the
first, pio into the second, and we'd have manual dispatch for the others.

>
> For these sort of virtual AddressSpaces, do we care about flattening
> things?
>
> For the serial device, you would:
>
> 1) Create a memory region using some form of addressing that makes
> sense for your device.  For serial, it would be a 0-8 single byte
> addressing.
>
> 2) Create a virtual AddressSpace, add the MemoryRegion from (1) to the
> AddressSpace
>
> 3) Export the AddressSpace from your device.  Serial would then expose
> an AddressSpace, not a MemoryRegion
>
> 4) MMSerial would create a MemoryRegion that implemented itl_shift by
> calling into the Serial device's AddressSpace
>
> Is this right?
>
> Honestly, it seems a bit complex to me without an obvious benefit. 
> The use of AddressSpace seems to be overkill since we really don't
> care about creating a linear table for the Serial device.

For serial, yes.  Just use it_shift during initialization.

>
> I actually think relaxing the requirement for MemoryRegions to be
> associated with an AddressSpace and having the ability to bounce from
> one MR to another is a bit more straight forward (with perhaps a loss
> in elegance).

It means duplicating the dispatch path; having an optimized path through
a radix tree and a dynamic path which walks the MemoryRegion tree and
considers ranges and priorities; this reduces maintainability.

Creating an address space and associating it with a MemoryRegion should
be one call, I don't think it will be that much of a burden.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> > Do we have a user interface issue here?
> 
> I want to separate backwards compatibility from ABI compatibility.  We
> should provide nice high level interfaces that are forever backwards
> compatible.  But when it comes to hooking up IRQs between devices, that
> interface should just be ABI compatible, not necessarily backwards
> compatible.
> 
> To achieve ABI compatibility, we just have to be strict about renaming
> types if they change significantly and introducing new field names instead
> of changing the semantics of existing fields.

Relying on type to disambiguate between different links to an object while 
only allowing one of those types to be stateful make using a single object to 
implement logically distinct functionality (e.g. Device v.s. Bus, or different 
types of device interface) is a user visible implementation detail.

In practice we really do want to inherit state (which presumably includes 
properties) from multiple classes.  I'd be amazed if we last many releases 
without breaking machine descriptions and/or the "qemu -device blah" because 
of this.

I haven't worked out the details, maybe we need a "Self" property, plus a 
policy of never having user visible stuff link to an primary device node.
If the primary object happens to implement/inherit from that Interface then it 
sets the property to itself.  Otherwise it creates a stateful bus object 
(maybe using composition).

This allows you to have i440fx implement PciBus directly when it's convenient, 
but the board description always attaches devices to ::i440x::pcibus.

I think I'm starting to see now why Java code is often a twisty mess of 
interfaces and adaptors.

> >>> I don't see how this can work without a closure object.  We need a
> >>> central device that is capable of recieving signals from many client
> >>> devices.  Those client devices don't know where they are, they just
> >>> shout down a point-point link. I'd say this is a fairly common
> >>> topology.
> >>> 
> >>> If the central device implements that point-point interface directly
> >>> then it has no idea which device is talking to it.  We need to create
> >>> child objects with a port ID property and implementing the p-p
> >>> interface, then bind each client device to one of these child objects.
> >>>  The child objects can't do anything useful on their own, so need to
> >>> proxy the signal to an interface on the main object, adding in their
> >>> port id.
> >> 
> >> If you aren't using inheritance, yes, you need to pass closures to the
> >> child objects.  I dislike that kind of proxy modeling.
> > 
> > How would you solve this using inheritance?
> > 
> > I can see how it works when the device knows its address, but it seems
> > kinda lame to tell a device "You have a dedicated communication channel.
> >  But I'm lazy and will smush them all together.  Please add this
> > additional token to every signal you send".
> 
> Yes, adding a token is how you would have to do it.

Ugh. Except it's worse than I thought.  That token has to come from the user.  
Either via some arbitrary property on the client device, which is going to be 
different for every device especially when a device can link to multiple 
interfaces of the same class.  Or we need some mechanism for attaching data to 
a link, rather than just conecting the two interfaces together.  Neither of 
which sound desirable.

There's also the issue that the token itsef is device specific.  Identifying 
physical incarnations of logically independent interfaces is well outside the 
scome of the relevant bus, and varies from device to device. e.g. one 
interrupt controller mugh device to label its pins A-P, or 0-16, or 128-144, 
or "alice"/"bob".

Paul

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/15/2011 12:59 PM, Paul Brook wrote:
>>> Do we have a user interface issue here?
>>
>> I want to separate backwards compatibility from ABI compatibility.  We
>> should provide nice high level interfaces that are forever backwards
>> compatible.  But when it comes to hooking up IRQs between devices, that
>> interface should just be ABI compatible, not necessarily backwards
>> compatible.
>>
>> To achieve ABI compatibility, we just have to be strict about renaming
>> types if they change significantly and introducing new field names instead
>> of changing the semantics of existing fields.
>
> Relying on type to disambiguate between different links to an object while
> only allowing one of those types to be stateful make using a single object to
> implement logically distinct functionality (e.g. Device v.s. Bus, or different
> types of device interface) is a user visible implementation detail.

But there are two distinct classes of user.  One class of user really is thinking:

"I want a KVM virtual machine, with 3 disks and 2 network cards"

They could give a flying hoot whether the i440fx inherits from pcidevice or 
implements pcibus.

We need to provide an obviously distinct UI and API for these users.  The vast 
majority of command line users fall into this category.  And once this user 
learns how do create a guest with 3 disks and 2 network cards, they should never 
have to learn another way of doing it.

There is a second class of "user" that is doing very sophisticated things and 
cares about this information.  But this sophisticated user (i.e. libvirt) wants 
to be able to probe QEMU to understand what features are available because it 
very likely is evolving just as quickly as QEMU is.

For this user, it's more import to provide this introspection interface than it 
is to guarantee that we never add/remove devices and interfaces.  We can be 
flexible with this class of user provided they have clear and obvious ways to 
figure out what we're doing.

> In practice we really do want to inherit state (which presumably includes
> properties) from multiple classes.

I think before we can declare something "in practice", we have to actually 
experience it, in practice :-)

Object oriented design is not an exact science.  There is no Right Way to model 
things because all models are lossy in some way.  We can debate the merits of MI 
verses SI w/interfaces literally for years but there's enough existence proof 
out there that with SI w/interfaces, you can build complex systems.

>I'd be amazed if we last many releases
> without breaking machine descriptions and/or the "qemu -device blah" because
> of this.
>
> I haven't worked out the details, maybe we need a "Self" property, plus a
> policy of never having user visible stuff link to an primary device node.
> If the primary object happens to implement/inherit from that Interface then it
> sets the property to itself.  Otherwise it creates a stateful bus object
> (maybe using composition).

You're advocating only connecting properties to properties, and never an object 
to a property?  I think that's needlessly complicated.  In the vast majority of 
cases, you just case about saying "connect this CharDriverState to this Serial 
device".  We should make it much more complicated than that.

>
> This allows you to have i440fx implement PciBus directly when it's convenient,
> but the board description always attaches devices to ::i440x::pcibus.
>
> I think I'm starting to see now why Java code is often a twisty mess of
> interfaces and adaptors.

Java is a pure OO language.  There is no such thing as a free standing function. 
  As a result, there are numerous things that are very complicated because 
things that you would naturally express as a function end up being expressed as 
an Adaptor class or something like that.

Just about anything taken to it's logical extreme is bad..

>>>>> I don't see how this can work without a closure object.  We need a
>>>>> central device that is capable of recieving signals from many client
>>>>> devices.  Those client devices don't know where they are, they just
>>>>> shout down a point-point link. I'd say this is a fairly common
>>>>> topology.
>>>>>
>>>>> If the central device implements that point-point interface directly
>>>>> then it has no idea which device is talking to it.  We need to create
>>>>> child objects with a port ID property and implementing the p-p
>>>>> interface, then bind each client device to one of these child objects.
>>>>>   The child objects can't do anything useful on their own, so need to
>>>>> proxy the signal to an interface on the main object, adding in their
>>>>> port id.
>>>>
>>>> If you aren't using inheritance, yes, you need to pass closures to the
>>>> child objects.  I dislike that kind of proxy modeling.
>>>
>>> How would you solve this using inheritance?
>>>
>>> I can see how it works when the device knows its address, but it seems
>>> kinda lame to tell a device "You have a dedicated communication channel.
>>>   But I'm lazy and will smush them all together.  Please add this
>>> additional token to every signal you send".
>>
>> Yes, adding a token is how you would have to do it.
>
> Ugh. Except it's worse than I thought.  That token has to come from the user.
> Either via some arbitrary property on the client device, which is going to be
> different for every device especially when a device can link to multiple
> interfaces of the same class.  Or we need some mechanism for attaching data to
> a link, rather than just conecting the two interfaces together.  Neither of
> which sound desirable.

But this entire use-case seems to be synthetic.  Do you have a real case where 
you would want to inherit twice from the same interface?

Regards,

Anthony Liguori

>
> There's also the issue that the token itsef is device specific.  Identifying
> physical incarnations of logically independent interfaces is well outside the
> scome of the relevant bus, and varies from device to device. e.g. one
> interrupt controller mugh device to label its pins A-P, or 0-16, or 128-144,
> or "alice"/"bob".
>
> Paul
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> But there are two distinct classes of user.  One class of user really is
> thinking:
> 
> "I want a KVM virtual machine, with 3 disks and 2 network cards"
> 
> They could give a flying hoot whether the i440fx inherits from pcidevice or
> implements pcibus.
> 
> We need to provide an obviously distinct UI and API for these users.  The
> vast majority of command line users fall into this category.  And once
> this user learns how do create a guest with 3 disks and 2 network cards,
> they should never have to learn another way of doing it.

Ok.

> There is a second class of "user" that is doing very sophisticated things
> and cares about this information.  But this sophisticated user (i.e.
> libvirt) wants to be able to probe QEMU to understand what features
> are probably available because it very likely is evolving just as quickly as
> QEMU is.

I disagree.  Maybe this is because you're coming from the PC/Linux world where 
all hardware is basically the same.

In the embedded world there are many more vastly different machines.  And some 
of those have several similar but different enough to matter variants.  
There's a good chance the guest OS needs exactly the right one.

Take the Stellaris boards as an example.  We currently implement two boards.  
There are well over a hundred variants in this SoC family.  A good proportion 
of these chips will be used in projects that include a custom PCB design.  
There are at least three different reference boards for the LM3S6965 alone.
This is not unusual.

I really don't want to have to teach qemu about hunderds of different SoCs 
connected to thousands of different "motherboards", most of which we'll never 
even know about.  I'd like for users to be able to create their own hardware 
config without having to track qemu implementation details.

I'm fine with having to hack/rebuild qemu to add a new device implementation.  
Having to do that for every trivial rewiring of existing devices is not so 
fun.

> > I haven't worked out the details, maybe we need a "Self" property, plus a
> > policy of never having user visible stuff link to an primary device node.
> > If the primary object happens to implement/inherit from that Interface
> > then it sets the property to itself.  Otherwise it creates a stateful
> > bus object (maybe using composition).
> 
> You're advocating only connecting properties to properties, and never an
> object to a property?  I think that's needlessly complicated.  In the vast
> majority of cases, you just case about saying "connect this
> CharDriverState to this Serial device".  We should make it much more
> complicated than that.

Yes, that's what I'm saying.  I'm finding it hard to believe that more than 
one link being stateful is such an exceptional case.

Am I right in thinking that you're effectively implementing multiple 
inheritance via properties?  If so I guess works around the single-inheritance 
limitation in many cases.

> >>>> If you aren't using inheritance, yes, you need to pass closures to the
> >>>> child objects.  I dislike that kind of proxy modeling.
> >>> 
> >>> How would you solve this using inheritance?
> >>> 
> >>> I can see how it works when the device knows its address, but it seems
> >>> kinda lame to tell a device "You have a dedicated communication
> >>> channel.
> >>> 
> >>>   But I'm lazy and will smush them all together.  Please add this
> >>> 
> >>> additional token to every signal you send".
> >> 
> >> Yes, adding a token is how you would have to do it.
> > 
> > Ugh. Except it's worse than I thought.  That token has to come from the
> > user. Either via some arbitrary property on the client device, which is
> > going to be different for every device especially when a device can link
> > to multiple interfaces of the same class.  Or we need some mechanism for
> > attaching data to a link, rather than just conecting the two interfaces
> > together.  Neither of which sound desirable.
> 
> But this entire use-case seems to be synthetic.  Do you have a real case
> where you would want to inherit twice from the same interface?

A GPIO controller (of which interrupt controllers are a subset).  We want to 
expose and use many single-bit control line interfaces.

I suppose pckbd.c is annother example.  This implements a pair of PS/2 serial 
busses.  Currently we don't model these and use hardcoded callbacks when 
creating the keyboard/mouse devices.  Once modelled properly I'd expect a 
board to be able to replace either with a third [as yet unimplemented] PS/2 
device.  Any of the those devices should also be usable with the single-port 
ARM PL050 controller.

Paul

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Anthony Liguori]

On 12/15/2011 03:28 PM, Paul Brook wrote:
>> There is a second class of "user" that is doing very sophisticated things
>> and cares about this information.  But this sophisticated user (i.e.
>> libvirt) wants to be able to probe QEMU to understand what features
>> are probably available because it very likely is evolving just as quickly as
>> QEMU is.
>
> I disagree.  Maybe this is because you're coming from the PC/Linux world where
> all hardware is basically the same.
>
> In the embedded world there are many more vastly different machines.  And some
> of those have several similar but different enough to matter variants.
> There's a good chance the guest OS needs exactly the right one.
>
> Take the Stellaris boards as an example.  We currently implement two boards.
> There are well over a hundred variants in this SoC family.  A good proportion
> of these chips will be used in projects that include a custom PCB design.
> There are at least three different reference boards for the LM3S6965 alone.
> This is not unusual.
>
> I really don't want to have to teach qemu about hunderds of different SoCs
> connected to thousands of different "motherboards", most of which we'll never
> even know about.  I'd like for users to be able to create their own hardware
> config without having to track qemu implementation details.

Yes, we're on the same page here.  One of my primary objectives in this 
refactoring is to eliminate the notion of a "machine" as anything that is at all 
special.  I want the ability to have a "pc" device that has (via composition) 
all of the normal PC components via composition.

But don't confuse flexibility with compatibility.  If you're hooking up IRQ 
lines to make a custom SOC, then I don't think you're the type of user that 
can't handle the fact that we may change whether the PIIX inherits from or 
implements it's bus in a new version.

I also don't want the user to have to always make the decision about how to hook 
up IRQs for every single device because in a lot of circumstances, there's no point.

A basic premise for me is that simple things should be simple.  It shouldn't 
take a 800 line config file to create a PC.

>> You're advocating only connecting properties to properties, and never an
>> object to a property?  I think that's needlessly complicated.  In the vast
>> majority of cases, you just case about saying "connect this
>> CharDriverState to this Serial device".  We should make it much more
>> complicated than that.
>
> Yes, that's what I'm saying.  I'm finding it hard to believe that more than
> one link being stateful is such an exceptional case.
>
> Am I right in thinking that you're effectively implementing multiple
> inheritance via properties?  If so I guess works around the single-inheritance
> limitation in many cases.

No.  Properties are strictly for composition.  A lot of times people advocate 
using composition instead of inheritance.  In fact, it's covered extensively in 
"Effective C++" if you're interested.

>> But this entire use-case seems to be synthetic.  Do you have a real case
>> where you would want to inherit twice from the same interface?
>
> A GPIO controller (of which interrupt controllers are a subset).  We want to
> expose and use many single-bit control line interfaces.

I don't see it but perhaps it's because I don't have sufficient imagination. 
Can you point me to a concrete example?

I very much want to avoid over engineering things so I would prefer to only 
worry problems we know we need to solve.

Honestly, the details we're discussing right now are minor and it wouldn't be 
that hard to change the way inheritance works if we wanted to support true MI.

> I suppose pckbd.c is annother example.  This implements a pair of PS/2 serial
> busses.

I've dropped the notion of a "bus" in QOM.  They don't exist at all.

The way this gets modeled in QOM is just to have a link<PS2Device> named kbd and 
a link<PS2Device> named aux.

pckbd implements PS2Bus and PS2Bus looks something like:

typedef struct PS2Bus {
     Interface parent;
     void (*set_irq)(PS2Bus *bus, PS2Device *dev, int level);
} PS2Bus;

PS2Device looks like:

typedef struct PS2Device {
     Device parent;

     PS2Bus *bus;
     // ...
} PS2Device;

The device just does:

void myfunc(PS2Keyboard *s)
{
    ps2_bus_set_irq(s->bus, PS2_DEVICE(s), 1);
}

You could call bus "controller", "other-device", "fubar", or whatever.

> Currently we don't model these and use hardcoded callbacks when
> creating the keyboard/mouse devices.

Yes, it sucks, and once my qom-next tree is merged I'll fix this so it works as 
above and you can instantiate a PS2Mouse and set the i8054.aux property to the 
mouse device that you create.

> Once modelled properly I'd expect a
> board to be able to replace either with a third [as yet unimplemented] PS/2
> device.

The i8042 is not as generic as you think, but yes, if you made a PS/2 AUX 
compatible device (perhaps a tablet or something), you could just drop it in.

> Any of the those devices should also be usable with the single-port
> ARM PL050 controller.

Yup, it would all work out very nicely.  PL050 would implement PS2Bus and have 
just a single link<PS2Device>.

Regards,

Anthony Liguori

>
> Paul
>

Re: [Qemu-devel] [RFC] Plan for moving forward with QOM

[Paul Brook]

> I also don't want the user to have to always make the decision about how to
> hook up IRQs for every single device because in a lot of circumstances,
> there's no point.

How else are we going to figure out how the IRQ lines are wired up?
 
> A basic premise for me is that simple things should be simple.  It
> shouldn't take a 800 line config file to create a PC.

IMO the PC is a bad example.  There's basically only one way things are ever 
wired up.  For other targets there is no standard reference design, and the 
hardware configuration is entirely a the whim of the vendor.

> >> But this entire use-case seems to be synthetic.  Do you have a real case
> >> where you would want to inherit twice from the same interface?
> > 
> > A GPIO controller (of which interrupt controllers are a subset).  We want
> > to expose and use many single-bit control line interfaces.
> 
> I don't see it but perhaps it's because I don't have sufficient
> imagination. Can you point me to a concrete example?

Pick pretty much an of them.  pl190 and pl061 are fairly standard examples.  
The former has 32 input pins to which other devices can link, and two output 
pins that can be linked to other output pins.  The latter has 8 inputs and 9 
outputs. The distinction between IRQ lines and GPIO output pins is a qdev wart 
that we should not repeat.  Both should be identified by name.

The contraints here are that each output pin will be linked to at most one 
input pin, and vice-versa.  Any output pin can by linked to any input pin.

> > I suppose pckbd.c is annother example.  This implements a pair of PS/2
> > serial busses.
> 
> I've dropped the notion of a "bus" in QOM.  They don't exist at all.
>
> The way this gets modeled in QOM is just to have a link<PS2Device> named
> kbd and a link<PS2Device> named aux.

This much I understand.

> pckbd implements PS2Bus and PS2Bus looks something like:
> 
> typedef struct PS2Bus {
>      Interface parent;
>      void (*set_irq)(PS2Bus *bus, PS2Device *dev, int level);
> } PS2Bus;

This is where I get a bit sketchy.  Is set_irq a pure virtual function that 
pckbk is expected to override?  And pckbd has to figure out which link this is 
based on the otherwise unnecessary dev argument?

> PS2Device looks like:
> 
> typedef struct PS2Device {
>      Device parent;
> 
>      PS2Bus *bus;
>      // ...
> } PS2Device;
> 
> The device just does:
> 
> void myfunc(PS2Keyboard *s)
> {
>     ps2_bus_set_irq(s->bus, PS2_DEVICE(s), 1);
> }

How does s->bus get set?

I guess you're expecting pckbd to do something like:

static void i8049_set_irq(PS2Bus *bus, PS2Device *dev, int level)
{
  Device *me = dynamic_cast<Device> bus;
  if (dev == get_dev_property(me, "kbd")) {
    kbd_update_kbd_irq(me, level);
  } else if (dev == get_dev_property(me, "aux")) {
    kbd_update_aux_irq(me, level);
  } else {
    abort();
  }
}

I guess that'll work, as long as both keyboard and mouse aren't implemented by 
the same device (which they might be for IRQ lines).

Paul