[Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Paul Brook]

CVSROOT:	/sources/qemu
Module name:	qemu
Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41

Modified files:
	.              : Makefile.target vl.h 
	hw             : acpi.c adlib.c apb_pci.c arm_gic.c arm_pic.c 
	                 arm_pic.h arm_timer.c cs4231.c cuda.c 
	                 eepro100.c es1370.c fdc.c grackle_pci.c 
	                 gt64xxx.c heathrow_pic.c i8254.c i8259.c ide.c 
	                 integratorcp.c lsi53c895a.c m48t59.c m48t59.h 
	                 mc146818rtc.c mips_int.c mips_malta.c 
	                 mips_r4k.c mips_timer.c ne2000.c openpic.c 
	                 parallel.c pc.c pci.c pckbd.c pcnet.c pcspk.c 
	                 piix_pci.c pl011.c pl050.c pl080.c pl110.c 
	                 pl181.c pl190.c ppc.c ppc_chrp.c ppc_prep.c 
	                 prep_pci.c realview.c rtl8139.c sb16.c serial.c 
	                 shix.c slavio_intctl.c slavio_misc.c 
	                 slavio_serial.c slavio_timer.c smc91c111.c 
	                 sparc32_dma.c sun4m.c sun4u.c unin_pci.c 
	                 usb-ohci.c usb-uhci.c usb.h versatile_pci.c 
	                 versatilepb.c 
	target-mips    : cpu.h 
	target-ppc     : cpu.h 
Added files:
	hw             : irq.c irq.h 

Log message:
	Unify IRQ handling.

CVSWeb URLs:
http://cvs.savannah.gnu.org/viewcvs/qemu/Makefile.target?cvsroot=qemu&r1=1.158&r2=1.159
http://cvs.savannah.gnu.org/viewcvs/qemu/vl.h?cvsroot=qemu&r1=1.207&r2=1.208
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/acpi.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/adlib.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/apb_pci.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_gic.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_pic.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_pic.h?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_timer.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/cs4231.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/cuda.c?cvsroot=qemu&r1=1.11&r2=1.12
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/eepro100.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/es1370.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/fdc.c?cvsroot=qemu&r1=1.21&r2=1.22
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/grackle_pci.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/gt64xxx.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/heathrow_pic.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/i8254.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/i8259.c?cvsroot=qemu&r1=1.21&r2=1.22
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ide.c?cvsroot=qemu&r1=1.57&r2=1.58
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/integratorcp.c?cvsroot=qemu&r1=1.14&r2=1.15
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/lsi53c895a.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/m48t59.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/m48t59.h?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mc146818rtc.c?cvsroot=qemu&r1=1.7&r2=1.8
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_int.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_malta.c?cvsroot=qemu&r1=1.22&r2=1.23
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_r4k.c?cvsroot=qemu&r1=1.41&r2=1.42
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_timer.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ne2000.c?cvsroot=qemu&r1=1.24&r2=1.25
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/openpic.c?cvsroot=qemu&r1=1.11&r2=1.12
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/parallel.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pc.c?cvsroot=qemu&r1=1.73&r2=1.74
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pci.c?cvsroot=qemu&r1=1.36&r2=1.37
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pckbd.c?cvsroot=qemu&r1=1.17&r2=1.18
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pcnet.c?cvsroot=qemu&r1=1.9&r2=1.10
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pcspk.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/piix_pci.c?cvsroot=qemu&r1=1.9&r2=1.10
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl011.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl050.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl080.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl110.c?cvsroot=qemu&r1=1.7&r2=1.8
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl181.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl190.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ppc.c?cvsroot=qemu&r1=1.13&r2=1.14
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ppc_chrp.c?cvsroot=qemu&r1=1.30&r2=1.31
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ppc_prep.c?cvsroot=qemu&r1=1.32&r2=1.33
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/prep_pci.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/realview.c?cvsroot=qemu&r1=1.7&r2=1.8
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/rtl8139.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sb16.c?cvsroot=qemu&r1=1.22&r2=1.23
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/serial.c?cvsroot=qemu&r1=1.15&r2=1.16
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/shix.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_intctl.c?cvsroot=qemu&r1=1.10&r2=1.11
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_misc.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_serial.c?cvsroot=qemu&r1=1.12&r2=1.13
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_timer.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/smc91c111.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sparc32_dma.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sun4m.c?cvsroot=qemu&r1=1.31&r2=1.32
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sun4u.c?cvsroot=qemu&r1=1.15&r2=1.16
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/unin_pci.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/usb-ohci.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/usb-uhci.c?cvsroot=qemu&r1=1.14&r2=1.15
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/usb.h?cvsroot=qemu&r1=1.12&r2=1.13
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/versatile_pci.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/versatilepb.c?cvsroot=qemu&r1=1.12&r2=1.13
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/irq.c?cvsroot=qemu&rev=1.1
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/irq.h?cvsroot=qemu&rev=1.1
http://cvs.savannah.gnu.org/viewcvs/qemu/target-mips/cpu.h?cvsroot=qemu&r1=1.28&r2=1.29
http://cvs.savannah.gnu.org/viewcvs/qemu/target-ppc/cpu.h?cvsroot=qemu&r1=1.35&r2=1.36

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[J. Mayer]

On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> CVSROOT:	/sources/qemu
> Module name:	qemu
> Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41

The patches in the PowerPC target seem complete nonsense.
Furthermore, this kind of patch that break other guys work would likely
to be discussed and not beeing imposed.

Please revert NOW.

-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Paul Brook]

On Saturday 07 April 2007 19:32, J. Mayer wrote:
> On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> > CVSROOT:	/sources/qemu
> > Module name:	qemu
> > Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41
>
> The patches in the PowerPC target seem complete nonsense.

Can you give specific examples?
The CHRP code looks a bit broken, but no more so than before I started.

> Furthermore, this kind of patch that break other guys work would likely
> to be discussed and not beeing imposed.

It's been mentioned several times on this list (by Fabrice, specifically) that 
this is the way to go.

I applied this now specifically because there are big ARM patches waiting to 
be applied, and wanted to get this cleanup done before they went in.

Paul

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Blue Swirl]

>On Saturday 07 April 2007 19:32, J. Mayer wrote:
> > On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> > > CVSROOT:	/sources/qemu
> > > Module name:	qemu
> > > Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41
> >
> > The patches in the PowerPC target seem complete nonsense.
>
>Can you give specific examples?
>The CHRP code looks a bit broken, but no more so than before I started.
>
> > Furthermore, this kind of patch that break other guys work would likely
> > to be discussed and not beeing imposed.
>
>It's been mentioned several times on this list (by Fabrice, specifically) 
>that
>this is the way to go.
>
>I applied this now specifically because there are big ARM patches waiting 
>to
>be applied, and wanted to get this cleanup done before they went in.

Sparc seems undamaged, devices using IRQs work as before. SMP BogoMIPS gets 
calculated and that needs per-CPU IRQs.

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Paul Brook]

> Sparc seems undamaged, devices using IRQs work as before. SMP BogoMIPS gets
> calculated and that needs per-CPU IRQs.

For the record I tested win2k, knoppix and x86 debian, mips test image, sparc 
debian installer, arm test image, and a preinstalled ARM linux.

I didn't test ppc (other than that it built) because it didn't work to start 
with. I've never managed to get the ppc target to successfully boot anything, 
and there is no test image.

Paul

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[J. Mayer]

On Sat, 2007-04-07 at 20:10 +0100, Paul Brook wrote:
> On Saturday 07 April 2007 19:32, J. Mayer wrote:
> > On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> > > CVSROOT:	/sources/qemu
> > > Module name:	qemu
> > > Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41
> >
> > The patches in the PowerPC target seem complete nonsense.
> 
> Can you give specific examples?

I'm talking about the CPU code.
There is NO notion of external IRQ allocation in the PowerPC
specification.
IRQ  are completely out of the scope of the CPU emulation so the table
of 32 void *IRQ pointers is the CPU structure is a complete nonsense.
Where do you see in the PowerPC specification that those CPU have any
notion of how the EXTERNAL IRQ controler works ? Where do you see that a
machine with a PowerPC cannot manage more than 32 IRQ ? Where do you see
ANY NOTION OF EXTERNAL IRQ MANAGEMENT in the PowerPC specification ?
EXTERNAL IRQ MANAGEMENT IS NO WAY RELATED WITH CPU ! It's private to
each IRQ controller. 
Saying anything else is just completely ignoring how real hardware
works.
SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
HAS TO BE REVERTED.
If you don't, I'LL REVERT ALL POWERPC CODE AFFECTED BY THIS PATCH.

> The CHRP code looks a bit broken, but no more so than before I started.
> 
> > Furthermore, this kind of patch that break other guys work would likely
> > to be discussed and not beeing imposed.
> 
> It's been mentioned several times on this list (by Fabrice, specifically) that 
> this is the way to go.

I did not received any single mail AT ALL saying "we're going to break
your code, you can now throw away all the work you're doing". NOT ONE.
So don't say "it's been discussed several times".

> 
> I applied this now specifically because there are big ARM patches waiting to 
> be applied, and wanted to get this cleanup done before they went in.

I don't contest you that you can do what is needed for ARM. JUST DON'T
DO WEIRD THINGS IN CODE YOU 
SHOULD NOT MODIFY.

J. Mayer <l_indien@magic.fr>
Never organized

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Paul Brook]

On Saturday 07 April 2007 21:28, J. Mayer wrote:
> On Sat, 2007-04-07 at 20:10 +0100, Paul Brook wrote:
> > On Saturday 07 April 2007 19:32, J. Mayer wrote:
> > > On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> > > > CVSROOT:	/sources/qemu
> > > > Module name:	qemu
> > > > Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41
> > >
> > > The patches in the PowerPC target seem complete nonsense.
> >
> > Can you give specific examples?
>
> I'm talking about the CPU code.
> There is NO notion of external IRQ allocation in the PowerPC
> specification.

Quoting from ppc.c (both before and after my patch):

/*****************************************************************************/
/* PowerPC internal fake IRQ controller
 * used to manage multiple sources hardware events

My patch did not change how this worked at all. It just changed it from using 
an ad-hoc system of callpacks and opaque parameters to using a standard 
mechanism for connecting interrupt sources, be they internal or external.

MIPS works exactly the same way.

> Where do you see that a
> machine with a PowerPC cannot manage more than 32 IRQ ?

Any PPC that needs more that 32 internal interrupt sources is already broken. 
Notice how the old bitmask is a uint32_t.

> SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
> HAS TO BE REVERTED.

I can only fix the code that I have access to.

IMHO your request that I revert changes because they allegedly break code that 
noone else has access to is unreasonable. My changes are a significant 
cleanup to qemu internals.

However if there is agreement from other maintainers I will revert my patch.

Paul

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[J. Mayer]

It seems that you don't figure out how real hardware works AT ALL.
IRQ callbacks and private data CANNOT BE SHARED as they are INTERNAL TO
ONE IRQ CONTROLLER. IRQ management, once again, is completelly specific
to an IRQ controller technology and absolutely not related to the CPU
you may use on the same board. So it's a complete nonsense to try to
share IRQ notion in the CPU. IRQ notions and management is completely
different to one IRQ controller technology to another.

On Sat, 2007-04-07 at 21:45 +0100, Paul Brook wrote:
> On Saturday 07 April 2007 21:28, J. Mayer wrote:
> > On Sat, 2007-04-07 at 20:10 +0100, Paul Brook wrote:
> > > On Saturday 07 April 2007 19:32, J. Mayer wrote:
> > > > On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> > > > > CVSROOT:	/sources/qemu
> > > > > Module name:	qemu
> > > > > Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41
> > > >
> > > > The patches in the PowerPC target seem complete nonsense.
> > >
> > > Can you give specific examples?
> >
> > I'm talking about the CPU code.
> > There is NO notion of external IRQ allocation in the PowerPC
> > specification.
> 
> Quoting from ppc.c (both before and after my patch):
> 
> /*****************************************************************************/
> /* PowerPC internal fake IRQ controller
>  * used to manage multiple sources hardware events
> 
> My patch did not change how this worked at all. It just changed it from using 
> an ad-hoc system of callpacks and opaque parameters to using a standard 
> mechanism for connecting interrupt sources, be they internal or external.
> 
> MIPS works exactly the same way.

No. MIPS CPU have an internal IRQ controller and often one or many
external ones. It's completely different and cannot be easily compared
to what done internally in a PowerPC.

> > Where do you see that a
> > machine with a PowerPC cannot manage more than 32 IRQ ?
> 
> Any PPC that needs more that 32 internal interrupt sources is already broken. 
> Notice how the old bitmask is a uint32_t.

You don't understand a single word of what I wrote or you don't
understand at all what is the difference between a CPU and an IRQ
controller.
The code you quote concerns INTERNAL EXCEPTION MANAGEMENT. It's no way
related with an IRQ controller and will never be. One or more exception,
like in any CPU, can be connected to an IRQ controller (or more,
cascaded, for example). The IRQ management is done inside the IRQ
controller, not in the CPU and all the hardware IRQ interface are
SPECIFIC to the IRQ controller. Then, this is to be emulated internally
in the controller and has no notion to be shared in the CPU (as there is
none in the real hardware).
The existing code ACTUALLY WORKS with more than 32 IRQs as you can
cascade any number of IRQ controller one to the other. Each IRQ
controller handle its own events and manage the callbacks to the
cascade. There is only one controller in the chain connected to the CPU.
Then, you can virually have an infinite number of IRQ, managed by a
chain of controllers, using different technologies then cannot share any
data execpt the IRQ events on the cascade. THAT's HOW REAL HARDWARE
WORKS AND THAT'S HOW IT'S EMULATED FOR POWERPC IN THE CODE BEFORE YOU
BREAK IT !

> > SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
> > HAS TO BE REVERTED.
> 
> I can only fix the code that I have access to.

You broke the commited code.... And you also completely break any new
development for the PowerPC target as most real machine cannot be
emulated now (even PREP is broken, using 3 IRQ controllers).

> However if there is agreement from other maintainers I will revert my patch.

I only ask you to revert all that affects the PowerPC target. If the
other developpers want their target to follow this broken way, it does
not hurt me as long as it does not break fundamental concept for PowerPC
emulation.

-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Thiemo Seufer]

J. Mayer wrote:
[snip]
> > > SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
> > > HAS TO BE REVERTED.
> > 
> > I can only fix the code that I have access to.
> 
> You broke the commited code.... And you also completely break any new
> development for the PowerPC target as most real machine cannot be
> emulated now (even PREP is broken, using 3 IRQ controllers).

It might help here if you could provide a PREP sample image which can be
used as a basic smoketest, like the ones for other architectures on
Fabrice's page.


Thiemo

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Paul Brook]

On Saturday 07 April 2007 23:18, J. Mayer wrote:
> It seems that you don't figure out how real hardware works AT ALL.
> IRQ callbacks and private data CANNOT BE SHARED as they are INTERNAL TO
> ONE IRQ CONTROLLER. IRQ management, once again, is completelly specific
> to an IRQ controller technology and absolutely not related to the CPU
> you may use on the same board. So it's a complete nonsense to try to
> share IRQ notion in the CPU. IRQ notions and management is completely
> different to one IRQ controller technology to another.
<snip>

You appear to have misunderstood what my patch does. The arguments you use in 
the part of the email I snipped are the same resons why I created this patch.


The most important thing to understand is that the "qemu_irq" object is simply 
an abstract interface for connecting two devices. One of which produces a 
single-bit output, and the other does something in response to that signal. 
qemu_irq isn't an ideal name because it's used for things other than physical 
IRQ pins. However it's the best we (Me, Theimo, or anyone else on #qemu about 
24 hours ago) could come up with.

The purpose of this abstraction is precisely so that interrupt sources and 
sinks can be chained and rearranged in arbitrary fashion.


In the specific case of PPC a typical system, as you say, has several cascaded 
levels of interrupt handling. You have multiple external interrupt 
controllers, including interrupts from PCI bus based devices. 

Contrary to your assertion, a PPC cpu does contains an internal vectored 
interrupt controller. This handles the external IRQ pin, plus various 
internal interrupt sources (as defined by the PPC_INTERRUPT_* constants). 
This is what the IRQ array in the cpu state structure is for. I see no 
fundamental reason why this should be treated any differently to an external 
interrupt controller.

Because these are currently implemented in the same file I didn't bother 
converting them fully and they mostly still use ppc_set_irq dircetly instead 
of going via qemu_set_irq. If/when ppc.c is split up and/or cpus with more 
than one external IRQ pin are added the generic IRQ routing mechanisms allow 
this to happen without having to export ppc_set_irq.

PPC is a slightly bad example because it still has ppc_openpic_irq. This is 
exactly the sort of horrible system specific hack I'm trying to replace.


Before my patch chained interrupts were handled with a messy ad-hoc 
combination of callbacks and opaque parameters. Some devices were properly 
parameterized, others assumed that their IRQ output went to a specific device 
or CPU.

My patch introduces a generic mechanism for devices to talk to an interrupt 
controller. It means that the code signalling the interrupt doesn't need to 
know or care where it ends up. The logic for implementing a particular 
interrupt controller is still private to that device.

I have not created a single flat IRQ super-controller, just a consistent 
mechanism for glueing the existing bits of emulation together.

> You broke the commited code.... And you also completely break any new
> development for the PowerPC target as most real machine cannot be
> emulated now (even PREP is broken, using 3 IRQ controllers).

Can you send me a testcase of something that worked before my patch and 
doesn't afterwards? As I said above, handling of multiple IRQ controllers is 
significantly better after my patch than it was before.

Paul

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[J. Mayer]

On Sun, 2007-04-08 at 00:13 +0100, Paul Brook wrote:
> On Saturday 07 April 2007 23:18, J. Mayer wrote:
> > It seems that you don't figure out how real hardware works AT ALL.
> > IRQ callbacks and private data CANNOT BE SHARED as they are INTERNAL TO
> > ONE IRQ CONTROLLER. IRQ management, once again, is completelly specific
> > to an IRQ controller technology and absolutely not related to the CPU
> > you may use on the same board. So it's a complete nonsense to try to
> > share IRQ notion in the CPU. IRQ notions and management is completely
> > different to one IRQ controller technology to another.
> <snip>
> 
> You appear to have misunderstood what my patch does. The arguments you use in 
> the part of the email I snipped are the same resons why I created this patch.

So your patch is not useful for the PowerPC target as cascading could
already works and does not need anything hardcoded in the CPU (once
again, IRQ sources is no way related to the CPU).

> The most important thing to understand is that the "qemu_irq" object is simply 
> an abstract interface for connecting two devices. One of which produces a 
> single-bit output, and the other does something in response to that signal. 
> qemu_irq isn't an ideal name because it's used for things other than physical 
> IRQ pins. However it's the best we (Me, Theimo, or anyone else on #qemu about 
> 24 hours ago) could come up with.
> 
> The purpose of this abstraction is precisely so that interrupt sources and 
> sinks can be chained and rearranged in arbitrary fashion.

Which was already done. Why change what works, so ? Why BREAK ?

> In the specific case of PPC a typical system, as you say, has several cascaded 
> levels of interrupt handling. You have multiple external interrupt 
> controllers, including interrupts from PCI bus based devices. 
> 
> Contrary to your assertion, a PPC cpu does contains an internal vectored 
> interrupt controller. This handles the external IRQ pin, plus various 
> internal interrupt sources (as defined by the PPC_INTERRUPT_* constants). 
> This is what the IRQ array in the cpu state structure is for. I see no 
> fundamental reason why this should be treated any differently to an external 
> interrupt controller.

I guess you make a confusion between IRQ, which are external and managed
with an IRQ controller and exceptions, which are handled inside the
PowerPC core. There is no IRQ controller inside the PowerPC. You can
read the specification in any way you want, you will find none.
IRQ controllers are peripheral devices. So, nothing about them can be
managed in the CPU structure. What when there is more than one CPU ? IRQ
number is a notion internal to one IRQ controller and has no meaning
outside of it. How can you even think of using it to reference 

> Because these are currently implemented in the same file I didn't bother 
> converting them fully and they mostly still use ppc_set_irq dircetly instead 
> of going via qemu_set_irq. If/when ppc.c is split up and/or cpus with more 
> than one external IRQ pin are added the generic IRQ routing mechanisms allow 
> this to happen without having to export ppc_set_irq.
> 
> PPC is a slightly bad example because it still has ppc_openpic_irq. This is 
> exactly the sort of horrible system specific hack I'm trying to replace.

Once again, I guess you don't understand what this code does. Openpic,
as many IRQ controllers, has a lot of inputs and not only a single
output.This code is there to connect external IRQ sources comming from
an OpenPIC to internal exceptions of the PowerPC. As this is hardwired
in the PowerPC core and cannot be changed, it's not especially a hack.
To be _really_ generic and do exactly what is done is real hardware
devices, it would need another cascaded stage. I did not add this stage
because it would have just add performance penalties but would not have
added any real advantage, imho.

> Before my patch chained interrupts were handled with a messy ad-hoc 
> combination of callbacks and opaque parameters. Some devices were properly 
> parameterized, others assumed that their IRQ output went to a specific device 
> or CPU.

So, you just have to fix the messy devices. Not to break the CPU
emulation code or structure.

> My patch introduces a generic mechanism for devices to talk to an interrupt 
> controller. It means that the code signalling the interrupt doesn't need to 
> know or care where it ends up. 

That's exactly why some IRQ callbacks have been added in some devices,
just not to have to care where the interrupt ends up or even if the IRQ
is actually raised. No need to break anything. Just fix the old devices
that did not use this logic.

> The logic for implementing a particular 
> interrupt controller is still private to that device.

Your patch is even worse than what already exists and work, add
confusion by breaking concepts (the IRQ controller is an external
device, whatever the CPU you use), add garbage in the CPU structure, add
a mess to handle multi-cpu/multi PIC systems (which are more than
common, these days), add arbitrary limits with no justifications, ....
So, no new features, broken concepts, new mess, new limitations and more
complicated code...
It's unacceptable.

To give you an real example why arbitrary limits are not acceptable AT
ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
and about 500 IRQ sources. How can you even pretend add a limited
structure in the CPUState structure when this is exactly the kind of
device some people want to emulate in Qemu ? Your concept is completely
broken, you have to admit it. You can never put peripheral informations
in the CPUState structure. What if I decide to put the NIC registers in
the CPU structure ? It's exactly the same as what you did...


The funny thing is that you even did not fix the only bug that makes the
OpenPic not really cascadable: the last argument of the openpic_init has
to be changed to be void ** instead of CPUState ** (same in
the .IRQ_dst_t structure). No need for more (but this patch is needed, I
did not commit it because I don't need it just now...)

[...]

-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Thiemo Seufer]

J. Mayer wrote:
[snip]
> To give you an real example why arbitrary limits are not acceptable AT
> ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
> and about 500 IRQ sources.

Care to tell which one this is?

> How can you even pretend add a limited
> structure in the CPUState structure when this is exactly the kind of
> device some people want to emulate in Qemu ? Your concept is completely
> broken, you have to admit it. You can never put peripheral informations
> in the CPUState structure.

At least for MIPS it makes sense to put the CPU-internal controller
in exactly that place.


Thiemo

Re: IRQ handling (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[J. Mayer]

On Sun, 2007-04-08 at 01:04 +0100, Thiemo Seufer wrote:
> J. Mayer wrote:
> [snip]
> > To give you an real example why arbitrary limits are not acceptable AT
> > ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
> > and about 500 IRQ sources.
> 
> Care to tell which one this is?

I'm sorry, I'm no sure I can (NDA rules....).
Let's say it's a chips used in some consumer electronics products.

> > How can you even pretend add a limited
> > structure in the CPUState structure when this is exactly the kind of
> > device some people want to emulate in Qemu ? Your concept is completely
> > broken, you have to admit it. You can never put peripheral informations
> > in the CPUState structure.
> 
> At least for MIPS it makes sense to put the CPU-internal controller
> in exactly that place.

It does not. If you look well, the IRQ controller is not in the CPU.
Only the exception are managed in the CPU. The "internal" IRQ controller
is a peripheral device located in the CP0. OK, the CP0 is tightly
coupled to the CPU so it's easier to consider it as part of the CPU,
when emulating it. But it seems like you could imagine a MIPS CPU
without a CP0 coprocessor (even if it will never happen in the real
world), no ?

The problem is also: what does this patch adds, but complexity and
arbitrary limitations ?
What do you need to route an IRQ ?
-> A peripheral destination
What we got now ?
-> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
state
Is more needed to have a generic routing approach ?
-> no. This can work to route any signal
Can we do with less ?
-> no. You need the 3 parameters.
Can we share data ?
-> It seems not. The opaque is used internally by the IRQ controller.
The PIN number has a meaning only inside the IRQ controller. And the
meaning of the state depends on the IRQ controller state.
The only thing that could be changed to hide the complexity is define a
structure that contain all information, like the struct IRQState.
But this structure cannot be shared as it has no signification outside
of an IRQ controller.
What can be done (but once again, it changes nothing, just hide the
"complexity"), is to replace the  { callback, n_IRQ } in devices
structure by a IRQState structure and have the inline functions.

static inline void qemu_irq_set_state(struct IRQState *st, int level)
{
      if (st->handler != NULL)
        (*st->handler)(st->opaque, st->n, level);
}
static inline void qemu_irq_raise(struct IRQState *st)
{
    qemu_irq_set_state(st, 1);
}
static inline void qemu_irq_lower(struct IRQState *st)
{
    qemu_irq_set_state(st, 0);
}

If you want to be generic, as it is said it's supposed to be (but is not) to be able to manage any signal, then this is not sufficient.
One will also have to get the value. If you want to emulate GPIOs, for example, you need to store the current state
to be able to get it back easily. And for GPIO, you may prefer that changing the state has no action but just be able
to read back the current PIN level at any time.

To achieve this, you have to have a structure:
struct PINState {
    qemu_pin_handler handler;
    void *opaque;
    int n;
    int level;
};

static inline void qemu_pin_set_state(struct PINState *st, int level)
{
      if (st->handler != NULL && level != st->level != level) {
        (*st->handler)(st->opaque, st->n, level);
        st->level = level;
    }
}
static inline int qemu_pin_get_state(struct PINState *st)
{
	return st->level;
}

static inline void qemu_ping_raise(struct PINState *st)
{
    qemu_pin_pin_state(st, 1);
}
static inline void qemu_pin_lower(struct PINState *st)
{
    qemu_pin_set_state(st, 0);
}

The hw/irq.c code is not needed and meaningless. The first function should be inlined, unless the goal is to slowdown the emulator.
The second function has no meaning: you can never globally allocate resources that have no meaning outside of a limited
scope, which is the case for IRQ or any physical connection on a real hardware.

The PINState structure can be store only in the peripheral device that will change the PIN state and the peripheral that will read this state.
Trying to store it in a generic place is completely meaningless: a PIN state (I talk about real hardware, now) has absolutelly
no generic meaning. It's only significant between two devices. You  eventually can have more than one device changing the
PIN state (shared IRQ, I2C bus, ....) and more than one device using this PIN state (I2C, once again). But outside of the scope
of those devices, it means nothing. So the information is to be stored inside those devices, anywhere else is meaningless.


-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: IRQ handling (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[J. Mayer]

On Sun, 2007-04-08 at 09:49 +0200, J. Mayer wrote:

> I'm sorry, I'm no sure I can (NDA rules....).
> Let's say it's a chips used in some consumer electronics products.

I gave this example to show that we cannot pre-determine any limit to
the number of PINs we have to manage, even if it's a poor design and
that no one need those 500 IRQ sources (50 would be sufficient if it was
well designed but that the way the actual hardware has been
designed...).

Here's an example which is documented:
A CHRP machine is supposed to have an OpenPIC controller (up to 32 IRQ
sources and up to 4 output IRQ pins) and 2 i8259. This simple machine
already has more than 32 IRQs.
Some may also have 2 cascaded OpenPICs.
It will also have one or more PCI controllers (which have it's own IRQ
controller, n inputs, 4 outputs, on actual hardware).
Some complex PCI devices may also have an internal IRQ controller if
they can generate a lot of different hardware events. Those devices can
even be hotplugged.

Then, can you pre-determine the maximum number of IRQ of a hardware
platform ? I can't, because it's conceptually impossible.

[....]

Ooops... Lots of typo in what I wrote....

> static inline void qemu_pin_set_state(struct PINState *st, int level)
> {
>       if (st->handler != NULL && level != st->level != level) {

I meant  .... && level != st->level) {

[...]

> static inline void qemu_ping_raise(struct PINState *st)

I meant qemu_pin_raise (not ping).
You can even have an allocator to ease sharing the information between
the devices, like:
static inline PINState *qemu_pin_alloc (void *opaque, int n, int
initial_level)
{
    PINState *st;

    st = qemu_mallocz(sizeof(PINState));
    if (st != NULL) {
        st->opaque = opaque;
        st->n = n;
        st->level = initial_level;
    }
}

And note that if you really want to emulate GPIOs, you may also need to
add a default_level field to the structure and a bitmask to know which
devices are really driving the PIN (as it reaches its default state only
when no one is driving it). and a new function to signal the case where
the PIN is beeing released (not drived) by a peripheral. And a
peripheral need to identify itself when calling the pin_raise,
pin_lower, pin_release function. The only limitation of this approach
may be the number of peripheral driving one single signal on the board,
which can not be > of the size of the bitfield. But you can have as many
handled signals as you need too.

Once again, one of the two structure is not needed (qemu_irq /
IRQState): there is only one physical state so having 2 different
description of the same thing is conceptually broken. Nothing can be
stored in a generic place as a PIN connection is a completely local
concept between a set of devices. And once again, IRQ are not bound to a
CPU. Storing any informations about IRQs in the CPU structure is
meaningless. Where do you store the IRQ informations when you have 2, 4
CPUs connected to one OpenPIC ? Choose a random CPU ? Only use one ?

Then, we can see the code that has been commited does not solve the
problems of generic signal routing and is conceptually completelly
broken.

One last point:
I thought more about the ppc_openpic_irq function and I have to admit
it's not the good approach.
Yes, this has to be changed to store the IRQ destination inside the
OpenPIC controller and directly call the ppc_set_irq callback. In fact,
there should be a function replacing the ppc_openpic_irq one that
handles the external exception sources and callback ppc_set_irq (this
function also manages internal hardware exception sources). The code in
hw/ppc.c would not be changed a lot but the OpenPIC would be more
generic and may allow some crazy schemes where the outputs would not be
connected to the same destination device (usually the CPU). And it may
make easier to use OpenPIC on hardware that are not PowerPC base (with
no hack), which is imho more important (OpenPIC was designed for and
used on AMD SMP PC platforms, originally). I'll change this, once the
IRQ problems will be solved (which seems urgent, more than anything
else).

-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: IRQ handling (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Thiemo Seufer]

J. Mayer wrote:
> On Sun, 2007-04-08 at 01:04 +0100, Thiemo Seufer wrote:
> > J. Mayer wrote:
> > [snip]
> > > To give you an real example why arbitrary limits are not acceptable AT
> > > ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
> > > and about 500 IRQ sources.
> > 
> > Care to tell which one this is?
> 
> I'm sorry, I'm no sure I can (NDA rules....).
> Let's say it's a chips used in some consumer electronics products.
> 
> > > How can you even pretend add a limited
> > > structure in the CPUState structure when this is exactly the kind of
> > > device some people want to emulate in Qemu ? Your concept is completely
> > > broken, you have to admit it. You can never put peripheral informations
> > > in the CPUState structure.
> > 
> > At least for MIPS it makes sense to put the CPU-internal controller
> > in exactly that place.
> 
> It does not. If you look well, the IRQ controller is not in the CPU.
> Only the exception are managed in the CPU. The "internal" IRQ controller
> is a peripheral device located in the CP0.

It is not a peripheral but an integral part of any MIPS-compatible CPU.
The architecture allows since MIPS{32,64}R2 to optionally externalize
interupts (the so-called VEIC mode), but even those devices have to
implement the traditional "compatibility mode" interrupt handling.

This is spelt out e.g. in MD00091, page 32, as available from
http://www.mips.com/products/resource_library/product_materials/

> OK, the CP0 is tightly
> coupled to the CPU so it's easier to consider it as part of the CPU,
> when emulating it. But it seems like you could imagine a MIPS CPU
> without a CP0 coprocessor (even if it will never happen in the real
> world), no ?

No. Since MIPS{32,64}R2 the CP0 is standardized and a mandatory part of
a MIPS compatible CPU.

> The problem is also: what does this patch adds, but complexity and
> arbitrary limitations ?

For MIPS it adds an abstraction layer between the interrupt controller
and the CP0 registers which will be useful to implement support for
SMP devices.

> What do you need to route an IRQ ?
> -> A peripheral destination
> What we got now ?
> -> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
> state

A pin number doesn't look like a reasonable abstraction for a packetized
interrupt on an interconnection fabric (like it is used on e.g O2000).
It may do for the machines we currently emulate, though.


Thiemo

Re: IRQ handling (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[J. Mayer]

On Sun, 2007-04-08 at 15:41 +0100, Thiemo Seufer wrote:
> J. Mayer wrote:
> > On Sun, 2007-04-08 at 01:04 +0100, Thiemo Seufer wrote:
> > > J. Mayer wrote:
> > > [snip]
> > > > To give you an real example why arbitrary limits are not acceptable AT
> > > > ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
> > > > and about 500 IRQ sources.
> > > 
> > > Care to tell which one this is?
> > 
> > I'm sorry, I'm no sure I can (NDA rules....).
> > Let's say it's a chips used in some consumer electronics products.
> > 
> > > > How can you even pretend add a limited
> > > > structure in the CPUState structure when this is exactly the kind of
> > > > device some people want to emulate in Qemu ? Your concept is completely
> > > > broken, you have to admit it. You can never put peripheral informations
> > > > in the CPUState structure.
> > > 
> > > At least for MIPS it makes sense to put the CPU-internal controller
> > > in exactly that place.

For the internal IRQ controller, OK. For the external ones, it makes no
sense to put any informations about them into the CPU.

> > It does not. If you look well, the IRQ controller is not in the CPU.
> > Only the exception are managed in the CPU. The "internal" IRQ controller
> > is a peripheral device located in the CP0.
> 
> It is not a peripheral but an integral part of any MIPS-compatible CPU.
> The architecture allows since MIPS{32,64}R2 to optionally externalize
> interupts (the so-called VEIC mode), but even those devices have to
> implement the traditional "compatibility mode" interrupt handling.
> 
> This is spelt out e.g. in MD00091, page 32, as available from
> http://www.mips.com/products/resource_library/product_materials/

> > OK, the CP0 is tightly
> > coupled to the CPU so it's easier to consider it as part of the CPU,
> > when emulating it. But it seems like you could imagine a MIPS CPU
> > without a CP0 coprocessor (even if it will never happen in the real
> > world), no ?
> 
> No. Since MIPS{32,64}R2 the CP0 is standardized and a mandatory part of
> a MIPS compatible CPU.

Yes, I know MIPS want always CP0 to be present. I should have put a
smiley somewhere. I wanted just to point the fact that the CPU itself
does not need the CP0 controller to run and that one could easily
imagine designing a core without the CP0 controller. Yes, this would not
follow the specification but it would be able to run....
The biggest conceptual problem would you have to read a CP0 register when you want to know what coprocessor are actually implemented !
But all this is a foolish idea....

> > The problem is also: what does this patch adds, but complexity and
> > arbitrary limitations ?
> 
> For MIPS it adds an abstraction layer between the interrupt controller
> and the CP0 registers which will be useful to implement support for
> SMP devices.

Could you please explain me what you can do now that you were not able
too without this patch ?

> 
> > What do you need to route an IRQ ?
> > -> A peripheral destination
> > What we got now ?
> > -> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
> > state
> 
> A pin number doesn't look like a reasonable abstraction for a packetized
> interrupt on an interconnection fabric (like it is used on e.g O2000).
> It may do for the machines we currently emulate, though.

Could you please tell me more about this problem ?
And could you please explain me how the patch solves this, as it just
does not seem to me that it uses more than an IRQ number to identify an
IRQ ?

I though a little bit more about a generic IRQ and/or IO implementation
and it seems that the commited patch is far from solving any pending
problem about it.
What appears to me is that a IO can have 3 "natural state", when not
driven:
- pulled up
- pulled low
- floating
An IO driver can then have 3 actions on an IO line:
- drive it up
- drive it low
- release it
It appears to me that all the possible combinations of thoses natural
states / driving states have to be managed properly to emulate IRQ.
For ISA like IRQ, that are not sharable, IRQ lines are naturally
floating and can be drived up or low
For PCI like IRQs, that are sharable, IRQ lines are naturally pulled up
and can be drived down or released.
Some other designs may use (less often) pulled down sharable IRQ lines.
The first idea is to say that floating IOs can only be driven up or low
and pulled up/low IO can only be driven or not driven. It appears that
because of poorly designed hardware or (more often) software bugs, weird
situations can happen and must be handled: floating IO not driven (which
may gives undefined state on real hardware) or pulled up IO drived up
(same for down, this may give intermediate states between logical 0 and
1 thus product "strange" results).
All those situations can be seen on IRQ line so a generic IRQ
implementation have to manage all those cases.
Now, let's see the other side of the connection. A simplistic approach
would be to say we just need a callback or a stored value. Thinking a
little more about it shows that we cannot do without having at least 2
possible outputs. The reason is that you often have the possibility for
the driving device to read back the value of the IRQ pin and that you
cannot rely on the device internal state for this as an IO may be shared
or may not have the awaited value, especially if not driven properly by
the software.
Even worse, if you are to drive an IRQ with a pure GPIO pin you will
have to set this pin as an bidirectional I/O if you want to be able to
read its actual value back. On some embedded hardware, where I/O pins
cannot be set in a bidirectional mode, you may have to use 2 I/O to
drive and read back an IRQ line state. And it's very useful to be able
to read back hardware states, and quite usual.
My conclusion about it is that it seems that it is mandatory to manage
the 3 possible natural states of an IO, the 3 "driven" state, multiple
sources and multiple outputs if you want to have a correct emulation of
physical IRQ lines behavior. It seems all the work is still to be done
to achieve this.
And doing this would provide a correct emulation for IRQ line and it
seems to me that it will be perfectly usable to emulate any kind of GPIO
line.
But I still cannot agree that the informations about an IO line is to be
stored anywhere else than in the driver device and the receiver one. CPU
and other devices even don't have any "knowledge" of an IO line that is
not connected to them, thus should never have to carry any information
about it.
And I still cannot see why two different structures would be needed to
express one single simple concept of an IO connection.
Please explain me about those points if I'm wrong.

-- 
J. Mayer <l_indien@magic.fr>
Never organized

QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Natalia Portillo]

Hi all,

I have a huge list of operating systems (both closed and open source, that
works and that doesn't work under QEMU) that can be used to check that QEMU
doesn't broke (or even, that it corrects a non-working state).

I've already discussed that with Fabrice.

And I think it is better to check an installed image that to test it only
boots or installs (faster at least).

Fabrice and me discussed about taking screenshot per some seconds to compare
with known took screenshots and if something fails shutdown the VM and send
an email.

But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.

The best solution I think is to get a way to send QEMU the screenshot to
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)

What do you think?

I have enough spare space to hold the boot images (600Gibibytes free), and a
machine that should be able to automatically checkout and compile QEMU
(Athlon XP 2000+, 768Mb RAM, 600GiB free, Gentoo Linux).

Just, I have not the knowledge to make the script that boots qemu, says qemu
to take the screenshot, compares the took one with the last one, stops qemu,
sends the last screenshot by email, compresses all took screenshots, goes to
next VM, so on. (Preferibly without X11 running, as the machine is a mostly
headless P2P and File server)

If anyone can do this, I will make the boot images for all my OSes and start
it.

I think, it is a great idea (and a better way to update my -currently dead
but no more spammed- official OS support list).

Regards,
Natalia Portillo 

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Eduardo Felipe]

[-- Attachment #1: Type: text/plain, Size: 1256 bytes --]

>
>
> But that required some macro interface "click at x,y, wait some seconds,
> press 'k' key", that is not currently under QEMU.


It does exist... it's called VNC server. Maybe you can have a look at:

http://www.sodan.org/~penny/vncrec/ <http://www.sodan.org/%7Epenny/vncrec/>

or

http://suif.stanford.edu/vncplay/freenix05-html/

I don't know any of them, but they can be useful.

Also this one:

http://www.unixuser.org/~euske/vnc2swf/pyvnc2swf.html
<http://www.unixuser.org/%7Eeuske/vnc2swf/pyvnc2swf.html>

It's vncrec compatible and can record and replay sessions.

The best solution I think is to get a way to send QEMU the screenshot to
> file command some times and stop the VM when both are equal, then send the
> last took screenshot to a mail address so breakability can be checked. (If
> it is the login window, great, it BOOTS!, if not, it doesn't)
>
>
pyvnc2swf has the ability to create image files from VNC sessions. It is
easily customizable and a special instruction could be recorded into the
.vnc files meaning:

1) Request a full screen update
2) Turn screen buffer into an image
3) Compare image to reference

Control points could be chosen in recording time and inserted into the .vnc
by some key combination.

Regards,
Eduardo

[-- Attachment #2: Type: text/html, Size: 2005 bytes --]

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Natalia Portillo]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

It doesn't fullfil the requirements for an automated testing,  
however, it is a good idea, but not inside the plannings I proposed.

El 08/04/2007, a las 23:07, Eduardo Felipe escribió:

>
> But that required some macro interface "click at x,y, wait some  
> seconds,
> press 'k' key", that is not currently under QEMU.
>
> It does exist... it's called VNC server. Maybe you can have a look at:
>
> http://www.sodan.org/~penny/vncrec/
>
> or
>
> http://suif.stanford.edu/vncplay/freenix05-html/
>
> I don't know any of them, but they can be useful.
>
> Also this one:
>
> http://www.unixuser.org/~euske/vnc2swf/pyvnc2swf.html
>
> It's vncrec compatible and can record and replay sessions.
>
> The best solution I think is to get a way to send QEMU the  
> screenshot to
> file command some times and stop the VM when both are equal, then  
> send the
> last took screenshot to a mail address so breakability can be  
> checked. (If
> it is the login window, great, it BOOTS!, if not, it doesn't)
>
>
> pyvnc2swf has the ability to create image files from VNC sessions.  
> It is easily customizable and a special instruction could be  
> recorded into the .vnc files meaning:
>
> 1) Request a full screen update
> 2) Turn screen buffer into an image
> 3) Compare image to reference
>
> Control points could be chosen in recording time and inserted into  
> the .vnc by some key combination.
>
> Regards,
> Eduardo


[-- Attachment #2: Type: text/html, Size: 2389 bytes --]

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Eduardo Felipe]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

Sorry, I'm afraid I badly mixed up things in my mind.

vncrec and pyvnc2swf record VNC server responses, not client actions, so
they will hardly be of any help.

To record and replay client actions these tools are more appropriate:

http://cyberelk.net/tim/rfbproxy/
http://cyberelk.net/tim/rfbplaymacro/

Although they seem not to be available at this moment...

Regards,
Eduardo

[-- Attachment #2: Type: text/html, Size: 671 bytes --]

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Rob Landley]

On Sunday 08 April 2007 4:43 pm, Natalia Portillo wrote:
> Hi all,
> 
> I have a huge list of operating systems (both closed and open source, that
> works and that doesn't work under QEMU) that can be used to check that QEMU
> doesn't broke (or even, that it corrects a non-working state).

I vaguely intend to get Firmware Linux to rebuild itself under qemu.  Right 
now it's building target images for nine different platforms.  (Although 
that's not as impressive as it sounds.  Two of them, sparc and powerpc, don't 
actually run under qemu yet.  i586 and armv5l are trivial variants of i686 
and armv4l.  And and mips big endian just got added today and wasn't in the 
last release.)

Anyway, I've found compiling large software packages to be a reasonable stress 
test for most systems, and it's entirely scriptable.  Not much of a test of 
any of the peripheral hardware, though.

Rob
-- 
Penguicon 5.0 Apr 20-22, Linux Expo/SF Convention.  Bruce Schneier, Christine 
Peterson, Steve Jackson, Randy Milholland, Elizabeth Bear, Charlie Stross...

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Jamie Lokier]

Natalia Portillo wrote:
> But that required some macro interface "click at x,y, wait some seconds,
> press 'k' key", that is not currently under QEMU.
> 
> The best solution I think is to get a way to send QEMU the screenshot to
> file command some times and stop the VM when both are equal, then send the
> last took screenshot to a mail address so breakability can be checked. (If
> it is the login window, great, it BOOTS!, if not, it doesn't)

If there was a way to get QEMU to stop when the screen matches a
particular stored screenshot, that would make it a lot more scriptable.

For example, you could record the steps to install a particular OS
from installation media as a series of screenshots showing dialog
boxes, and script commands like pressing keys or clicking at x, y when
those screens match.

Another useful event to stop on would be when the processor hits an
x86 HLT instruction or whatever is the equivalent on other
architectures, or particular recognised waiting loops - in conjunction
with there being no outstanding I/O.  Maybe that's an appropriate time
to check for matching screenshots.

That kind of script would be nice when you want to create a new fresh
installation of some OS, but with a few options changed.  The script
would not be too hard to stop at a certain point, and set different
option in the installation, then continue.  Currently I do this
manually and it's quite tedious.

Similarly for actually running applications in a VM.

-- Jamie

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Pierre d'Herbemont]

On 8 avr. 07, at 22:43, Natalia Portillo wrote:

> But that required some macro interface "click at x,y, wait some  
> seconds,
> press 'k' key", that is not currently under QEMU.

There is cxtest (GPL) which has been created for that matter, it  
should be interesting to give it a try:
http://www.cxtest.org/

"Cxtest is an open source project that provides visual regression  
testing facilities for X11 based systems. It can automate basic X  
Window functionality, including finding X windows by title or by  
graphic picture"


Pierre.

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Dan Shearer]

On Sun, Apr 08, 2007 at 09:43:20PM +0100, Natalia Portillo wrote:

Whoops, slightly late reply :-)

> I have a huge list of operating systems (both closed and open source, that
> works and that doesn't work under QEMU) that can be used to check that QEMU
> doesn't broke (or even, that it corrects a non-working state).

Where the problem with QEMU is not something basic (eg doesn't boot) it
can be useful to use nested virtualisation. Although QEMU in QEMU is of
course slow because that case hasn't been optimised, if you have a
single image within which are many other images savevm'd just before
the error condition occurs, you can run a repeatable testsuite quite
quickly. And since you can snapshot the outer image, you know that you
are running these tests in exactly the same way every time. (There are a
lot of other uses for recursive virtualisation, I'm a fan of it!)

> And I think it is better to check an installed image that to test it only
> boots or installs (faster at least).

Agreed. As to installs I had to do something similar for different VM
systems a while ago and I found it useful to have a frozen image just
before the typical "detecting hardware" phase of an installation. The
point of this was not to compare the hardware that was detected, just to
detect major malfunctions. Hardware probing like this is uniquely
stressful. It isn't hard to notice if an installer crashes, or the VM
crashes, or there's lot's of errors. 

> Fabrice and me discussed about taking screenshot per some seconds to compare
> with known took screenshots and if something fails shutdown the VM and send
> an email.
> 
> But that required some macro interface "click at x,y, wait some seconds,
> press 'k' key", that is not currently under QEMU.

Why can't you redirect the QEMU monitor to something easily accessible,
eg a virtual serial port, and then use mouse_move and sendkey monitor
commands to drive testing, and screendump to save images for comparison?
Those commands have been in the monitor for years, so is there something
I'm not understanding here?

> The best solution I think is to get a way to send QEMU the screenshot to
> file command some times and stop the VM when both are equal, then send the
> last took screenshot to a mail address so breakability can be checked. (If
> it is the login window, great, it BOOTS!, if not, it doesn't)
> 
> What do you think?

The problem is synchronisation given that QEMU has no guaranteed time
domain, so you don't know if your target is failing or just being slow
so you keep on taking screenshots for a ridiculously long time.  You can
take fewer screenshots if you have some out-of-band signal. For example
knowing to expect a particular screenshot around the time that a
particular file is touched or a particular hardware device is
initialised, or a magic QEMU-specific CPU instruction is executed.

> I have enough spare space to hold the boot images (600Gibibytes free), and a
> machine that should be able to automatically checkout and compile QEMU
> (Athlon XP 2000+, 768Mb RAM, 600GiB free, Gentoo Linux).

Compiling is really important. Even without testing targets, this would
be a very good community service if you want to do even just this much.
Compiling a simulator under itself in all supported configurations is an
important set of tests (compile/run MIPS under IA32, IA32 under PPC32,
etc especially 32/64 mixes, then the different supported compiler
versions, and then a couple of different Linux distributions with their
different choices of library versions and configurations.) This is
probably 24 hours of hard work. I have seen simulators choke on these
sorts of tests.

> Just, I have not the knowledge to make the script that boots qemu, says qemu
> to take the screenshot, compares the took one with the last one, stops qemu,
> sends the last screenshot by email, compresses all took screenshots, goes to
> next VM, so on. (Preferibly without X11 running, as the machine is a mostly
> headless P2P and File server)

I think this can be simplified. Care to send a copy offlist of what
you've done and i'll have a look at it?

--
Dan Shearer
dan@shearer.org

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Anthony Liguori]

FYI, I've started building a VNC based automated tester.  You provide it 
a series of screenshots with masks of data that's likely to be 
semi-random and it'll wait for each screen shot.  It also has the smarts 
to find where the mouse is so that you can move it to an exact location.

I'll have something functional (along with a paper) within a month.

Regards,

Anthony Liguori

Dan Shearer wrote:
> On Sun, Apr 08, 2007 at 09:43:20PM +0100, Natalia Portillo wrote:
>
> Whoops, slightly late reply :-)
>
>   
>> I have a huge list of operating systems (both closed and open source, that
>> works and that doesn't work under QEMU) that can be used to check that QEMU
>> doesn't broke (or even, that it corrects a non-working state).
>>     
>
> Where the problem with QEMU is not something basic (eg doesn't boot) it
> can be useful to use nested virtualisation. Although QEMU in QEMU is of
> course slow because that case hasn't been optimised, if you have a
> single image within which are many other images savevm'd just before
> the error condition occurs, you can run a repeatable testsuite quite
> quickly. And since you can snapshot the outer image, you know that you
> are running these tests in exactly the same way every time. (There are a
> lot of other uses for recursive virtualisation, I'm a fan of it!)
>
>   
>> And I think it is better to check an installed image that to test it only
>> boots or installs (faster at least).
>>     
>
> Agreed. As to installs I had to do something similar for different VM
> systems a while ago and I found it useful to have a frozen image just
> before the typical "detecting hardware" phase of an installation. The
> point of this was not to compare the hardware that was detected, just to
> detect major malfunctions. Hardware probing like this is uniquely
> stressful. It isn't hard to notice if an installer crashes, or the VM
> crashes, or there's lot's of errors. 
>
>   
>> Fabrice and me discussed about taking screenshot per some seconds to compare
>> with known took screenshots and if something fails shutdown the VM and send
>> an email.
>>
>> But that required some macro interface "click at x,y, wait some seconds,
>> press 'k' key", that is not currently under QEMU.
>>     
>
> Why can't you redirect the QEMU monitor to something easily accessible,
> eg a virtual serial port, and then use mouse_move and sendkey monitor
> commands to drive testing, and screendump to save images for comparison?
> Those commands have been in the monitor for years, so is there something
> I'm not understanding here?
>
>   
>> The best solution I think is to get a way to send QEMU the screenshot to
>> file command some times and stop the VM when both are equal, then send the
>> last took screenshot to a mail address so breakability can be checked. (If
>> it is the login window, great, it BOOTS!, if not, it doesn't)
>>
>> What do you think?
>>     
>
> The problem is synchronisation given that QEMU has no guaranteed time
> domain, so you don't know if your target is failing or just being slow
> so you keep on taking screenshots for a ridiculously long time.  You can
> take fewer screenshots if you have some out-of-band signal. For example
> knowing to expect a particular screenshot around the time that a
> particular file is touched or a particular hardware device is
> initialised, or a magic QEMU-specific CPU instruction is executed.
>
>   
>> I have enough spare space to hold the boot images (600Gibibytes free), and a
>> machine that should be able to automatically checkout and compile QEMU
>> (Athlon XP 2000+, 768Mb RAM, 600GiB free, Gentoo Linux).
>>     
>
> Compiling is really important. Even without testing targets, this would
> be a very good community service if you want to do even just this much.
> Compiling a simulator under itself in all supported configurations is an
> important set of tests (compile/run MIPS under IA32, IA32 under PPC32,
> etc especially 32/64 mixes, then the different supported compiler
> versions, and then a couple of different Linux distributions with their
> different choices of library versions and configurations.) This is
> probably 24 hours of hard work. I have seen simulators choke on these
> sorts of tests.
>
>   
>> Just, I have not the knowledge to make the script that boots qemu, says qemu
>> to take the screenshot, compares the took one with the last one, stops qemu,
>> sends the last screenshot by email, compresses all took screenshots, goes to
>> next VM, so on. (Preferibly without X11 running, as the machine is a mostly
>> headless P2P and File server)
>>     
>
> I think this can be simplified. Care to send a copy offlist of what
> you've done and i'll have a look at it?
>
> --
> Dan Shearer
> dan@shearer.org
>
>
>
>   

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Dan Shearer]

On Fri, Jul 27, 2007 at 09:29:11AM -0500, Anthony Liguori wrote:
> FYI, I've started building a VNC based automated tester.  You provide it 
> a series of screenshots with masks of data that's likely to be 

That's excellent.

I see this as a different class of testing to the tinderbox-style
testing Natalia Portillo was talking about, checking out and compiling
under different circumstances. That's very valuable, and will help
address a problem that drives away a lot of potential QEMU users - they
just can't compile it.

-- 
Dan Shearer
dan@shearer.org

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Sunil Amitkumar Janki]

Dan Shearer wrote:
> On Fri, Jul 27, 2007 at 09:29:11AM -0500, Anthony Liguori wrote:
>   
>> FYI, I've started building a VNC based automated tester.  You provide it 
>> a series of screenshots with masks of data that's likely to be 
>>     
>
> That's excellent.
>
> I see this as a different class of testing to the tinderbox-style
> testing Natalia Portillo was talking about, checking out and compiling
> under different circumstances. That's very valuable, and will help
> address a problem that drives away a lot of potential QEMU users - they
> just can't compile it.

That would be great. I am regularly compiling and using QEMU CVS
snapshots because features get added all the time. A lot of the time
it happens that these build fine on x86 whereas they don't work on
other architectures because the developers don't have access to those.

I have been running self-built Slackware 12.0 on MIPS for the last
few months and would like to run QEMU on that architecture. previously
it built but didn't run, but now it doesn't even compile anymore. The
error message I get is the following:

> gcc -O2 -march=r4600 -Wall -O2 -g -fno-strict-aliasing -I. -I.. 
> -I/root/tmp/qemu-20070727/target-i386 -I/root/tmp/qemu-20070727 
> -I/root/tmp/qemu-20070727/linux-user 
> -I/root/tmp/qemu-20070727/linux-user/i386 -D_GNU_SOURCE 
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
> -I/root/tmp/qemu-20070727/fpu -DHAS_AUDIO -DHAS_AUDIO_CHOICE 
> -I/root/tmp/qemu-20070727/slirp    -c -o gdbstub.o 
> /root/tmp/qemu-20070727/gdbstub.c
> gcc -g  -Wl,-T,/root/tmp/qemu-20070727/mipsel.ld   -o qemu-i386 main.o 
> syscall.o mmap.o signal.o path.o osdep.o thunk.o elfload.o linuxload.o 
> vm86.o libqemu.a gdbstub.o   -lm -lrt -lasound
> /usr/bin/ld: cannot find -lm
> collect2: ld returned 1 exit status
> make[1]: *** [qemu-i386] Error 1
> make[1]: Leaving directory `/root/tmp/qemu-20070727/i386-linux-user'
> make: *** [subdir-i386-linux-user] Error 2
> make -C i386-linux-user all
> make[1]: Entering directory `/root/tmp/qemu-20070727/i386-linux-user'
> gcc -g  -Wl,-T,/root/tmp/qemu-20070727/mipsel.ld   -o qemu-i386 main.o 
> syscall.o mmap.o signal.o path.o osdep.o thunk.o elfload.o linuxload.o 
> vm86.o libqemu.a gdbstub.o   -lm -lrt -lasound
> /usr/bin/ld: cannot find -lm
> collect2: ld returned 1 exit status
> make[1]: *** [qemu-i386] Error 1
> make[1]: Leaving directory `/root/tmp/qemu-20070727/i386-linux-user'
> make: *** [subdir-i386-linux-user] Error 2

I will try to get SPARC Linux and Solaris running on my newly acquired Sun
Enterprise 250 server and compile there as well so I can test snapshots on
all the architectures I have at my disposal.

Sunil

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Dan Shearer]

On Fri, Jul 27, 2007 at 04:58:19PM +0200, Sunil Amitkumar Janki wrote:
> Dan Shearer wrote:
> > 
> >I see this as a different class of testing to the tinderbox-style
> >testing Natalia Portillo was talking about, checking out and compiling
> >under different circumstances. That's very valuable, and will help
> >address a problem that drives away a lot of potential QEMU users - they
> >just can't compile it.
> 
> That would be great. I am regularly compiling and using QEMU CVS
> snapshots because features get added all the time. A lot of the time
> it happens that these build fine on x86 whereas they don't work on
> other architectures because the developers don't have access to those.
> 
> I have been running self-built Slackware 12.0 on MIPS for the last
> few months and would like to run QEMU on that architecture. previously
> it built but didn't run, but now it doesn't even compile anymore. The
> error message I get is the following:

This is one of the three reasons why I say it is better to do
tinerbox-type testing inside a VM. If you could make the precise QEMU
MIPS target image that fails to build QEMU available online then it is
easier for everyone to see the context. (The other two reasons are that
it is a good workout for the simulator, and general dogfood-ness for the
developers who are forced to become aware of problems that would
otherwise only be noticed by minority users such as you :-)

> I will try to get SPARC Linux and Solaris running on my newly acquired Sun
> Enterprise 250 server and compile there as well so I can test snapshots on
> all the architectures I have at my disposal.

You do also have the architectures provided to you by QEMU :-) :-)

-- 
Dan Shearer
dan@shearer.org

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Sunil Amitkumar Janki]

Dan Shearer wrote:
> You do also have the architectures provided to you by QEMU :-) :-)
>   

I have been able to build modular Xorg for Armedslack in QEMU without
having the hardware but when I tried to port Slackware 12.0 to SPARC in
QEMU I found it was way too slow and started looking for some real hardware.

Maybe in the future where we have all kinds of cheap multicore processors
this will not matter much but for the moment limited resources are still 
some
kind of hindrance to realizing the benefits of multi-architecture emulation.

Maybe I should trade in my single-core Athlon system for some serious
quadcore x86_64 chip but I'd rather wait for 16-core Loongson 3 :-).

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Dan Shearer]

On Fri, Jul 27, 2007 at 05:50:05PM +0200, Sunil Amitkumar Janki wrote:
> Dan Shearer wrote:
> >You do also have the architectures provided to you by QEMU :-) :-)
> 
> Maybe in the future where we have all kinds of cheap multicore processors
> this will not matter much but for the moment limited resources are still 
> some
> kind of hindrance to realizing the benefits of multi-architecture emulation.

In the special case of repeat automated test compiles where the only
output is a short status message from time to time, the lack of speed is
less of a problem. You do end up with a machine that creates the maximum
amount of noise the hardware is capable of, and runs very hot :-)

-- 
Dan Shearer
dan@shearer.org

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Jan Marten Simons]

Sunil Amitkumar Janki schrieb:
> Dan Shearer wrote:
>> You do also have the architectures provided to you by QEMU :-) :-)
>>   
> 
> I have been able to build modular Xorg for Armedslack in QEMU without
> having the hardware but when I tried to port Slackware 12.0 to SPARC in
> QEMU I found it was way too slow and started looking for some real 
> hardware.
> 
> Maybe in the future where we have all kinds of cheap multicore processors
> this will not matter much but for the moment limited resources are still 
> some
> kind of hindrance to realizing the benefits of multi-architecture 
> emulation.
> 
> Maybe I should trade in my single-core Athlon system for some serious
> quadcore x86_64 chip but I'd rather wait for 16-core Loongson 3 :-).
> 

As Qemu cannot use multicore CPUs (partially due to missing thread 
safety), yet, you won't benefit from this unless you want to run 
multiple instances of Qemu in parallel.

regs,
Jan

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Thiemo Seufer]

Sunil Amitkumar Janki wrote:
[snip]
>> gcc -g  -Wl,-T,/root/tmp/qemu-20070727/mipsel.ld   -o qemu-i386 main.o 
>> syscall.o mmap.o signal.o path.o osdep.o thunk.o elfload.o linuxload.o 
>> vm86.o libqemu.a gdbstub.o   -lm -lrt -lasound
>> /usr/bin/ld: cannot find -lm
>> collect2: ld returned 1 exit status
>> make[1]: *** [qemu-i386] Error 1

A missing libm sounds like a build environment problem.


Thiemo

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Sunil Amitkumar Janki]

Thiemo Seufer wrote:
> A missing libm sounds like a build environment problem.
>
> Thiemo

Do you have a test for this because I reinstalled the entire
system from scratch and have had no problems building
lots of packages over the last week except for those
applications that assume that all the world is an x86?

The only thing I have had to do is getting a GCC 3.4.6
compiler setup which I did by configuring and packaging
it to  "/opt/gcc/3.4.6" and including it in the path and
making it the default using a script in "/usr/bin" and
reversing that again.

I *do* have it configured with "--with-arch=r4600", which
may have led to problems. It's really painful for me to
go back to some hacked up previous os release, that was
not done by me, on which I can rebuild GCC 3.4.6 without
this option, but I will if I really have to.

It works well for several packages that don't compile with
GCC 4.1.2, which is the system compiler and seems to
be the accepted default nowadays in all distributions.

I guess the best solution would be to change QEMU to
make it build with GCC4 but I understand that may not
be an easy endeavour to undertake because of fundamental
design issues.

Sunil

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Thiemo Seufer]

Sunil Amitkumar Janki wrote:
> Thiemo Seufer wrote:
>> A missing libm sounds like a build environment problem.
>>
>> Thiemo
>
> Do you have a test for this because I reinstalled the entire
> system from scratch and have had no problems building
> lots of packages over the last week except for those
> applications that assume that all the world is an x86?
>
> The only thing I have had to do is getting a GCC 3.4.6
> compiler setup which I did by configuring and packaging
> it to  "/opt/gcc/3.4.6" and including it in the path and
> making it the default using a script in "/usr/bin" and
> reversing that again.

The problem is the linker (presumably /usr/bin/ld) does not
find the math library (likely name is /lib/libm.so.6). You can
add "-v" to the gcc command line to find out what it looks for.

JFTR, building qemu on Debian mips worked for me some days ago,
but the resulting binary was broken (as expected with current CVS).

> I *do* have it configured with "--with-arch=r4600", which
> may have led to problems. It's really painful for me to
> go back to some hacked up previous os release, that was
> not done by me, on which I can rebuild GCC 3.4.6 without
> this option, but I will if I really have to.

A plain upstream tarball and the classic
"./configure && make && make install"
should work for that.


Thiemo

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Sunil Amitkumar Janki]

Thiemo Seufer wrote:
> The problem is the linker (presumably /usr/bin/ld) does not
> find the math library (likely name is /lib/libm.so.6). You can
> add "-v" to the gcc command line to find out what it looks for.
>
> JFTR, building qemu on Debian mips worked for me some days ago,
> but the resulting binary was broken (as expected with current CVS).
>   

I have rebuilt in a pure Slackware 12.0 chroot in which I have replaced the
GCC 4.1.2 system compiler that runs from /usr/bin  with a previously 
built GCC
3.4.6 that also runs from /usr/bin and I agree that it builds this way. 
I will keep
building in this chroot then for the foreseeable future until QEMU is 
ready for
GCC 4.

Do you have any idea of what else is needed to get QEMU to a usable state on
MIPS hosts? Are there any other ideas for virtualisation such as Xen or 
VMware/
SimOS? I have asked on the Xen-devel list but nobody gave any feedback 
there.

[Qemu-devel] Re: QEMU Automated Testing

[Stefan Weil]

QEMU can build and run on a MIPS32 host - it just needs a few patches
(which were already sent to the list but got lost in CVS head).

The patched sources I use to run QEMU MIPS Malta with Debian stable
inside of a QEMU Malta emulation on x86 hardware are available from URL
http://svn.berlios.de/viewcvs/ar7-firmware/qemu/branches/head/

Use this command to get them:
svn checkout
http://svn.berlios.de/svnroot/repos/ar7-firmware/qemu/branches/head/

Any combination of big and little endian works with pcnet32:
qemu-system-mipsel in qemu-system-mips,
qemu-system-mipsel in qemu-system-mipsel,
qemu-system-mips in qemu-system-mips,
qemu-system-mips in qemu-system-mipsel.

Other network emulations only work with some of these combinations.

User mode emulation won't work, but it does not work on any
platform - not even x86.

I did not check MIPS64 emulation.

Stefan

> Do you have any idea of what else is needed to get QEMU to a usable
> state on
> MIPS hosts? Are there any other ideas for virtualisation such as Xen
> or VMware/
> SimOS? I have asked on the Xen-devel list but nobody gave any feedback
> there.

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Andreas Färber]

Am 27.07.2007 um 16:58 schrieb Sunil Amitkumar Janki:

> Dan Shearer wrote:
>> I see this as a different class of testing
>> [...], checking out and compiling
>> under different circumstances. That's very valuable, and will help
>> address a problem that drives away a lot of potential QEMU users -  
>> they
>> just can't compile it.
>
> That would be great. I am regularly compiling and using QEMU CVS
> snapshots because features get added all the time. A lot of the time
> it happens that these build fine on x86 whereas they don't work on
> other architectures because the developers don't have access to those.
>
> I have been running self-built Slackware 12.0 on MIPS for the last
> few months and would like to run QEMU on that architecture. previously
> it built but didn't run, but now it doesn't even compile anymore. The
> error message I get is the following: [...]

I recently reported build failure of the ppc target on OS X due to an  
unsupported GCC feature but no-one has shown any interest in fixing  
it, neither applying my workaround patch nor proposing a different  
solution.

It's a strange phenomenon that many other open source projects don't  
suffer - qemu seems to expect everyone to patch it herself instead of  
fixing problems in the repository at their origin. Major example  
being qemu clinging on to GCC 3.x instead of supporting both 3 and 4  
by updating and integrating the existing patches that worked for 0.9.0.

Andreas

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Thiemo Seufer]

Andreas Färber wrote:
>
> Am 27.07.2007 um 16:58 schrieb Sunil Amitkumar Janki:
>
>> Dan Shearer wrote:
>>> I see this as a different class of testing
>>> [...], checking out and compiling
>>> under different circumstances. That's very valuable, and will help
>>> address a problem that drives away a lot of potential QEMU users - they
>>> just can't compile it.
>>
>> That would be great. I am regularly compiling and using QEMU CVS
>> snapshots because features get added all the time. A lot of the time
>> it happens that these build fine on x86 whereas they don't work on
>> other architectures because the developers don't have access to those.
>>
>> I have been running self-built Slackware 12.0 on MIPS for the last
>> few months and would like to run QEMU on that architecture. previously
>> it built but didn't run, but now it doesn't even compile anymore. The
>> error message I get is the following: [...]
>
> I recently reported build failure of the ppc target on OS X due to an 
> unsupported GCC feature but no-one has shown any interest in fixing it, 
> neither applying my workaround patch nor proposing a different solution.
>
> It's a strange phenomenon that many other open source projects don't suffer 
> - qemu seems to expect everyone to patch it herself instead of fixing 
> problems in the repository at their origin. Major example being qemu 
> clinging on to GCC 3.x instead of supporting both 3 and 4 by updating and 
> integrating the existing patches that worked for 0.9.0.

The patches I remember did either break other compilers or were a
maintenance nightmare of postprocessed assembler output.


Thiemo

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Andreas Färber]

Am 28.07.2007 um 12:36 schrieb Thiemo Seufer:

>> qemu seems to expect everyone to patch it herself instead of fixing
>> problems in the repository at their origin. Major example being qemu
>> clinging on to GCC 3.x instead of supporting both 3 and 4 by  
>> updating and
>> integrating the existing patches that worked for 0.9.0.
>
> The patches I remember did either break other compilers or were a
> maintenance nightmare of postprocessed assembler output.

I didn't say the particular patch was perfect, I can't tell. All I  
know is that there are platforms that don't have a GCC 3.x (the Intel  
Macs for instance), and syncing the existing 0.9.0 patches with CVS  
HEAD is like shooting for a running target - I didn't manage to  
successfully update the patch, it crashed badly. Having some initial,  
conditional support for GCC 4 (e.g. --enable-gcc4 or depending on $CC  
version) would be good. Do you know where exactly the problem is  
between the versions and what needs to be done to have it working  
(again)?

Andreas

Re: IRQ handling (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Jamie Lokier]

J. Mayer wrote:
> > No. Since MIPS{32,64}R2 the CP0 is standardized and a mandatory part of
> > a MIPS compatible CPU.
> 
> Yes, I know MIPS want always CP0 to be present. I should have put a
> smiley somewhere. I wanted just to point the fact that the CPU itself
> does not need the CP0 controller to run and that one could easily
> imagine designing a core without the CP0 controller.

I just want to say I have programmed a MIPS-ish core which did not
have a CP0 controller.  It was the Alteon AceNIC gigabit ethernet
controller, and it had two CPU cores implementing the MIPS integer
instruction set.  We used the standard mips-elf GCC to compile for it.

I'm under the impression that copying the basic MIPS instruction set
is not uncommon for the odd ASIC here and there, as it doesn't bump
into the licensing problems that copying ARM instructions does, and
it's quite simple.

-- Jamie

[Qemu-devel] Re: IRQ handling

[Paul Brook]

[replying to a couple of different mails]

> What do you need to route an IRQ ?
> -> A peripheral destination

Agreed.

> What we got now ?
> -> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
> state

We have this in some places. Other places only have some parts.

> Is more needed to have a generic routing approach ?
> -> no. This can work to route any signal

Agreed.

> Can we do with less ?
> -> no. You need the 3 parameters.

Agreed.


In summary the IRQ source (ie. device raising the IRQ) needs to keep track of 
4 values:
1) Callback
2) Opaque callback argument
3) PIN number
4) IRQ state.

In most cases we get (4) for free because it's a direct function of device 
state, so I'll ignore it for now.

I believe (1) and (2) are inherently linked, and it makes no sense to 
set/change them individually.

In some cases code has multiple instances of (3) sharing a single (2). This 
is, in general, incorrect as a device may have outputs connected to different 
interrupt controllers. I think there are examples of this on the ARM boards.

Thus an IRQ source can treat (1), (2) and (3) as a single block of 
information, with no loss of flexibility.

> The problem is also: what does this patch adds, but complexity 

I believe my patch concentrates the (necessary) complexity in a single place. 
For the record, the net effect of my patch was to remove approximately 32 
lines of code (71 files changed, 594 insertions(+), 626 deletions(-))

> and arbitrary limitations ?

You have stated several times that my patch adds arbitrary limitations.
I refute this assertion absolutely.

There are no limits on the number of IRQs, or the topology of entities 
(devices, interrupt handlers, and CPUs) that can be supported. 

Hotplugging is not a problem, neither are systems with thousands of IRQs. I 
have local patches for an arm-based core with several hundred IRQ lines. 
Convention is that IRQ objects are created at the same time as the rest of 
the device state for the IRQ sink (ie. interrupt controller or CPU).

In practice this means a particular device needs to know how many IRQ inputs 
it has when it is instantiated. I believe this is entirely reasonable. Last 
time I checked it wasn't feasible to dynamically solder new pins onto an IC 
while it was running [*1]. Note that this is the number of IRQs*per instance 
of a device. It's entirely possible to have different instances of the "same" 
device with arbitrarily different numbers of IRQs, and an arbitrary number of 
devices/IRQs in a system. [*2]

If you want to do anything other that simple 1-1 connections (eg. shared IRQ 
lines) you can create a fake device to perform the appropriate multiplexing. 
This is what the PCI code does. It creates a "PCI bus" interrupt controller 
that maps individual device IRQ pins onto the host interface IRQs.

> What can be done (but once again, it changes nothing, just hide the
> "complexity"), is to replace the  { callback, n_IRQ } in devices
> structure by a IRQState structure and have the inline functions.

This is what I did, except I chose to make it an opaque structure, to prevent 
devices for meddling with it directly. I'd be amazed if inlining qemu_set_irq 
made any measurable difference to execution speed.

You seem to be saying that making this change has no benefit. I disagree quite 
strongly. 

Having each device keep track of 3 values (callback, opaque and nIRQ; see 
earlier) is a real mess, evidenced by the fact that devices don't do this 
consistently, the PCI code has grown its own slightly different mechanisms 
for signalling IRQs, and the ARM boards had their own partially generic 
implementation. Adding simple and consistent infrastructure for signalling 
interrupts is IMHO a worthwhile change in its own right.

>..
> To achieve this, you have to have a structure:
> struct PINState {
>     qemu_pin_handler handler;
>     void *opaque;
>     int n;
>     int level;
> };

Yes, and the existing code can be extended to implement this without wasting 
any of the current changes.

What you're talking about (and in later emails with tristate pins) is a 
generic mechanism for emulating single-bit buses. I don't claim that my 
implementation can do this as-is.

I have implemented sufficient infrastructure for a single-master single-slave 
bus. The most common example of which is a IRQ line. I believe it also covers 
a usefully large subset GPIO pin uses.

I say that my changes are a necessary first step in implementing a fully 
generic single-bit bus framework. My implementation adds infrastructure and 
abstraction for the "master" device (IRQ source), while leaving the "slave" 
(IRQ sink) device code largely unchanged.

Paul

[*1] I guess you could theoretically do this with a self-modifying FPGA SoC. 
It's not impossible to model, just a bit hairy. Effectively an extreme case of 
hotplugging.

[*2] Technically you're limited by available memory on the host. However 
struct IRQState is very small, so you have other much larger problems before 
you even come close to that limit.

Re: [Qemu-devel] Re: IRQ handling

[J. Mayer]

On Mon, 2007-04-09 at 01:41 +0100, Paul Brook wrote:
> [replying to a couple of different mails]
> 
> > What do you need to route an IRQ ?
> > -> A peripheral destination
> 
> Agreed.
> 
> > What we got now ?
> > -> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
> > state
> 
> We have this in some places. Other places only have some parts.

So, here was the first point to fix. Imho, no need to change all the
code for this.

> > Is more needed to have a generic routing approach ?
> > -> no. This can work to route any signal
> 
> Agreed.
> 
> > Can we do with less ?
> > -> no. You need the 3 parameters.
> 
> Agreed.
> 
> 
> In summary the IRQ source (ie. device raising the IRQ) needs to keep track of 
> 4 values:
> 1) Callback
> 2) Opaque callback argument
> 3) PIN number
> 4) IRQ state.
> 
> In most cases we get (4) for free because it's a direct function of device 
> state, so I'll ignore it for now.

I don't agree. There are a lot of cases where the hardware state is not
equivalent to the device internal state. If an IRQ line is not driven,
then you cannot predict what the hardware state is, from the source. It
seems to me this is the most usual case for IRQs lines.

> I believe (1) and (2) are inherently linked, and it makes no sense to 
> set/change them individually.

OK.

> In some cases code has multiple instances of (3) sharing a single (2). This 
> is, in general, incorrect as a device may have outputs connected to different 
> interrupt controllers. I think there are examples of this on the ARM boards.
> 
> Thus an IRQ source can treat (1), (2) and (3) as a single block of 
> information, with no loss of flexibility.
> 
> > The problem is also: what does this patch adds, but complexity 
> 
> I believe my patch concentrates the (necessary) complexity in a single place. 
> For the record, the net effect of my patch was to remove approximately 32 
> lines of code (71 files changed, 594 insertions(+), 626 deletions(-))
> 
> > and arbitrary limitations ?
> 
> You have stated several times that my patch adds arbitrary limitations.
> I refute this assertion absolutely.
> 
> There are no limits on the number of IRQs, or the topology of entities 
> (devices, interrupt handlers, and CPUs) that can be supported. 

OK, then I'm wrong on this point.

> Hotplugging is not a problem, neither are systems with thousands of IRQs. I 
> have local patches for an arm-based core with several hundred IRQ lines. 
> Convention is that IRQ objects are created at the same time as the rest of 
> the device state for the IRQ sink (ie. interrupt controller or CPU).
> 
> In practice this means a particular device needs to know how many IRQ inputs 
> it has when it is instantiated. I believe this is entirely reasonable. Last 
> time I checked it wasn't feasible to dynamically solder new pins onto an IC 
> while it was running [*1].

Quite hard to achieve, I got to admit it.

>  Note that this is the number of IRQs*per instance 
> of a device. It's entirely possible to have different instances of the "same" 
> device with arbitrarily different numbers of IRQs, and an arbitrary number of 
> devices/IRQs in a system. [*2]
> 
> If you want to do anything other that simple 1-1 connections (eg. shared IRQ 
> lines) you can create a fake device to perform the appropriate multiplexing. 
> This is what the PCI code does. It creates a "PCI bus" interrupt controller 
> that maps individual device IRQ pins onto the host interface IRQs.

So, this does not change anything from the previous situation.

> > What can be done (but once again, it changes nothing, just hide the
> > "complexity"), is to replace the  { callback, n_IRQ } in devices
> > structure by a IRQState structure and have the inline functions.
> 
> This is what I did, except I chose to make it an opaque structure, to prevent 
> devices for meddling with it directly. I'd be amazed if inlining qemu_set_irq 
> made any measurable difference to execution speed.

> You seem to be saying that making this change has no benefit. I disagree quite 
> strongly. 
> 
> Having each device keep track of 3 values (callback, opaque and nIRQ; see 
> earlier) is a real mess, evidenced by the fact that devices don't do this 
> consistently, the PCI code has grown its own slightly different mechanisms 
> for signalling IRQs, and the ARM boards had their own partially generic 
> implementation. Adding simple and consistent infrastructure for signalling 
> interrupts is IMHO a worthwhile change in its own right.

So, you wanted just to hide all the mess in an opaque ?
But I still cannot see what functionality this could add, how this could
make SMP feasible, for example (I do think there was no problem for this
before), ...

> >..
> > To achieve this, you have to have a structure:
> > struct PINState {
> >     qemu_pin_handler handler;
> >     void *opaque;
> >     int n;
> >     int level;
> > };
> 
> Yes, and the existing code can be extended to implement this without wasting 
> any of the current changes.
> 
> What you're talking about (and in later emails with tristate pins) is a 
> generic mechanism for emulating single-bit buses. I don't claim that my 
> implementation can do this as-is.
> 
> I have implemented sufficient infrastructure for a single-master single-slave 
> bus. The most common example of which is a IRQ line. I believe it also covers 
> a usefully large subset GPIO pin uses.

I don't think so. For most hardware, you need to emulate the actual
hardware behavior if you want to emulate what happens on the GPIO pins.
And I maintain you need to read back the hardware state if you want to
know the logical level of an IO. Think about implementing IIC,
SPI,  .... using GPIO, which is not efficient but quite a usual
situation. You absolutely need to handle multiple sources - multiple
destinations cases to achieve this. And knowing the state of an IO
source is not sufficient to know the state of the IO line.
Even for IRQ, I think it should be done: this would avoid the kind of
local mess we have to handle PCI shared IRQ (I just did look the PREP
PCI 1.2 patch to say this, it may be done in a better way for other PCI
controllers).  And PCI IRQ is exactly a case where you cannot read back
the IRQ line state from the state of the IRQ source. And it's not rare
that an IRQ source has a register where you can read back the hardware
state of its IRQ line(s).
And you also have many cases you need to know the level of an IO at some
precise points but you do not want to track all the IO changes (the IO
line may be floating most of the time, for example), then not define any
callback but just read the actual IO value when you need to know it.

> I say that my changes are a necessary first step in implementing a fully 
> generic single-bit bus framework. My implementation adds infrastructure and 
> abstraction for the "master" device (IRQ source), while leaving the "slave" 
> (IRQ sink) device code largely unchanged.

I will try to sanitize the openpic, as you did suggest. I have to admit
what I did is not so good as I initially though it was.
As I already told here, I have added a fake internal IRQ controller in
the PowerPC core just to use the same API (the SetIRQ callback / nIRQ /
level) as the rest of the code to easily plug other IRQ controllers to a
PowerPC core. This fake IRQ controller is not in the PowerPC
specification and does not have any counterpart in actual
implementations:
"While the specific definition of an external input exception is
implementation dependant, it would typically be caused by the activation
of an asynchronous signal that is part of the processing system".
The same definition is used for other exception sources (critical
interrupt, ...) not generated by the PowerPC core itself. One should
take care that the word "interrupt" in the PowerPC specification is not
related to hardware IRQ (IBM likes to use standard vocabulary with
different meanings), which is quite confusing: "an interrupt is the
action in which the processor saves its context and begins execution at
a pre-determined interrupt-handler address. Exception are the events
that will, if enabled, cause the processor to take an interrupt".
This fake interrupt controller is just an IRQ sink, not a source, so I
maintain the presence of IRQ structure in the PowerPC CPU structure is
not to be. In addition of using the API of an IRQ controller, it also
cover a problem in the CPU emulation which is Qemu does not handle
exception queues and/or has not any "clean" support for multiple
exception sources raising at the same time. As suggested by Fabrice,
instead of adding mess in the Qemu core code, as I needed to add more
asynchronous exception sources, I decided to hide this using a bitmask
in the CPU structure, which seemed to be cleaner to me.

> [*1] I guess you could theoretically do this with a self-modifying FPGA SoC. 
> It's not impossible to model, just a bit hairy. Effectively an extreme case of 
> hotplugging.

Quite extreme, true.

> [*2] Technically you're limited by available memory on the host. However 
> struct IRQState is very small, so you have other much larger problems before 
> you even come close to that limit.
;-)

-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: [Qemu-devel] Re: IRQ handling

[Paul Brook]

> > In summary the IRQ source (ie. device raising the IRQ) needs to keep
> > track of 4 values:
> > 1) Callback
> > 2) Opaque callback argument
> > 3) PIN number
> > 4) IRQ state.
>...
> > I believe (1) and (2) are inherently linked, and it makes no sense to
> > set/change them individually.
>
> OK.
>
> > Having each device keep track of 3 values (callback, opaque and nIRQ; see
> > earlier) is a real mess, evidenced by the fact that devices don't do this
> > consistently, the PCI code has grown its own slightly different
> > mechanisms for signalling IRQs, and the ARM boards had their own
> > partially generic implementation. Adding simple and consistent
> > infrastructure for signalling interrupts is IMHO a worthwhile change in
> > its own right.
>
> So, you wanted just to hide all the mess in an opaque ?

Yes. As you agreed above, there's no need for the device to know the details.

> But I still cannot see what functionality this could add, how this could
> make SMP feasible, for example (I do think there was no problem for this
> before), ...

In its current implementation it doesn't provide any functional enhancements 
over a {callpack, opaque, nIRQ} triplet. What it does do is isolate the 
device (IRQ source) emulation from the implementation details.  As you agreed 
above, there's no need for the device to know these details.

It could be extended to do clever things like multiple sources, shared IRQ 
lines and tristate pins, but I have not done this. For the current code it is 
sufficient to model these cases as a virtual fixed-function IRQ controller.

> > I have implemented sufficient infrastructure for a single-master
> > single-slave bus. The most common example of which is a IRQ line. I
> > believe it also covers a usefully large subset GPIO pin uses.
>
> I don't think so. For most hardware, you need to emulate the actual
> hardware behavior if you want to emulate what happens on the GPIO pins.
> And I maintain you need to read back the hardware state if you want to
> know the logical level of an IO. Think about implementing IIC,
> SPI,  .... using GPIO, which is not efficient but quite a usual

We obviously have a different idea of what a "usefully large subset of GPIO 
pin uses" entails. Most of the hardware I'm familiar with has dedicated I2C 
hardware, and uses GPIO pins for interrupts and simple signals (eg. card 
present).

I agree that in some cases you may want to model eg. an I2C bus connected to a 
tristate GPIO pin. I don't have a good solution worked out for that. However 
I am fairly confident that the current infrastructure could be used as a base 
for whatever we do come up with.

>  I have added a fake internal IRQ controller in
> the PowerPC core just to use the same API (the SetIRQ callback / nIRQ /
> level) as the rest of the code to easily plug other IRQ controllers to a
> PowerPC core.
>...
> As suggested by Fabrice,
> instead of adding mess in the Qemu core code, as I needed to add more
> asynchronous exception sources, I decided to hide this using a bitmask
> in the CPU structure, which seemed to be cleaner to me.

Right, This is that same as what MIPS does, and probably ARM will soon.

All I did was to change the "same API" to be abstract object instead of 
messing about with pointless triplets of information. For PPC this conversion 
is incomplete because I don't understand the details well enough to tell how 
bogus the current code is. eg. the openpic emulation looks like it is capable 
of raising several outputs, but only one is ever used.

Paul

Re: [Qemu-devel] Re: IRQ handling

[J. Mayer]

On Mon, 2007-04-09 at 14:58 +0100, Paul Brook wrote:
[...]

> > > Having each device keep track of 3 values (callback, opaque and nIRQ; see
> > > earlier) is a real mess, evidenced by the fact that devices don't do this
> > > consistently, the PCI code has grown its own slightly different
> > > mechanisms for signalling IRQs, and the ARM boards had their own
> > > partially generic implementation. Adding simple and consistent
> > > infrastructure for signalling interrupts is IMHO a worthwhile change in
> > > its own right.
> >
> > So, you wanted just to hide all the mess in an opaque ?
> 
> Yes. As you agreed above, there's no need for the device to know the details.
> 
> > But I still cannot see what functionality this could add, how this could
> > make SMP feasible, for example (I do think there was no problem for this
> > before), ...
> 
> In its current implementation it doesn't provide any functional enhancements 
> over a {callpack, opaque, nIRQ} triplet. What it does do is isolate the 
> device (IRQ source) emulation from the implementation details.  As you agreed 
> above, there's no need for the device to know these details.

So, I was confused by some remarks saying that this would give new
features.

> It could be extended to do clever things like multiple sources, shared IRQ 
> lines and tristate pins, but I have not done this. For the current code it is 
> sufficient to model these cases as a virtual fixed-function IRQ controller.
> 
> > > I have implemented sufficient infrastructure for a single-master
> > > single-slave bus. The most common example of which is a IRQ line. I
> > > believe it also covers a usefully large subset GPIO pin uses.
> >
> > I don't think so. For most hardware, you need to emulate the actual
> > hardware behavior if you want to emulate what happens on the GPIO pins.
> > And I maintain you need to read back the hardware state if you want to
> > know the logical level of an IO. Think about implementing IIC,
> > SPI,  .... using GPIO, which is not efficient but quite a usual
> 
> We obviously have a different idea of what a "usefully large subset of GPIO 
> pin uses" entails. Most of the hardware I'm familiar with has dedicated I2C 
> hardware, and uses GPIO pins for interrupts and simple signals (eg. card 
> present).

I agree a lot of embedded micro-controllers implement hardware I2C but I
also know a lot that lacks support for some specialised busses, like
SPI, then force developpers to emulate those interfaces using GPIOs, or
share some IO pins for multiple functions.

> I agree that in some cases you may want to model eg. an I2C bus connected to a 
> tristate GPIO pin. I don't have a good solution worked out for that. However 
> I am fairly confident that the current infrastructure could be used as a base 
> for whatever we do come up with.
> 
> >  I have added a fake internal IRQ controller in
> > the PowerPC core just to use the same API (the SetIRQ callback / nIRQ /
> > level) as the rest of the code to easily plug other IRQ controllers to a
> > PowerPC core.
> >...
> > As suggested by Fabrice,
> > instead of adding mess in the Qemu core code, as I needed to add more
> > asynchronous exception sources, I decided to hide this using a bitmask
> > in the CPU structure, which seemed to be cleaner to me.
> 
> Right, This is that same as what MIPS does, and probably ARM will soon.
> 
> All I did was to change the "same API" to be abstract object instead of 
> messing about with pointless triplets of information. For PPC this conversion 
> is incomplete because I don't understand the details well enough to tell how 
> bogus the current code is. eg. the openpic emulation looks like it is capable 
> of raising several outputs, but only one is ever used.

Yes, the implementation is incomplete. I need to use more than one
outputs for embedded PowerPC emulation but the code is not finished.
I just started to rework all this code. If I want to follow your new
scheme, I have to add an emulation for the implementation dependant
hardware internal IRQ controller (fortunately, most usual PowerPC
implementations use the same scheme) and try to change the OpenPIC so it
can easily map its outputs pins to the input pins of the PowerPC, in a
machine dependant way.

-- 
J. Mayer <l_indien@magic.fr>
Never organized

Re: [Qemu-devel] Re: IRQ handling

[Paul Brook]

> > In its current implementation it doesn't provide any functional
> > enhancements over a {callpack, opaque, nIRQ} triplet. What it does do is
> > isolate the device (IRQ source) emulation from the implementation
> > details.
>
> So, I was confused by some remarks saying that this would give new
> features.

I probably meant functionality improvements to the devices that didn't 
previously implement the full triplet. Sorry for any confusion.

> Yes, the implementation is incomplete. I need to use more than one
> outputs for embedded PowerPC emulation but the code is not finished.
> I just started to rework all this code. If I want to follow your new
> scheme, I have to add an emulation for the implementation dependant
> hardware internal IRQ controller (fortunately, most usual PowerPC
> implementations use the same scheme) and try to change the OpenPIC so it
> can easily map its outputs pins to the input pins of the PowerPC, in a
> machine dependant way.

That is what I would recommend doing.

Paul

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Fabrice Bellard]

Hi,

> I can only fix the code that I have access to.
> 
> IMHO your request that I revert changes because they allegedly break code that 
> noone else has access to is unreasonable. My changes are a significant 
> cleanup to qemu internals.
> 
> However if there is agreement from other maintainers I will revert my patch.

I don't think you need to revert your patch. Personally, I would have 
changed the following:

1) The code is more generic than IRQs, it can be applied to any logic 
signal, so the naming should show it.

2) I don't like hiding pointers in types, so I would use for example 
'QEMUSignal *' instead of 'qemu_irq'.

Regarding the PPC problem, maybe you could just revert the patches in 
the MIPS and PowerPC CPUs structures, as I don't think it is necessary 
to move the IRQ code at this level now.

BTW, I have a small PREP test image that I can publish.

I am planning since some time to add to the QEMU web site an automatic 
regression testing system which will automatically launch many OSes with 
the current CVS version. I had no time to do it recently, but if no one 
does it before I'll try to complete it !

Regards,

Fabrice.

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Wang Cheng Yeh]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

>
> BTW, I have a small PREP test image that I can publish.
>
> I am planning since some time to add to the QEMU web site an automatic
> regression testing system which will automatically launch many OSes with
> the current CVS version. I had no time to do it recently, but if no one
> does it before I'll try to complete it !
>
> Regards,
>
> Fabrice.
>
>
> I think the installation will cover more than just kernel booting.
I have debian auto installation files for regression test.
They are just a auto config file and a script to do regression test.
The only problem is if the system hangs, the script will also hang.
If anyone needs these files, I will release here.
The installation just follows  the URL
http://www.aurel32.net/info/debian_arm_qemu.php

[-- Attachment #2: Type: text/html, Size: 1035 bytes --]

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Thiemo Seufer]

Wang Cheng Yeh wrote:
> >
> >BTW, I have a small PREP test image that I can publish.
> >
> >I am planning since some time to add to the QEMU web site an automatic
> >regression testing system which will automatically launch many OSes with
> >the current CVS version. I had no time to do it recently, but if no one
> >does it before I'll try to complete it !
> >
> >Regards,
> >
> >Fabrice.
> >
> >
> >I think the installation will cover more than just kernel booting.
> I have debian auto installation files for regression test.
> They are just a auto config file and a script to do regression test.
> The only problem is if the system hangs, the script will also hang.
> If anyone needs these files, I will release here.
> The installation just follows  the URL
> http://www.aurel32.net/info/debian_arm_qemu.php

Maybe the debian-installer regression testing is helpful as well. It uses
qemu for some of its configs.

http://people.debian.org/~joeyh/d-i/test-logs.html is the results page,
http://d-i.alioth.debian.org/svn/debian-installer/scripts/digress/ points
to the source of the test harness.


Thiemo

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Paul Brook]

> 2) I don't like hiding pointers in types, so I would use for example
> 'QEMUSignal *' instead of 'qemu_irq'.

FWIW I did it this way so that devices are agnostic about whether it was 
implemented as a pointer or an integer index.

Paul

Re: [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...

[Thiemo Seufer]

J. Mayer wrote:
> On Sat, 2007-04-07 at 20:10 +0100, Paul Brook wrote:
> > On Saturday 07 April 2007 19:32, J. Mayer wrote:
> > > On Sat, 2007-04-07 at 18:14 +0000, Paul Brook wrote:
> > > > CVSROOT:	/sources/qemu
> > > > Module name:	qemu
> > > > Changes by:	Paul Brook <pbrook>	07/04/07 18:14:41
> > >
> > > The patches in the PowerPC target seem complete nonsense.
> > 
> > Can you give specific examples?
> 
> I'm talking about the CPU code.
> There is NO notion of external IRQ allocation in the PowerPC
> specification.
> IRQ  are completely out of the scope of the CPU emulation so the table
> of 32 void *IRQ pointers is the CPU structure is a complete nonsense.
> Where do you see in the PowerPC specification that those CPU have any
> notion of how the EXTERNAL IRQ controler works ? Where do you see that a
> machine with a PowerPC cannot manage more than 32 IRQ ? Where do you see
> ANY NOTION OF EXTERNAL IRQ MANAGEMENT in the PowerPC specification ?
> EXTERNAL IRQ MANAGEMENT IS NO WAY RELATED WITH CPU ! It's private to
> each IRQ controller. 
> Saying anything else is just completely ignoring how real hardware
> works.
> SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
> HAS TO BE REVERTED.

So you are saying the old code is similiarily flawed? Why would reverting
to the old version be an improvement then?

> If you don't, I'LL REVERT ALL POWERPC CODE AFFECTED BY THIS PATCH.

I assume you are close to have working code which solves the aforementioned
problems. In that case it might make most sense to stick with the current
version for the moment and replace it with yours as soon as it works.

> > The CHRP code looks a bit broken, but no more so than before I started.
> > 
> > > Furthermore, this kind of patch that break other guys work would likely
> > > to be discussed and not beeing imposed.
> > 
> > It's been mentioned several times on this list (by Fabrice, specifically) that 
> > this is the way to go.
> 
> I did not received any single mail AT ALL saying "we're going to break
> your code, you can now throw away all the work you're doing". NOT ONE.
> So don't say "it's been discussed several times".

It is a bit hard to figure that out when the existence of such code isn't
known. Btw, I fail to see why it amounts to "throw away all work".
Presumably you could "revert" the offending bits in you local copy and
merge the remaining interface changes.


Thiemo

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[n schembr]

----- Original Message ----
From: Jan Marten Simons <marten@xtal.rwth-aachen.de>
To: qemu-devel@nongnu.org
Sent: Friday, July 27, 2007 12:50:51 PM
Subject: Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

Sunil Amitkumar Janki schrieb:
> Dan Shearer wrote:
>> You do also have the architectures provided to you by QEMU :-) :-)
>>   
> 
> I have been able to build modular Xorg for Armedslack in QEMU without
> having the hardware but when I tried to port Slackware 12.0 to SPARC in
> QEMU I found it was way too slow and started looking for some real 
> hardware.
> 
> Maybe in the future where we have all kinds of cheap multicore processors
> this will not matter much but for the moment limited resources are still 
> some
> kind of hindrance to realizing the benefits of multi-architecture 
> emulation.
> 
> Maybe I should trade in my single-core Athlon system for some serious
> quadcore x86_64 chip but I'd rather wait for 16-core Loongson 3 :-).
> 

>As Qemu cannot use multicore CPUs (partially due to missing thread 
>safety), yet, you won't benefit from this unless you want to run 
>multiple instances of Qemu in parallel.

>regs,
>Jan

how close is thread safey?

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Paul Brook]

> >As Qemu cannot use multicore CPUs (partially due to missing thread
> >safety), yet, you won't benefit from this unless you want to run
> >multiple instances of Qemu in parallel.
>
> how close is thread safey?

In a useful form: a fair way off.

It's relatively simple to hack something together than runs. Making it work 
correctly and go fast is much harder though. My current prototype (running on 
2 cores) runs about a quarter the speed of normal qemu, and dies shortly 
after booting because the guest atomic synchronisation primitives don't work 
right.

Either way, for a tinderbox or automated testing yo *do* have lots of 
different tests you can run in parallel, so it doesn't matter that each qemu 
instance only uses one core. We're only talking about small SMP here - maybe 
16 cores, but not thousands.

Paul

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Anthony Liguori]

Paul Brook wrote:
>>> As Qemu cannot use multicore CPUs (partially due to missing thread
>>> safety), yet, you won't benefit from this unless you want to run
>>> multiple instances of Qemu in parallel.
>>>       
>> how close is thread safey?
>>     
>
> In a useful form: a fair way off.
>
> It's relatively simple to hack something together than runs. Making it work 
> correctly and go fast is much harder though. My current prototype (running on 
> 2 cores) runs about a quarter the speed of normal qemu, and dies shortly 
> after booting because the guest atomic synchronisation primitives don't work 
> right.
>   

This later problem seems like the hardest to solve to me.  Did you have 
any ideas here that don't involve hand coding the translation for atomic 
instructions?

Regards,

Anthony Liguori

> Either way, for a tinderbox or automated testing yo *do* have lots of 
> different tests you can run in parallel, so it doesn't matter that each qemu 
> instance only uses one core. We're only talking about small SMP here - maybe 
> 16 cores, but not thousands.
>
> Paul
>
>
>
>   

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[Paul Brook]

> >> how close is thread safey?
> >
> > In a useful form: a fair way off.
> >
> > It's relatively simple to hack something together than runs. Making it
> > work correctly and go fast is much harder though. My current prototype
> > (running on 2 cores) runs about a quarter the speed of normal qemu, and
> > dies shortly after booting because the guest atomic synchronisation
> > primitives don't work right.
>
> This later problem seems like the hardest to solve to me.  Did you have
> any ideas here that don't involve hand coding the translation for atomic
> instructions?

Yes. I do exclusive access locking at the TLB level. i.e. creating a TLB entry 
for a writable page forces that page to be flushed from all the other CPU 
TLBs. 

There's some wiggle room in the definition of a "writable" page. If other 
critera are met it should be sufficient to just do this exclusion for atomic 
accesses.

If necessary the same technique can be used to avoid write ordering and 
coherency problems without having to accurately map guest barriers onto the 
equivalent host operations. This is handy when most of the guest barriers are 
implicit, e.g. when emulating a strictly ordered guest on a weakly ordered 
host.

In theory this could be taken to extremes and used to split emulation of a 
single machine over multiple address spaces/nodes.  In practice the 
contention on a normal SMP operating system is high enough that this is not 
practical.

Paul

Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

[n schembr]

----- Original Message ----
From: Paul Brook <paul@codesourcery.com>
To: qemu-devel@nongnu.org
Cc: n schembr <nschembr@yahoo.com>
Sent: Friday, July 27, 2007 1:38:19 PM
Subject: Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)

> >As Qemu cannot use multicore CPUs (partially due to missing thread
> >safety), yet, you won't benefit from this unless you want to run
> >multiple instances of Qemu in parallel.
>
> how close is thread safey?

In a useful form: a fair way off.

It's relatively simple to hack something together than runs. Making it work 
correctly and go fast is much harder though. My current prototype (running on 
2 cores) runs about a quarter the speed of normal qemu, and dies shortly 
after booting because the guest atomic synchronisation primitives don't work 
right.

Either way, for a tinderbox or automated testing yo *do* have lots of 
different tests you can run in parallel, so it doesn't matter that each qemu 
instance only uses one core. We're only talking about small SMP here - maybe 
16 cores, but not thousands.

Paul
-------------------------------------------------------------

Is the the issue I had with qemu moving from cpu to cpu?  taskset -c 1 qemu or notsc

http://lists.gnu.org/archive/html/qemu-devel/2007-03/msg00652.html  http://qemu-forum.ipi.fi/viewtopic.php?t=3714&highlight=taskset



What about preforking the processes one per virtual cpu?