RE: Emulating Solaris 10 on SPARC64 sun4u

[<jasper.lowell@bt.com>]

Excuse the delay. I believe the reason why I am unable to locate the error string "Interrupt not seen after set_features" in the OpenSolaris source code is because it belongs to a proprietary driver that was not distributed with OpenSolaris. Rather than rely on source code I've had to debug this problem by observing Solaris 10's behaviour.

I previously linked https://docs.oracle.com/cd/E23824_01/html/821-1475/uata-7d.html that seems to indicate that this error is fatal.
Considering that the CMD646 IDE controller driver experiences a fatal error during the bootstrapping of the system, I suspect that the file system on the CDROM might not be accessible.
I'm not sure if this is directly related to the unresponsive serial console but I wouldn't be surprised.

When configuring devices, Solaris 10 uses the SET_FEATURE command on the CMD646 to set the transfer mode to MDMA mode.
From what I can tell, this is successful and the emulated IDE controller raises an interrupt acknowledging that the command was completed successfully. To determine whether or not this interrupt was successfully propagated to Solaris 10, I made manual changes to ensure that the interrupt was not raised for this event at this specific time. This resulted in a new error from Solaris 10 regarding "set_features".
- Solaris 10 appears to be able to see the interrupt from the completion of the SET_FEATURE command.
- Solaris 10 appears to then perform two reads on the status register. From what I understand, this has the side effect of clearing interrupts.
- Solaris 10 then writes to the device/head register.
- Solaris 10 then spins on ARTTIM23_INTR_CH1 expecting it to be set. When it is not set, the operation times out and we are presented with the fatal error regarding set_features.

I am not intimately familiar with the workings of the CMD646 or the ATA specification so I can only speculate.
- If the interrupt that Solaris 10 expects is the one from the SET_FEATURE command, then Solaris 10 is not expecting reading from the status register to clear ARTTIM23_INTR_CH1.
- If the interrupt that Solaris 10 expects is not the one from the SET_FEATURE command, then it must expect an interrupt to occur from writing to the device/head register.

I found it strange that Solaris 10 was spinning on ARTTIM23_INTR_CH1. Is it possible that Solaris 10 is not expecting the values of ARTTIM23_INTR_CH1 and MRDMODE_INTR_CH1 to be synced? I made changes to disable the syncing and the fatal error from Solaris 10 disappeared. Unfortunately, I can't tell whether or not this actually improved the emulation of Solaris 10 as the serial console is still unresponsive.

If there is a bug in the Solaris 10 driver I would expect this error to be more widely referenced online. I suspect that the emulated CMD646 is not perfectly faithful to the hardware and this is causing problems for Solaris 10.
I am not convinced that this problem is related to IRQ routing as Solaris 10 appears to recognise interrupts when they happen (or don't). Because of this, I don't think this error is related  to the DMA problem under MorphOS but I could be wrong.

Does anyone have any ideas that might explain why Solaris 10 insists that ARTTIM23_INTR_CH1 is set despite two previous reads of the status register?

Thanks,
Jasper Lowell.

-----Original Message-----
From: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> 
Sent: Sunday, 9 February 2020 10:26 PM
To: Lowell,J,Jasper,VIM R <jasper.lowell@bt.com>; qemu-devel@nongnu.org
Cc: atar4qemu@gmail.com
Subject: Re: Emulating Solaris 10 on SPARC64 sun4u

On 05/02/2020 06:31, jasper.lowell@bt.com wrote:

> I'm currently working towards emulating Solaris 10 on sun4u.
> 
>  
> 
> The Solaris 10 ISO image I am attempting to boot is the one from the 
> Oracle
> 
> download page at
> https://www.oracle.com/solaris/solaris10/downloads/solaris10-get-jsp-downloads.html.
> 
> Image: sol-10-u11-ga-sparc-dvd.iso
> 
> MD5:   53e8b066f7f250ce2fd2cef063f8072b
> 
>  
> 
> I am using QEMU commit 7bd9d0a9e26c7a3c67c0f174f0009ba19969b158.
> 
>  
> 
> The command I am using to run QEMU is:
> 
> ./qemu/sparc64-softmmu/qemu-system-sparc64 -bios 
> ./openbios/obj-sparc64/openbios-builtin.elf -cdrom 
> ./iso/solaris/sol-10-u11-ga-sparc-dvd.iso -boot d -nographic -m 3G
> 
>  
> 
> ```
> 
> CPUs: 1 x SUNW,UltraSPARC-IIi
> 
> UUID: 00000000-0000-0000-0000-000000000000
> 
> Welcome to OpenBIOS v1.1 built on Feb 5 2020 19:15
> 
>   Type 'help' for detailed information
> 
> Trying cdrom:f...
> 
> Not a bootable ELF image
> 
> Not a bootable a.out image
> 
>  
> 
> Loading FCode image...
> 
> Loaded 7420 bytes
> 
> entry point is 0x4000
> 
> Evaluating FCode...
> 
> Evaluating FCode...
> 
> Ignoring failed claim for va 1000000 memsz af6d6!
> 
> Ignoring failed claim for va 1402000 memsz 4dcc8!
> 
> Ignoring failed claim for va 1800000 memsz 510c8!
> 
> SunOS Release 5.10 Version Generic_147147-26 64-bit
> 
> Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
> 
> could not find debugger-vocabulary-hook>threads:interpret: exception 
> -13 caught
> 
> interpret \ Copyright (c) 1995-1999 by Sun Microsystems, Inc.
> 
> \ All rights reserved.
> 
> \
> 
> \ ident "@(#)data64.fth  1.3     00/07/17 SMI"
> 
>  
> 
> hex
> 
>  
> 
> only forth also definitions
> 
> vocabulary kdbg-words
> 
> also kdbg-words definitions
> 
>  
> 
> defer p@
> 
> defer p!
> 
> ['] x@ is p@
> 
> ['] x! is p!
> 
>  
> 
> 8 constant ptrsize
> 
>  
> 
> d# 32 constant nbitsminor
> 
> h# ffffffff constant maxmin
> 
> \
> 
> \ Copyright 2008 Sun Microsystems, Inc.  All rights reserved.
> 
> \ Use is subject to license terms.
> 
> \
> 
>  
> 
> \ #pragma ident  "@(#)kdbg.fth    1.20    08/06/06 SMI"
> 
>  
> 
> h# 7ff constant v9bias
> 
> h# unix-tte:interpret: exception -13 caught
> 
> interpret ' unix-tte is va>tte-data failed with error ffffffffffffffed
> 
> WARNING: consconfig: cannot find driver for screen device 
> /pci@1fe,0/pci@1,1/QEMU,VGA@2
> 
> Configuring devices.
> 
> WARNING: Interrupt not seen after set_features
> 
> Using RPC Bootparams for network configuration information.
> 
> Attempting to configure interface hme0...
> 
> WARNING: Power off requested from power button or SC, powering down the system!
> 
> Skipped interface hme0
> 
> svc:/system/filesystem/local:default: WARNING: /usr/sbin/zfs mount -a 
> failed: one or more file systems failed to mount
> 
> Serial console, reverting to text install
> 
> Beginning system identification...
> 
> Searching for configuration file(s)...
> 
> Search complete.
> 
> Discovering additional network configuration...
> 
> ```
> 
>  
> 
> The installation menu is shown after but the console is unresponsive.
> 
>  
> 
> After some debugging, it looks like the QEMU front-end is correctly 
> filling
> 
> the serial receive buffer with characters, and then starts dropping 
> them once
> 
> the number of characters in the buffer reach the interrupt level. The 
> interrupt
> 
> level happens to be 1 when booting Solaris 10. This looks like normal 
> behaviour
> 
> to me.
> 
>  
> 
> I started looking at why the serial receive buffer might not be 
> consumed and
> 
> considered that interrupts might not be being raised correctly. I ran 
> with
> 
> tracing and there were no interrupts for IRQ 0x2b like there are when 
> using
> 
> OpenBSD. When inspecting the registers of the serial device it looks 
> like the
> 
> Interrupt Enable Register is set to zero.
> 
>  
> 
> If Solaris 10 was using the device is polling mode, it should be 
> reading the RBR
> 
> or at least the LSR. When tracing serial_ioport_read and 
> serial_ioport_write,
> 
> once the menu is hit, I don't see any read or writes to the serial 
> device
> 
> registers despite me trying to send characters and use the menu.
> 
>  
> 
> The driver Solaris 10 is using for the device appears to be 
> similar/same as
> 
> /usr/src/uts/sun4/io/su_driver.c in the OpenSolaris code found at 
> https://github.com/nxmirrors/onnv.
> 
>  
> 
> ```
> 
> asy->asy_hwtype = ASY16550AF;
> 
> OUTB(FIFOR, 0x00); /* clear fifo register */
> 
> asy->asy_trig_level = 0x00; /* sets the fifo Threshold to 1 */
> 
>  
> 
> /* set/Enable FIFO */
> 
> OUTB(FIFOR, FIFO_ON | FIFODMA | FIFOTXFLSH | FIFORXFLSH |
> 
> (asy->asy_trig_level & 0xff));
> 
>  
> 
> if ((INB(ISR) & 0xc0) == 0xc0)
> 
>     asy->asy_use_fifo = FIFO_ON; /* QEMU REACHES HERE. */
> 
> else {
> 
>     asy->asy_hwtype = ASY8250;
> 
>     OUTB(FIFOR, 0x00); /* NO FIFOs */
> 
>     asy->asy_trig_level = 0;
> 
> }
> 
> ```
> 
>  
> 
> From what I can tell when tracing serial_ioport_write and 
> serial_ioport_read,
> 
> Solaris 10 correctly identifies the serial device and successfully attaches it.
> 
> In the asyattach function (OpenSolaris driver), interrupts are 
> disabled by zeroing the
> 
> Interrupt Enable Register. From what I'm reading in OpenSolaris source 
> code, interrupts
> 
> are reenabled when the device is "opened". This seems like consistent 
> and
> 
> correct behaviour though I'm not sure why the device is not being 
> opened to be
> 
> used by the serial console.
> 
>  
> 
> Is this an issue anyone else has tried to debug?
> 
> Are there any leads that I can follow up on for why the serial console 
> becomes unresponsive
> 
> on Solaris 10?

It has been a while since I've looked at booting Solaris >= 10 but certainly the messages above about set_features and the frozen console suggest that something is going amiss with interrupt routing, although since Linux and NetBSD were fine the last time I ran my OpenBIOS release tests then Solaris must be doing something different here.

Note that the serial interrupts are routed from the ebus into sabre so the first thing to check would be that the end-to-end routing from device to CPU looks correct when using Solaris.


ATB,

Mark.