On 03/27/2017 09:05, Ralf Baechle wrote:
> On Sun, Mar 26, 2017 at 09:50:08PM -0400, Joshua Kinard wrote:
> 
>> Does anyone know if the R1x000 family of CPUs support the "wait" instruction?
>> The 'check_wait' function in arch/mips/kernel/idle.c doesn't have a case for
>> any of the R10K CPUs, and I can't find any specific guidance in the final R10K
>> manual produced by Renesas, nor in the MIPS-IV instruction set.  It appears
>> this was added in MIPS-II, and the R4K CPUs use it, with one version for when
>> interrupts are enabled, and one where they're disabled.  Since a lot of CPUs
>> tend to reuse R4K-compatible code, I wasn't sure.
> 
> Interesting, didn't know Renesas did another R10000 manual.  Presumably
> they only rebranded NEC's manual?

Yup, mostly.  It's document U10278EJ4V0UM00, 4th edition, though I can't get
the search engine on their site to cough a link up, so I've attached the copy I
found.  The manual looks like it was dated from March 2001, which covers at
least up to R12000, but has a "phased-out/discontinued" stamp and a notice to
customers dated April 2010.

I still, to this date, have never come across any documentation on the R14000
or R16000.  That must be internal to SGI and HP (used in their NonStop servers)
and never saw the light of day.  Technically HP now, since they recently
scooped up what was left of SGI.


> If you have any documentation to indicate a MIPS II CPU to support WAIT,
> I'm interested.  From all that I know the feature was introduced by the
> R4600.

One of my Google searches, using the keywords "mips wait instruction", has been
returning search results from a Harvard computer sciences course detailing the
"instructional operating system OS/161", which uses an instructional CPU that
borrows heavily from the R3000 (which they dub MIPS-161):

http://os161.eecs.harvard.edu/documentation/sys161-1.99.07/mips.html

In there, they state:
"The WAIT instruction has been borrowed from MIPS-II. This operation puts the
processor into a low-power state and suspends execution until some external
event occurs, such as an interrupt. Since the exact behavior of WAIT is not
clearly specified anywhere I could find, the MIPS-161 behavior is as follows"

So it could be that the instructor for that course simply got some wrong
information, but good luck teaching Google that.  I figure once its spider
crawls this e-mail in the archives, it'll further strengthen hits like the above.


>> Kinda-assuming it doesn't, since the R10K lacks any notion of reduced power
>> operation.
> 
> The R10000 like many of the older MIPS CPU took a hardware-only approach
> to low-power operation, that is there are no knobs, no instructions for
> software to optimize the power consumption.  That also means, no WAIT
> instruction.  Other bits of low-power support in R4x00-processors are
> fake also, for example the c0_status.rp "reduced power" bit is documented
> but fairly hidden erratas say it's not implemneted.  So just throw in another
> plutonium slab and all will be good ;-)

It might make sense to add the R10K cases to the switch in cpu_wait, right
above the "default" block, with a comment stating that R10K doesn't support
wait.  That way it's documented in the right spot in case this question ever
comes up in the future.  I can send a patch for this later on if interested.


> But back to the WAIT instruction.  WAIT uses major opcode COP0 which has
> bits 31..27 = 010000.  Traditionally MIPS doesn't fully decode the minor
> opcodes, so an unsupported WAIT instruction will not cause an Reserved
> Instruction exception anyway because the COP0 opcode does exist.

That would explain why when I tried it briefly on an Octane kernel, nothing
happened.  So I wasn't sure if it was supported or not.


> So one could basically pretend WAIT did exist on all MIPS CPUs, even
> R2000 even though it doesn't - but check_wait is paranoid and really
> avoids WAIT unless it's officially supported and useful.
> 
> Another complexity is that WAIT instruction was defined for many years
> to leave it to an implementation if the pipeline would ever restart
> after a WAIT instruction was executed with interrupts disabled.  This
> is fairly braindead because in a typical idle loop of an OS such as Linux
> 
>         for (;;) {
>                 if (wait_available && !need_resched())
>                         asm("wait");
>                 schedule();
>         }
> 
> there is the race condition between checking need_resched() and
> actually executing WAIT that need_resched() might change, so WAIT-mode
> might be entered falsely causing a scheduling delay of up to one tick.
> The solution is to disable interrupts like:
> 
>         for (;;) {
> 		disable_irq();
>                 if (wait_available && !need_resched())
>                         asm("wait");
> 		enable_irq();
>                 schedule();
>         }
> 
> But some MIPS CPUs might lockup for good if this is attempted ...
> 
> This turned out to be a longer writeup than I meant to because I
> recognized much of this is not documented anywhere so I did elaborate
> a bit and add it as a new wiki page:
> 
> https://www.linux-mips.org/wiki/WAIT_instruction

And this is where I think I am seeing the IP27's main issue, a lock up having
to do with a race condition in enabling/disabling IRQs, and possibly the
registers getting clobbered in the process, leading to the resumed process
getting garbage data.

Several of my google searches on a race in enabling/disabling IRQs kept coming
back with hits to the wait instruction, and since I couldn't find any
information on if the R10K supported it, decided to write my e-mail.  So I can
scratch that angle off of my list.

Thanks!

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
6144R/F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And our
lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic