RE: [PATCH v2 03/14] x86/retpoline: Simplify retpolines

[David Laight <David.Laight@ACULAB.COM>]

From: Peter Zijlstra
> Sent: 18 March 2021 17:11
> 
> Due to commit c9c324dc22aa ("objtool: Support stack layout changes
> in alternatives"), it is possible to simplify the retpolines.
> 
...
> Notice that since the longest alternative sequence is now:
> 
>    0:   e8 07 00 00 00          callq  c <.altinstr_replacement+0xc>
>    5:   f3 90                   pause
>    7:   0f ae e8                lfence
>    a:   eb f9                   jmp    5 <.altinstr_replacement+0x5>
>    c:   48 89 04 24             mov    %rax,(%rsp)
>   10:   c3                      retq
> 
> 17 bytes, we have 15 bytes NOP at the end of our 32 byte slot. (IOW,
> if we can shrink the retpoline by 1 byte we can pack it more dense)

I'm intrigued about the lfence after the pause.
Clearly this is for very warped cpu behaviour.
To get to the pause you have to be speculating past an
unconditional call.

To get to the lfence you have to (mostly) have ignored the pause.
Which is commented:
	_mm_pause(); /* Abort speculation */
in a couple of examples in 248966-033.

I wonder what effect replacing the lfence with hlt would have?
It would certainly save 2 bytes and allow the entire retpoline
be put into a single 16byte code fetch block.

248966-033 also contains a note that the instruction(s) after
an indirect jump may get executed.
It suggests adding a pause (or illegal instruction) to stop
anything odd happening (they knew it could be horrid in June 2016.
But then go on to say the adding pause may be a performance issue.
(Presumably because if it is speculatively executed it takes ages.)

I do remember something from even longer ago about trying to never
speculate any of the trig opcodes - because at last some cpu couldn't
abort the instruction.

This may also mean that a big pile of 0x90 nops after the jmp (%eax)
is actually better than 2 or 3 'big' nops.
Of course, if you execute the nops you always want the big ones.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)