[PATCH 1/2] powerpc: Document issues with the DAWR on POWER9

[PATCH 1/2] powerpc: Document issues with the DAWR on POWER9

[Michael Neuling]

Signed-off-by: Michael Neuling <mikey@neuling.org>
---
 Documentation/powerpc/DAWR-POWER9.txt | 58 +++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)
 create mode 100644 Documentation/powerpc/DAWR-POWER9.txt

diff --git a/Documentation/powerpc/DAWR-POWER9.txt b/Documentation/powerpc/DAWR-POWER9.txt
new file mode 100644
index 0000000000..cc6e69b69b
--- /dev/null
+++ b/Documentation/powerpc/DAWR-POWER9.txt
@@ -0,0 +1,58 @@
+DAWR issues on POWER9
+============================
+
+On POWER9 the DAWR can cause a checkstop if it points to cache
+inhibited (CI) memory. Currently Linux has no way to disinguish CI
+memory when configuring the DAWR, so (for now) the DAWR is disabled by
+this commit:
+
+    commit 9654153158d3e0684a1bdb76dbababdb7111d5a0
+    Author: Michael Neuling <mikey@neuling.org>
+    Date:   Tue Mar 27 15:37:24 2018 +1100
+    powerpc: Disable DAWR in the base POWER9 CPU features
+
+Technical Details:
+============================
+
+DAWR has 6 different ways of being set.
+1) ptrace
+2) h_set_mode(DAWR)
+3) h_set_dabr()
+4) kvmppc_set_one_reg()
+5) xmon
+
+For ptrace, we now advertise zero breakpoints on POWER9 via the
+PPC_PTRACE_GETHWDBGINFO call. This results in GDB falling back to
+software emulation of the watchpoint (which is slow).
+
+h_set_mode(DAWR) and h_set_dabr() will now return an error to the
+guest on a POWER9 host. Current Linux guests ignore this error, so
+they will silently not get the DAWR.
+
+kvmppc_set_one_reg() will store the value in the vcpu but won't
+actually set it on POWER9 hardware. This is done so we don't break
+migration from POWER8 to POWER9, at the cost of silently losing the
+DAWR on the migration.
+
+For xmon, the 'bd' command will return an error on P9.
+
+Consequences for users
+============================
+
+For GDB watchpoints (ie 'watch' command) on POWER9 bare metal , GDB
+will accept the command. Unfortunatley since there is no hardware
+support for the watchpoint, GDB will software emulate the watchpoint
+making it run very slowly.
+
+The same will also be true for any guests started on a POWER9
+host. The watchpoint will fail and GDB will fall back to software
+emulation.
+
+If a guest is started on a POWER8 host, GDB will accept the watchpoint
+and configure the hardware to use the DAWR. This will run at full
+speed since it can use the hardware emualation. Unfortnatley if this
+guest is migrated to a POWER9 host, the watchpoint will be lost on the
+POWER9. Loads and stores to the watchpoint locations will not be
+trapped in GDB. The watchpoint is remembered, so if the guest is
+migrated back to the POWER8 host, it will start working again.
+
-- 
2.17.1

[PATCH 2/2] powerpc: Document issues with TM on POWER9

[Michael Neuling]

Signed-off-by: Michael Neuling <mikey@neuling.org>
---
 .../powerpc/transactional_memory.txt          | 44 +++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/Documentation/powerpc/transactional_memory.txt b/Documentation/powerpc/transactional_memory.txt
index e32fdbb4c9..b254eab517 100644
--- a/Documentation/powerpc/transactional_memory.txt
+++ b/Documentation/powerpc/transactional_memory.txt
@@ -198,3 +198,47 @@ presented).  The transaction cannot then be continued and will take the failure
 handler route.  Furthermore, the transactional 2nd register state will be
 inaccessible.  GDB can currently be used on programs using TM, but not sensibly
 in parts within transactions.
+
+POWER9
+======
+
+TM on POWER9 has issues with storing the complete register state. This
+is described in this commit:
+
+    commit 4bb3c7a0208fc13ca70598efd109901a7cd45ae7
+    Author: Paul Mackerras <paulus@ozlabs.org>
+    Date:   Wed Mar 21 21:32:01 2018 +1100
+    KVM: PPC: Book3S HV: Work around transactional memory bugs in POWER9
+
+To account for this different POWER9 chips have TM enabled in
+different ways.
+
+On POWER9N DD2.01 and below, TM is disabled. ie
+HWCAP2[PPC_FEATURE2_HTM] is not set.
+
+On POWER9N DD2.1 TM is configured by firmware to always abort a
+transaction when tm suspend occurs. So tsuspend will cause a
+transaction to be aborted and rolled back. Kernel exceptions will also
+cause the transaction to be aborted and rolled back and the exception
+will not occur. If userspace constructs a sigcontext that enables TM
+suspend, the sigcontext will be rejected by the kernel. This mode is
+advertised to users with HWCAP2[PPC_FEATURE2_HTM_NO_SUSPEND] set.
+HWCAP2[PPC_FEATURE2_HTM] is not set in this mode.
+
+On POWER9N DD2.2 and above, KVM and POWERVM emulate TM for guests (as
+descibed in commit 4bb3c7a0208f), hence TM is enabled for guests
+ie. HWCAP2[PPC_FEATURE2_HTM] is set for guest userspace. Guests that
+makes heavy use of TM suspend (tsuspend or kernel suspend) will result
+in traps into the hypervisor and hence will suffer a performance
+degredation. Host userspace has TM disabled
+ie. HWCAP2[PPC_FEATURE2_HTM] is not set. (although we make enable it
+at some point in the future if we bring the emulation into host
+userspace context switching).
+
+POWER9C DD1.2 and above are only avaliable with POWERNV and hence
+Linux only runs as a guest. On these systems TM is emulated like on
+POWER9N DD2.2.
+
+Guest migration from POWER8 to POWER9 will work with POWER9N DD2.2 and
+POWER9C DD1.2. Since earlier POWER9 processors don't support TM
+emulation, migration from POWER8 to POWER9 is not supported there.
-- 
2.17.1

Re: [PATCH 2/2] powerpc: Document issues with TM on POWER9

[Stewart Smith]

Michael Neuling <mikey@neuling.org> writes:
> +POWER9C DD1.2 and above are only avaliable with POWERNV and hence

PowerVM, not POWERNV


-- 
Stewart Smith
OPAL Architect, IBM.

Re: [PATCH 2/2] powerpc: Document issues with TM on POWER9

[Segher Boessenkool]

On Fri, Jun 22, 2018 at 04:14:52PM +1000, Michael Neuling wrote:
> +degredation. Host userspace has TM disabled

"degradation"

> +POWER9C DD1.2 and above are only avaliable with POWERNV and hence

"available"


Segher

Re: [PATCH 1/2] powerpc: Document issues with the DAWR on POWER9

[Stewart Smith]

Michael Neuling <mikey@neuling.org> writes:
> Signed-off-by: Michael Neuling <mikey@neuling.org>
> ---
>  Documentation/powerpc/DAWR-POWER9.txt | 58 +++++++++++++++++++++++++++
>  1 file changed, 58 insertions(+)
>  create mode 100644 Documentation/powerpc/DAWR-POWER9.txt

Acked-by: Stewart Smith <stewart@linux.ibm.com>

-- 
Stewart Smith
OPAL Architect, IBM.

Re: [PATCH 1/2] powerpc: Document issues with the DAWR on POWER9

[Segher Boessenkool]

On Fri, Jun 22, 2018 at 04:14:51PM +1000, Michael Neuling wrote:
> +will accept the command. Unfortunatley since there is no hardware

"unfortunately".

> +speed since it can use the hardware emualation. Unfortnatley if this

It is not your favourite word to type ;-)


Segher

Re: [PATCH 1/2] powerpc: Document issues with the DAWR on POWER9

[Michael Neuling]

On Fri, 2018-06-22 at 12:23 -0500, Segher Boessenkool wrote:
> On Fri, Jun 22, 2018 at 04:14:51PM +1000, Michael Neuling wrote:
> > +will accept the command. Unfortunatley since there is no hardware
>=20
> "unfortunately".
>=20
> > +speed since it can use the hardware emualation. Unfortnatley if this
>=20
> It is not your favourite word to type ;-)

Or emulation apparently :-/

Thanks.
Mikey