[PATCH v2 0/5] ARM: OMAP2+: AM33XX/AM43XX: Add suspend-resume support

[PATCH v2 0/5] ARM: OMAP2+: AM33XX/AM43XX: Add suspend-resume support

[Dave Gerlach]

Hi,
This series contains the remaining code to enable suspend to mem and standby on
am335x and am437x platforms. It depends on the ti-emif-sram series sent here
[1], and the first 3 patches in v1 found here [2] which have been queued for
merge already and dropped from this series. Apart from dropping the first three
patches, changes from v1 are to update the sram_exec_copy usage to treat the
return address as the proper callable pointer when copying functions and some
small cleanups to the assembly code in the first two patches. However, as no
review was done on v1 due to sram_exec_copy discussion I am including the
original cover letter below.

This code also depends on firmware for the wkup_m3 that is available here [3]
that can either be placed in /lib/firmware or built in to the kernel.

Patches 1 and 2 add the required low-level PM suspend code that is relocated to
SRAM for each platform.  Patch 3 and patch 4 serve as the split between code
that can live in drivers and code that must remain under mach-omap2. The idea
here was to maintain the minimal amount of ARM and OMAP specific code inside
mach-omap2 while providing ops that can be called into from a module that lives
in drivers and contains the bulk of the PM code. Patch 3 also contains
additions to the ARM asm-offsets file to allow us to automatically generate C
struct member offsets and sizes to be used by the assembly code introduced
earlier, more info can be seen in the changelog.  Patch 4 introduces the pm33xx
driver which makes use of the wkup_m3_ipc and ti-emif-sram drivers to enable PM
on the system.  Because pm33xx depends on modules it must be able to defer
probe so patch 5 creates a dummy pm33xx platform device to cause pm33xx to
probe initially but also defer if needed.

As with the ti-emif-sram driver, which also contains relocatable SRAM code, we
can only get read-only exectuable memory regions from the mmio-sram driver so
we also provide a second region to the driver that is read/write to be used for
data during execution. This is the reason two sram regions are mapped and so
many offsets must be provided to the code section.

For testing the drivers required are:

CONFIG_MAILBOX
CONFIG_OMAP2PLUS_MBOX
CONFIG_WKUP_M3_RPROC
CONFIG_SOC_TI
CONFIG_WKUP_M3_IPC
CONFIG_TI_EMIF_SRAM
CONFIG_AMX3_PM

There is still a device tree patch series to follow that will completely enable
PM but I will wait to send that until this series and the emif series have been
reviewed, until then all patches have been pushed here for use [4].

Regards,
Dave

[1] https://www.spinics.net/lists/arm-kernel/msg582278.html
[2] https://www.spinics.net/lists/arm-kernel/msg572535.html
[3] https://git.ti.com/processor-firmware/ti-amx3-cm3-pm-firmware/blobs/7eb9c0856a9e8b3b42bf64f761da135852b8eea7/bin/am335x-pm-firmware.elf
[4] https://github.com/dgerlach/linux-pm/tree/upstream/v4.12/amx3-suspend-dev

Dave Gerlach (5):
  ARM: OMAP2+: Introduce low-level suspend code for AM33XX
  ARM: OMAP2+: Introduce low-level suspend code for AM43XX
  ARM: OMAP2+: pm33xx-core: Add platform code needed for PM
  soc: ti: Add pm33xx driver for basic suspend support
  ARM: OMAP2+: Create dummy platform_device for pm33xx

 arch/arm/kernel/asm-offsets.c        |   2 +
 arch/arm/mach-omap2/Kconfig          |   1 +
 arch/arm/mach-omap2/Makefile         |   4 +
 arch/arm/mach-omap2/common.h         |   7 +
 arch/arm/mach-omap2/io.c             |   2 +
 arch/arm/mach-omap2/pm.h             |   5 +
 arch/arm/mach-omap2/pm33xx-core.c    | 193 +++++++++++++++++
 arch/arm/mach-omap2/sleep33xx.S      | 221 ++++++++++++++++++++
 arch/arm/mach-omap2/sleep43xx.S      | 394 +++++++++++++++++++++++++++++++++++
 drivers/soc/ti/Kconfig               |   9 +
 drivers/soc/ti/Makefile              |   1 +
 drivers/soc/ti/pm33xx.c              | 337 ++++++++++++++++++++++++++++++
 include/linux/platform_data/pm33xx.h |  69 ++++++
 13 files changed, 1245 insertions(+)
 create mode 100644 arch/arm/mach-omap2/pm33xx-core.c
 create mode 100644 arch/arm/mach-omap2/sleep33xx.S
 create mode 100644 arch/arm/mach-omap2/sleep43xx.S
 create mode 100644 drivers/soc/ti/pm33xx.c
 create mode 100644 include/linux/platform_data/pm33xx.h

-- 
2.13.0

[PATCH v2 1/5] ARM: OMAP2+: Introduce low-level suspend code for AM33XX

[Dave Gerlach]

In preparation for suspend-resume support for AM33XX, add
the assembly file with the code which is copied to internal
memory (OCMC RAM) during bootup and runs from there.

As part of the low power entry (DeepSleep0 mode in AM33XX TRM),
the code running from OCMC RAM does the following
1. Calls routine to store the EMIF configuration
2. Calls routine to place external memory in self-refresh
3. Disables EMIF clock
4. Executes WFI after writing to MPU_CLKCTRL register.

If no interrupts have come, WFI execution on MPU gets registered
as an interrupt with the WKUP-M3. WKUP-M3 takes care of disabling
some clocks which MPU should not (L3, L4, OCMC RAM etc) and takes
care of clockdomain and powerdomain transitions as part of the
DeepSleep0 mode entry.

In case a late interrupt comes in, WFI ends up as a NOP and MPU
continues execution from internal memory. The 'abort path' code
undoes whatever was done as part of the low power entry and indicates
a suspend failure by passing a non-zero value to the cpu_resume routine.

The 'resume path' code is similar to the 'abort path' with the key
difference of MMU being enabled in the 'abort path' but being
disabled in the 'resume path' due to MPU getting powered off.

Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 arch/arm/mach-omap2/sleep33xx.S | 221 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 221 insertions(+)
 create mode 100644 arch/arm/mach-omap2/sleep33xx.S

diff --git a/arch/arm/mach-omap2/sleep33xx.S b/arch/arm/mach-omap2/sleep33xx.S
new file mode 100644
index 000000000000..1141bfb5108f
--- /dev/null
+++ b/arch/arm/mach-omap2/sleep33xx.S
@@ -0,0 +1,221 @@
+/*
+ * Low level suspend code for AM33XX SoCs
+ *
+ * Copyright (C) 2012-2017 Texas Instruments Incorporated - http://www.ti.com/
+ *	Dave Gerlach, Vaibhav Bedia
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/linkage.h>
+#include <linux/ti-emif-sram.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/memory.h>
+
+#include "iomap.h"
+#include "cm33xx.h"
+
+#define AM33XX_CM_CLKCTRL_MODULESTATE_DISABLED			0x00030000
+#define AM33XX_CM_CLKCTRL_MODULEMODE_DISABLE			0x0003
+#define AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE			0x0002
+
+	.arm
+	.align 3
+
+ENTRY(am33xx_do_wfi)
+	stmfd	sp!, {r4 - r11, lr}	@ save registers on stack
+
+	/*
+	 * Flush all data from the L1 and L2 data cache before disabling
+	 * SCTLR.C bit.
+	 */
+	ldr	r1, kernel_flush
+	blx	r1
+
+	/*
+	 * Clear the SCTLR.C bit to prevent further data cache
+	 * allocation. Clearing SCTLR.C would make all the data accesses
+	 * strongly ordered and would not hit the cache.
+	 */
+	mrc	p15, 0, r0, c1, c0, 0
+	bic	r0, r0, #(1 << 2)	@ Disable the C bit
+	mcr	p15, 0, r0, c1, c0, 0
+	isb
+
+	/*
+	 * Invalidate L1 and L2 data cache.
+	 */
+	ldr	r1, kernel_flush
+	blx	r1
+
+	adr	r9, am33xx_emif_sram_table
+
+	ldr	r3, [r9, #EMIF_PM_SAVE_CONTEXT_OFFSET]
+	blx	r3
+
+	ldr	r3, [r9, #EMIF_PM_ENTER_SR_OFFSET]
+	blx	r3
+
+	/* Disable EMIF */
+	ldr     r1, virt_emif_clkctrl
+	ldr     r2, [r1]
+	bic     r2, r2, #AM33XX_CM_CLKCTRL_MODULEMODE_DISABLE
+	str     r2, [r1]
+
+	ldr	r1, virt_emif_clkctrl
+wait_emif_disable:
+	ldr	r2, [r1]
+	mov	r3, #AM33XX_CM_CLKCTRL_MODULESTATE_DISABLED
+	cmp	r2, r3
+	bne	wait_emif_disable
+
+	/*
+	 * For the MPU WFI to be registered as an interrupt
+	 * to WKUP_M3, MPU_CLKCTRL.MODULEMODE needs to be set
+	 * to DISABLED
+	 */
+	ldr	r1, virt_mpu_clkctrl
+	ldr	r2, [r1]
+	bic	r2, r2, #AM33XX_CM_CLKCTRL_MODULEMODE_DISABLE
+	str	r2, [r1]
+
+	/*
+	 * Execute an ISB instruction to ensure that all of the
+	 * CP15 register changes have been committed.
+	 */
+	isb
+
+	/*
+	 * Execute a barrier instruction to ensure that all cache,
+	 * TLB and branch predictor maintenance operations issued
+	 * have completed.
+	 */
+	dsb
+	dmb
+
+	/*
+	 * Execute a WFI instruction and wait until the
+	 * STANDBYWFI output is asserted to indicate that the
+	 * CPU is in idle and low power state. CPU can specualatively
+	 * prefetch the instructions so add NOPs after WFI. Thirteen
+	 * NOPs as per Cortex-A8 pipeline.
+	 */
+	wfi
+
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+
+	/* We come here in case of an abort due to a late interrupt */
+
+	/* Set MPU_CLKCTRL.MODULEMODE back to ENABLE */
+	ldr	r1, virt_mpu_clkctrl
+	mov	r2, #AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE
+	str	r2, [r1]
+
+	/* Re-enable EMIF */
+	ldr	r1, virt_emif_clkctrl
+	mov	r2, #AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE
+	str	r2, [r1]
+wait_emif_enable:
+	ldr	r3, [r1]
+	cmp	r2, r3
+	bne	wait_emif_enable
+
+
+	ldr	r1, [r9, #EMIF_PM_ABORT_SR_OFFSET]
+	blx	r1
+
+	/*
+	 * Set SCTLR.C bit to allow data cache allocation
+	 */
+	mrc	p15, 0, r0, c1, c0, 0
+	orr	r0, r0, #(1 << 2)	@ Enable the C bit
+	mcr	p15, 0, r0, c1, c0, 0
+	isb
+
+	/* Let the suspend code know about the abort */
+	mov	r0, #1
+	ldmfd	sp!, {r4 - r11, pc}	@ restore regs and return
+ENDPROC(am33xx_do_wfi)
+
+	.align
+ENTRY(am33xx_resume_offset)
+	.word . - am33xx_do_wfi
+
+ENTRY(am33xx_resume_from_deep_sleep)
+	/* Re-enable EMIF */
+	ldr	r0, phys_emif_clkctrl
+	mov	r1, #AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE
+	str	r1, [r0]
+wait_emif_enable1:
+	ldr	r2, [r0]
+	cmp	r1, r2
+	bne	wait_emif_enable1
+
+	adr	r9, am33xx_emif_sram_table
+
+	ldr	r1, [r9, #EMIF_PM_RESTORE_CONTEXT_OFFSET]
+	blx	r1
+
+	ldr	r1, [r9, #EMIF_PM_EXIT_SR_OFFSET]
+	blx	r1
+
+resume_to_ddr:
+	/* We are back. Branch to the common CPU resume routine */
+	mov	r0, #0
+	ldr	pc, resume_addr
+ENDPROC(am33xx_resume_from_deep_sleep)
+
+/*
+ * Local variables
+ */
+	.align
+resume_addr:
+	.word	cpu_resume - PAGE_OFFSET + 0x80000000
+kernel_flush:
+	.word   v7_flush_dcache_all
+virt_mpu_clkctrl:
+	.word	AM33XX_CM_MPU_MPU_CLKCTRL
+virt_emif_clkctrl:
+	.word	AM33XX_CM_PER_EMIF_CLKCTRL
+phys_emif_clkctrl:
+	.word	(AM33XX_CM_BASE + AM33XX_CM_PER_MOD + \
+		AM33XX_CM_PER_EMIF_CLKCTRL_OFFSET)
+
+.align 3
+/* DDR related defines */
+am33xx_emif_sram_table:
+	.space EMIF_PM_FUNCTIONS_SIZE
+
+ENTRY(am33xx_pm_sram)
+.word am33xx_do_wfi
+.word am33xx_do_wfi_sz
+.word am33xx_resume_offset
+.word am33xx_emif_sram_table
+.word am33xx_pm_ro_sram_data
+
+.align 3
+ENTRY(am33xx_pm_ro_sram_data)
+	.space AMX3_PM_RO_SRAM_DATA_SIZE
+
+ENTRY(am33xx_do_wfi_sz)
+	.word	. - am33xx_do_wfi
-- 
2.13.0

[PATCH v2 2/5] ARM: OMAP2+: Introduce low-level suspend code for AM43XX

[Dave Gerlach]

Although similar to AM33XX, introduce a new low-level asm file for
suspend containing new context save and restore paths for EMIF and l2
cache disabling and enabling.

Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 arch/arm/mach-omap2/sleep43xx.S | 394 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 394 insertions(+)
 create mode 100644 arch/arm/mach-omap2/sleep43xx.S

diff --git a/arch/arm/mach-omap2/sleep43xx.S b/arch/arm/mach-omap2/sleep43xx.S
new file mode 100644
index 000000000000..a3d71b53e936
--- /dev/null
+++ b/arch/arm/mach-omap2/sleep43xx.S
@@ -0,0 +1,394 @@
+/*
+ * Low level suspend code for AM43XX SoCs
+ *
+ * Copyright (C) 2013-2017 Texas Instruments Incorporated - http://www.ti.com/
+ *	Dave Gerlach, Vaibhav Bedia
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/linkage.h>
+#include <linux/ti-emif-sram.h>
+
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/hardware/cache-l2x0.h>
+#include <asm/memory.h>
+
+#include "cm33xx.h"
+#include "common.h"
+#include "iomap.h"
+#include "omap-secure.h"
+#include "omap44xx.h"
+#include "prm33xx.h"
+#include "prcm43xx.h"
+
+#define AM33XX_CM_CLKCTRL_MODULESTATE_DISABLED		0x00030000
+#define AM33XX_CM_CLKCTRL_MODULEMODE_DISABLE		0x0003
+#define AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE		0x0002
+
+#define AM43XX_EMIF_POWEROFF_ENABLE			0x1
+#define AM43XX_EMIF_POWEROFF_DISABLE			0x0
+
+#define AM43XX_CM_CLKSTCTRL_CLKTRCTRL_SW_SLEEP		0x1
+#define AM43XX_CM_CLKSTCTRL_CLKTRCTRL_HW_AUTO		0x3
+
+#define AM43XX_CM_BASE					0x44DF0000
+
+#define AM43XX_CM_REGADDR(inst, reg)                           \
+       AM33XX_L4_WK_IO_ADDRESS(AM43XX_CM_BASE + (inst) + (reg))
+
+#define AM43XX_CM_MPU_CLKSTCTRL AM43XX_CM_REGADDR(AM43XX_CM_MPU_INST, \
+					AM43XX_CM_MPU_MPU_CDOFFS)
+#define AM43XX_CM_MPU_MPU_CLKCTRL AM43XX_CM_REGADDR(AM43XX_CM_MPU_INST, \
+					AM43XX_CM_MPU_MPU_CLKCTRL_OFFSET)
+#define AM43XX_CM_PER_EMIF_CLKCTRL  AM43XX_CM_REGADDR(AM43XX_CM_PER_INST, \
+					AM43XX_CM_PER_EMIF_CLKCTRL_OFFSET)
+#define AM43XX_PRM_EMIF_CTRL_OFFSET			0x0030
+
+	.arm
+	.align 3
+
+ENTRY(am43xx_do_wfi)
+	stmfd	sp!, {r4 - r11, lr}	@ save registers on stack
+
+	/* Retrieve l2 cache virt address BEFORE we shut off EMIF */
+	ldr	r1, get_l2cache_base
+	blx	r1
+	mov	r8, r0
+
+	/*
+	 * Flush all data from the L1 and L2 data cache before disabling
+	 * SCTLR.C bit.
+	 */
+	ldr	r1, kernel_flush
+	blx	r1
+
+	/*
+	 * Clear the SCTLR.C bit to prevent further data cache
+	 * allocation. Clearing SCTLR.C would make all the data accesses
+	 * strongly ordered and would not hit the cache.
+	 */
+	mrc	p15, 0, r0, c1, c0, 0
+	bic	r0, r0, #(1 << 2)	@ Disable the C bit
+	mcr	p15, 0, r0, c1, c0, 0
+	isb
+	dsb
+
+	/*
+	 * Invalidate L1 and L2 data cache.
+	 */
+	ldr	r1, kernel_flush
+	blx	r1
+
+#ifdef CONFIG_CACHE_L2X0
+	/*
+	 * Clean and invalidate the L2 cache.
+	 */
+#ifdef CONFIG_PL310_ERRATA_727915
+	mov	r0, #0x03
+	mov	r12, #OMAP4_MON_L2X0_DBG_CTRL_INDEX
+	dsb
+	smc	#0
+	dsb
+#endif
+	mov	r0, r8
+	adr	r4, am43xx_pm_ro_sram_data
+	ldr	r3, [r4, #AMX3_PM_RO_SRAM_DATA_VIRT_OFFSET]
+
+	mov	r2, r0
+	ldr	r0, [r2, #L2X0_AUX_CTRL]
+	str	r0, [r3, #AMX3_PM_L2_AUX_CTRL_VAL_OFFSET]
+	ldr	r0, [r2, #L310_PREFETCH_CTRL]
+	str	r0, [r3, #AMX3_PM_L2_PREFETCH_CTRL_VAL_OFFSET]
+
+	ldr	r0, l2_val
+	str	r0, [r2, #L2X0_CLEAN_INV_WAY]
+wait:
+	ldr	r0, [r2, #L2X0_CLEAN_INV_WAY]
+	ldr	r1, l2_val
+	ands	r0, r0, r1
+	bne	wait
+#ifdef CONFIG_PL310_ERRATA_727915
+	mov	r0, #0x00
+	mov	r12, #OMAP4_MON_L2X0_DBG_CTRL_INDEX
+	dsb
+	smc	#0
+	dsb
+#endif
+l2x_sync:
+	mov	r0, r8
+	mov	r2, r0
+	mov	r0, #0x0
+	str	r0, [r2, #L2X0_CACHE_SYNC]
+sync:
+	ldr	r0, [r2, #L2X0_CACHE_SYNC]
+	ands	r0, r0, #0x1
+	bne	sync
+#endif
+
+	adr     r9, am43xx_emif_sram_table
+
+	ldr     r3, [r9, #EMIF_PM_SAVE_CONTEXT_OFFSET]
+	blx     r3
+
+	ldr     r3, [r9, #EMIF_PM_ENTER_SR_OFFSET]
+	blx     r3
+
+	/* Disable EMIF */
+	ldr	r1, am43xx_virt_emif_clkctrl
+	ldr	r2, [r1]
+	bic	r2, r2, #AM33XX_CM_CLKCTRL_MODULEMODE_DISABLE
+	str	r2, [r1]
+
+wait_emif_disable:
+	ldr	r2, [r1]
+	mov	r3, #AM33XX_CM_CLKCTRL_MODULESTATE_DISABLED
+	cmp	r2, r3
+	bne	wait_emif_disable
+
+	/*
+	 * For the MPU WFI to be registered as an interrupt
+	 * to WKUP_M3, MPU_CLKCTRL.MODULEMODE needs to be set
+	 * to DISABLED
+	 */
+	ldr	r1, am43xx_virt_mpu_clkctrl
+	ldr	r2, [r1]
+	bic	r2, r2, #AM33XX_CM_CLKCTRL_MODULEMODE_DISABLE
+	str	r2, [r1]
+
+	/*
+	 * Put MPU CLKDM to SW_SLEEP
+	 */
+	ldr	r1, am43xx_virt_mpu_clkstctrl
+	mov	r2, #AM43XX_CM_CLKSTCTRL_CLKTRCTRL_SW_SLEEP
+	str	r2, [r1]
+
+	/*
+	 * Execute a barrier instruction to ensure that all cache,
+	 * TLB and branch predictor maintenance operations issued
+	 * have completed.
+	 */
+	dsb
+	dmb
+
+	/*
+	 * Execute a WFI instruction and wait until the
+	 * STANDBYWFI output is asserted to indicate that the
+	 * CPU is in idle and low power state. CPU can specualatively
+	 * prefetch the instructions so add NOPs after WFI. Sixteen
+	 * NOPs as per Cortex-A9 pipeline.
+	 */
+	wfi
+
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+	nop
+
+	/* We come here in case of an abort due to a late interrupt */
+	ldr	r1, am43xx_virt_mpu_clkstctrl
+	mov	r2, #AM43XX_CM_CLKSTCTRL_CLKTRCTRL_HW_AUTO
+	str	r2, [r1]
+
+	/* Set MPU_CLKCTRL.MODULEMODE back to ENABLE */
+	ldr	r1, am43xx_virt_mpu_clkctrl
+	mov	r2, #AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE
+	str	r2, [r1]
+
+	/* Re-enable EMIF */
+	ldr	r1, am43xx_virt_emif_clkctrl
+	mov	r2, #AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE
+	str	r2, [r1]
+wait_emif_enable:
+	ldr	r3, [r1]
+	cmp	r2, r3
+	bne	wait_emif_enable
+
+	/*
+	 * Set SCTLR.C bit to allow data cache allocation
+	 */
+	mrc	p15, 0, r0, c1, c0, 0
+	orr	r0, r0, #(1 << 2)	@ Enable the C bit
+	mcr	p15, 0, r0, c1, c0, 0
+	isb
+
+	ldr     r1, [r9, #EMIF_PM_ABORT_SR_OFFSET]
+	blx     r1
+
+	/* Let the suspend code know about the abort */
+	mov	r0, #1
+	ldmfd	sp!, {r4 - r11, pc}	@ restore regs and return
+ENDPROC(am43xx_do_wfi)
+
+	.align
+ENTRY(am43xx_resume_offset)
+	.word . - am43xx_do_wfi
+
+ENTRY(am43xx_resume_from_deep_sleep)
+	/* Set MPU CLKSTCTRL to HW AUTO so that CPUidle works properly */
+	ldr	r1, am43xx_virt_mpu_clkstctrl
+	mov	r2, #AM43XX_CM_CLKSTCTRL_CLKTRCTRL_HW_AUTO
+	str	r2, [r1]
+
+	/* For AM43xx, use EMIF power down until context is restored */
+	ldr	r2, am43xx_phys_emif_poweroff
+	mov	r1, #AM43XX_EMIF_POWEROFF_ENABLE
+	str	r1, [r2, #0x0]
+
+	/* Re-enable EMIF */
+	ldr	r1, am43xx_phys_emif_clkctrl
+	mov	r2, #AM33XX_CM_CLKCTRL_MODULEMODE_ENABLE
+	str	r2, [r1]
+wait_emif_enable1:
+	ldr	r3, [r1]
+	cmp	r2, r3
+	bne	wait_emif_enable1
+
+	adr     r9, am43xx_emif_sram_table
+
+	ldr     r1, [r9, #EMIF_PM_RESTORE_CONTEXT_OFFSET]
+	blx     r1
+
+	ldr     r1, [r9, #EMIF_PM_EXIT_SR_OFFSET]
+	blx     r1
+
+	ldr     r2, am43xx_phys_emif_poweroff
+	mov     r1, #AM43XX_EMIF_POWEROFF_DISABLE
+	str     r1, [r2, #0x0]
+
+#ifdef CONFIG_CACHE_L2X0
+	ldr	r2, l2_cache_base
+	ldr	r0, [r2, #L2X0_CTRL]
+	and	r0, #0x0f
+	cmp	r0, #1
+	beq	skip_l2en			@ Skip if already enabled
+
+	adr	r4, am43xx_pm_ro_sram_data
+	ldr	r3, [r4, #AMX3_PM_RO_SRAM_DATA_PHYS_OFFSET]
+	ldr     r0, [r3, #AMX3_PM_L2_PREFETCH_CTRL_VAL_OFFSET]
+
+	ldr	r12, l2_smc1
+	dsb
+	smc	#0
+	dsb
+set_aux_ctrl:
+	ldr     r0, [r3, #AMX3_PM_L2_AUX_CTRL_VAL_OFFSET]
+	ldr	r12, l2_smc2
+	dsb
+	smc	#0
+	dsb
+
+	/* L2 invalidate on resume */
+	ldr	r0, l2_val
+	ldr	r2, l2_cache_base
+	str	r0, [r2, #L2X0_INV_WAY]
+wait2:
+	ldr	r0, [r2, #L2X0_INV_WAY]
+	ldr	r1, l2_val
+	ands	r0, r0, r1
+	bne	wait2
+#ifdef CONFIG_PL310_ERRATA_727915
+	mov	r0, #0x00
+	mov	r12, #OMAP4_MON_L2X0_DBG_CTRL_INDEX
+	dsb
+	smc	#0
+	dsb
+#endif
+l2x_sync2:
+	ldr	r2, l2_cache_base
+	mov	r0, #0x0
+	str	r0, [r2, #L2X0_CACHE_SYNC]
+sync2:
+	ldr	r0, [r2, #L2X0_CACHE_SYNC]
+	ands	r0, r0, #0x1
+	bne	sync2
+
+	mov	r0, #0x1
+	ldr	r12, l2_smc3
+	dsb
+	smc	#0
+	dsb
+#endif
+skip_l2en:
+	/* We are back. Branch to the common CPU resume routine */
+	mov	r0, #0
+	ldr	pc, resume_addr
+ENDPROC(am43xx_resume_from_deep_sleep)
+
+/*
+ * Local variables
+ */
+	.align
+resume_addr:
+	.word	cpu_resume - PAGE_OFFSET + 0x80000000
+get_l2cache_base:
+	.word	omap4_get_l2cache_base
+kernel_flush:
+	.word   v7_flush_dcache_all
+ddr_start:
+	.word	PAGE_OFFSET
+
+am43xx_phys_emif_poweroff:
+	.word   (AM43XX_CM_BASE + AM43XX_PRM_DEVICE_INST + \
+		 AM43XX_PRM_EMIF_CTRL_OFFSET)
+am43xx_virt_mpu_clkstctrl:
+	.word	(AM43XX_CM_MPU_CLKSTCTRL)
+am43xx_virt_mpu_clkctrl:
+	.word	(AM43XX_CM_MPU_MPU_CLKCTRL)
+am43xx_virt_emif_clkctrl:
+	.word	(AM43XX_CM_PER_EMIF_CLKCTRL)
+am43xx_phys_emif_clkctrl:
+	.word	(AM43XX_CM_BASE + AM43XX_CM_PER_INST + \
+		 AM43XX_CM_PER_EMIF_CLKCTRL_OFFSET)
+
+/* L2 cache related defines for AM437x */
+l2_cache_base:
+	.word	OMAP44XX_L2CACHE_BASE
+l2_smc1:
+	.word	OMAP4_MON_L2X0_PREFETCH_INDEX
+l2_smc2:
+	.word	OMAP4_MON_L2X0_AUXCTRL_INDEX
+l2_smc3:
+	.word	OMAP4_MON_L2X0_CTRL_INDEX
+l2_val:
+	.word	0xffff
+
+.align 3
+/* DDR related defines */
+ENTRY(am43xx_emif_sram_table)
+	.space EMIF_PM_FUNCTIONS_SIZE
+
+ENTRY(am43xx_pm_sram)
+.word am43xx_do_wfi
+.word am43xx_do_wfi_sz
+.word am43xx_resume_offset
+.word am43xx_emif_sram_table
+.word am43xx_pm_ro_sram_data
+
+.align 3
+
+ENTRY(am43xx_pm_ro_sram_data)
+	.space AMX3_PM_RO_SRAM_DATA_SIZE
+
+ENTRY(am43xx_do_wfi_sz)
+	.word	. - am43xx_do_wfi
-- 
2.13.0

[PATCH v2 3/5] ARM: OMAP2+: pm33xx-core: Add platform code needed for PM

[Dave Gerlach]

Most of the PM code needed for am335x and am437x can be moved into a
module under drivers but some core code must remain in mach-omap2 at the
moment. This includes some internal clockdomain APIs and low-level ARM
APIs which are also not exported for use by modules.

Implement a few functions that handle these low-level platform
operations can be passed to the pm33xx module through the use of
platform data.

In addition to this, to be able to share data structures between C and
the sleep33xx and sleep43xx assembly code, we can automatically generate
all of the C struct member offsets and sizes as macros by making use of
the ARM asm-offsets file. In the same header that we define our data
structures in we also define all the macros in an inline function and by
adding a call to this in the asm_offsets file all macros are properly
generated and available to the assembly code without cluttering up the
asm-offsets file.

Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 arch/arm/kernel/asm-offsets.c        |   2 +
 arch/arm/mach-omap2/Kconfig          |   1 +
 arch/arm/mach-omap2/Makefile         |   4 +
 arch/arm/mach-omap2/pm.h             |   5 +
 arch/arm/mach-omap2/pm33xx-core.c    | 181 +++++++++++++++++++++++++++++++++++
 include/linux/platform_data/pm33xx.h |  69 +++++++++++++
 6 files changed, 262 insertions(+)
 create mode 100644 arch/arm/mach-omap2/pm33xx-core.c
 create mode 100644 include/linux/platform_data/pm33xx.h

diff --git a/arch/arm/kernel/asm-offsets.c b/arch/arm/kernel/asm-offsets.c
index d728b5660e36..62253e7bfac4 100644
--- a/arch/arm/kernel/asm-offsets.c
+++ b/arch/arm/kernel/asm-offsets.c
@@ -28,6 +28,7 @@
 #include <asm/vdso_datapage.h>
 #include <asm/hardware/cache-l2x0.h>
 #include <linux/kbuild.h>
+#include <linux/platform_data/pm33xx.h>
 #include <linux/ti-emif-sram.h>
 
 /*
@@ -187,6 +188,7 @@ int main(void)
 #if defined(CONFIG_SOC_AM33XX) || defined(CONFIG_SOC_AM43XX)
   BLANK();
   ti_emif_offsets();
+  amx3_pm_asm_offsets();
 #endif
 
   return 0; 
diff --git a/arch/arm/mach-omap2/Kconfig b/arch/arm/mach-omap2/Kconfig
index 0465338183c7..940173fa0992 100644
--- a/arch/arm/mach-omap2/Kconfig
+++ b/arch/arm/mach-omap2/Kconfig
@@ -72,6 +72,7 @@ config SOC_AM43XX
 	select ARM_ERRATA_754322
 	select ARM_ERRATA_775420
 	select OMAP_INTERCONNECT
+	select ARM_CPU_SUSPEND if PM
 
 config SOC_DRA7XX
 	bool "TI DRA7XX"
diff --git a/arch/arm/mach-omap2/Makefile b/arch/arm/mach-omap2/Makefile
index c89757abb0ae..8475ae3f9ff1 100644
--- a/arch/arm/mach-omap2/Makefile
+++ b/arch/arm/mach-omap2/Makefile
@@ -88,6 +88,8 @@ omap-4-5-pm-common			+= pm44xx.o
 obj-$(CONFIG_ARCH_OMAP4)		+= $(omap-4-5-pm-common)
 obj-$(CONFIG_SOC_OMAP5)			+= $(omap-4-5-pm-common)
 obj-$(CONFIG_SOC_DRA7XX)		+= $(omap-4-5-pm-common)
+obj-$(CONFIG_SOC_AM33XX)		+= pm33xx-core.o sleep33xx.o
+obj-$(CONFIG_SOC_AM43XX)		+= pm33xx-core.o sleep43xx.o
 obj-$(CONFIG_PM_DEBUG)			+= pm-debug.o
 
 obj-$(CONFIG_POWER_AVS_OMAP)		+= sr_device.o
@@ -95,6 +97,8 @@ obj-$(CONFIG_POWER_AVS_OMAP_CLASS3)    += smartreflex-class3.o
 
 AFLAGS_sleep24xx.o			:=-Wa,-march=armv6
 AFLAGS_sleep34xx.o			:=-Wa,-march=armv7-a$(plus_sec)
+AFLAGS_sleep33xx.o			:=-Wa,-march=armv7-a$(plus_sec)
+AFLAGS_sleep43xx.o			:=-Wa,-march=armv7-a$(plus_sec)
 
 endif
 
diff --git a/arch/arm/mach-omap2/pm.h b/arch/arm/mach-omap2/pm.h
index b668719b9b25..2f9649b89053 100644
--- a/arch/arm/mach-omap2/pm.h
+++ b/arch/arm/mach-omap2/pm.h
@@ -81,6 +81,11 @@ extern unsigned int omap3_do_wfi_sz;
 /* ... and its pointer from SRAM after copy */
 extern void (*omap3_do_wfi_sram)(void);
 
+struct am33xx_pm_platform_data *am33xx_pm_get_pdata(void);
+
+extern struct am33xx_pm_sram_addr am33xx_pm_sram;
+extern struct am33xx_pm_sram_addr am43xx_pm_sram;
+
 /* save_secure_ram_context function pointer and size, for copy to SRAM */
 extern int save_secure_ram_context(u32 *addr);
 extern unsigned int save_secure_ram_context_sz;
diff --git a/arch/arm/mach-omap2/pm33xx-core.c b/arch/arm/mach-omap2/pm33xx-core.c
new file mode 100644
index 000000000000..c84ffc4de2e9
--- /dev/null
+++ b/arch/arm/mach-omap2/pm33xx-core.c
@@ -0,0 +1,181 @@
+/*
+ * AM33XX Arch Power Management Routines
+ *
+ * Copyright (C) 2016-2017 Texas Instruments Incorporated - http://www.ti.com/
+ *	Dave Gerlach
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <asm/smp_scu.h>
+#include <asm/suspend.h>
+#include <linux/platform_data/pm33xx.h>
+
+#include "cm33xx.h"
+#include "common.h"
+#include "control.h"
+#include "clockdomain.h"
+#include "iomap.h"
+#include "omap_hwmod.h"
+#include "pm.h"
+#include "powerdomain.h"
+#include "prm33xx.h"
+#include "soc.h"
+#include "sram.h"
+
+static struct powerdomain *cefuse_pwrdm, *gfx_pwrdm, *per_pwrdm, *mpu_pwrdm;
+static struct clockdomain *gfx_l4ls_clkdm;
+static void __iomem *scu_base;
+
+static int __init am43xx_map_scu(void)
+{
+	scu_base = ioremap(scu_a9_get_base(), SZ_256);
+
+	if (!scu_base)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static int amx3_common_init(void)
+{
+	gfx_pwrdm = pwrdm_lookup("gfx_pwrdm");
+	per_pwrdm = pwrdm_lookup("per_pwrdm");
+	mpu_pwrdm = pwrdm_lookup("mpu_pwrdm");
+
+	if ((!gfx_pwrdm) || (!per_pwrdm) || (!mpu_pwrdm))
+		return -ENODEV;
+
+	(void)clkdm_for_each(omap_pm_clkdms_setup, NULL);
+
+	/* CEFUSE domain can be turned off post bootup */
+	cefuse_pwrdm = pwrdm_lookup("cefuse_pwrdm");
+	if (cefuse_pwrdm)
+		omap_set_pwrdm_state(cefuse_pwrdm, PWRDM_POWER_OFF);
+	else
+		pr_err("PM: Failed to get cefuse_pwrdm\n");
+
+	return 0;
+}
+
+static int am33xx_suspend_init(void)
+{
+	int ret;
+
+	gfx_l4ls_clkdm = clkdm_lookup("gfx_l4ls_gfx_clkdm");
+
+	if (!gfx_l4ls_clkdm) {
+		pr_err("PM: Cannot lookup gfx_l4ls_clkdm clockdomains\n");
+		return -ENODEV;
+	}
+
+	ret = amx3_common_init();
+
+	return ret;
+}
+
+static int am43xx_suspend_init(void)
+{
+	int ret = 0;
+
+	ret = am43xx_map_scu();
+	if (ret) {
+		pr_err("PM: Could not ioremap SCU\n");
+		return ret;
+	}
+
+	ret = amx3_common_init();
+
+	return ret;
+}
+
+static void amx3_pre_suspend_common(void)
+{
+	omap_set_pwrdm_state(gfx_pwrdm, PWRDM_POWER_OFF);
+}
+
+static void amx3_post_suspend_common(void)
+{
+	int status;
+	/*
+	 * Because gfx_pwrdm is the only one under MPU control,
+	 * comment on transition status
+	 */
+	status = pwrdm_read_pwrst(gfx_pwrdm);
+	if (status != PWRDM_POWER_OFF)
+		pr_err("PM: GFX domain did not transition: %x\n", status);
+}
+
+static int am33xx_suspend(unsigned int state, int (*fn)(unsigned long))
+{
+	int ret = 0;
+
+	amx3_pre_suspend_common();
+	ret = cpu_suspend(0, fn);
+	amx3_post_suspend_common();
+
+	/*
+	 * BUG: GFX_L4LS clock domain needs to be woken up to
+	 * ensure thet L4LS clock domain does not get stuck in
+	 * transition. If that happens L3 module does not get
+	 * disabled, thereby leading to PER power domain
+	 * transition failing
+	 */
+
+	clkdm_wakeup(gfx_l4ls_clkdm);
+	clkdm_sleep(gfx_l4ls_clkdm);
+
+	return ret;
+}
+
+static int am43xx_suspend(unsigned int state, int (*fn)(unsigned long))
+{
+	int ret = 0;
+
+	amx3_pre_suspend_common();
+	scu_power_mode(scu_base, SCU_PM_POWEROFF);
+	ret = cpu_suspend(0, fn);
+	scu_power_mode(scu_base, SCU_PM_NORMAL);
+	amx3_post_suspend_common();
+
+	return ret;
+}
+
+static struct am33xx_pm_sram_addr *amx3_get_sram_addrs(void)
+{
+	if (soc_is_am33xx())
+		return &am33xx_pm_sram;
+	else if (soc_is_am437x())
+		return &am43xx_pm_sram;
+	else
+		return NULL;
+}
+
+static struct am33xx_pm_platform_data am33xx_ops = {
+	.init = am33xx_suspend_init,
+	.soc_suspend = am33xx_suspend,
+	.get_sram_addrs = amx3_get_sram_addrs,
+};
+
+static struct am33xx_pm_platform_data am43xx_ops = {
+	.init = am43xx_suspend_init,
+	.soc_suspend = am43xx_suspend,
+	.get_sram_addrs = amx3_get_sram_addrs,
+};
+
+struct am33xx_pm_platform_data *am33xx_pm_get_pdata(void)
+{
+	if (soc_is_am33xx())
+		return &am33xx_ops;
+	else if (soc_is_am437x())
+		return &am43xx_ops;
+	else
+		return NULL;
+}
diff --git a/include/linux/platform_data/pm33xx.h b/include/linux/platform_data/pm33xx.h
new file mode 100644
index 000000000000..c191ab681093
--- /dev/null
+++ b/include/linux/platform_data/pm33xx.h
@@ -0,0 +1,69 @@
+/*
+ * TI pm33xx platform data
+ *
+ * Copyright (C) 2016-2017 Texas Instruments, Inc.
+ *	Dave Gerlach <d-gerlach@ti.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _LINUX_PLATFORM_DATA_PM33XX_H
+#define _LINUX_PLATFORM_DATA_PM33XX_H
+
+#include <linux/kbuild.h>
+
+#ifndef __ASSEMBLER__
+struct am33xx_pm_sram_addr {
+	void (*do_wfi)(void);
+	unsigned long *do_wfi_sz;
+	unsigned long *resume_offset;
+	unsigned long *emif_sram_table;
+	unsigned long *ro_sram_data;
+};
+
+struct am33xx_pm_platform_data {
+	int	(*init)(void);
+	int	(*soc_suspend)(unsigned int state, int (*fn)(unsigned long));
+	struct  am33xx_pm_sram_addr *(*get_sram_addrs)(void);
+};
+
+struct am33xx_pm_sram_data {
+	u32 wfi_flags;
+	u32 l2_aux_ctrl_val;
+	u32 l2_prefetch_ctrl_val;
+};
+
+struct am33xx_pm_ro_sram_data {
+	u32 amx3_pm_sram_data_virt;
+	u32 amx3_pm_sram_data_phys;
+};
+
+extern inline void amx3_pm_asm_offsets(void)
+{
+	DEFINE(AMX3_PM_WFI_FLAGS_OFFSET,
+	       offsetof(struct am33xx_pm_sram_data, wfi_flags));
+	DEFINE(AMX3_PM_L2_AUX_CTRL_VAL_OFFSET,
+	       offsetof(struct am33xx_pm_sram_data, l2_aux_ctrl_val));
+	DEFINE(AMX3_PM_L2_PREFETCH_CTRL_VAL_OFFSET,
+	       offsetof(struct am33xx_pm_sram_data, l2_prefetch_ctrl_val));
+	DEFINE(AMX3_PM_SRAM_DATA_SIZE, sizeof(struct am33xx_pm_sram_data));
+
+	BLANK();
+
+	DEFINE(AMX3_PM_RO_SRAM_DATA_VIRT_OFFSET,
+	       offsetof(struct am33xx_pm_ro_sram_data, amx3_pm_sram_data_virt));
+	DEFINE(AMX3_PM_RO_SRAM_DATA_PHYS_OFFSET,
+	       offsetof(struct am33xx_pm_ro_sram_data, amx3_pm_sram_data_phys));
+	DEFINE(AMX3_PM_RO_SRAM_DATA_SIZE,
+	       sizeof(struct am33xx_pm_ro_sram_data));
+}
+
+#endif /* __ASSEMBLER__ */
+#endif /* _LINUX_PLATFORM_DATA_PM33XX_H */
-- 
2.13.0

[PATCH v2 4/5] soc: ti: Add pm33xx driver for basic suspend support

[Dave Gerlach]

AM335x and AM437x support various low power modes as documented
in section 8.1.4.3 of the AM335x Technical Reference Manual and
section 6.4.3 of the AM437x Technical Reference Manual.

DeepSleep0 mode offers the lowest power mode with limited
wakeup sources without a system reboot and is mapped as
the suspend state in the kernel. In this state, MPU and
PER domains are turned off with the internal RAM held in
retention to facilitate the resume process. As part of
the boot process, the assembly code is copied over to OCMCRAM
so it can be executed to turn of the EMIF and put DDR into self
refresh.

Both platforms have a Cortex-M3 (WKUP_M3) which assists the MPU
in DeepSleep0 entry and exit. WKUP_M3 takes care
of the clockdomain and powerdomain transitions based on the
intended low power state. MPU needs to load the appropriate
WKUP_M3 binary onto the WKUP_M3 memory space before it can
leverage any of the PM features like DeepSleep. This loading
is handled by the remoteproc driver wkup_m3_rproc.

Communication with the WKUP_M3 is handled by a wkup_m3_ipc
driver that exposes the specific PM functionality to be used
the PM code.

In the current implementation when the suspend process
is initiated, MPU interrupts the WKUP_M3 to let it know about
the intent of entering DeepSleep0 and waits for an ACK. When
the ACK is received MPU continues with its suspend process
to suspend all the drivers and then jumps to assembly in
OCMC RAM. The assembly code puts the external RAM in self-refresh
mode, gates the MPU clock, and then finally executes the WFI
instruction. Execution of the WFI instruction with MPU clock gated
triggers another interrupt to the WKUP_M3 which then continues
with the power down sequence wherein the clockdomain and
powerdomain transition takes place. As part of the sleep sequence,
WKUP_M3 unmasks the interrupt lines for the wakeup sources. WFI
execution on WKUP_M3 causes the hardware to disable the main
oscillator of the SoC and from here system remains in sleep state
until a wake source brings the system into resume path.

When a wakeup event occurs, WKUP_M3 starts the power-up
sequence by switching on the power domains and finally
enabling the clock to MPU. Since the MPU gets powered down
as part of the sleep sequence in the resume path ROM code
starts executing. The ROM code detects a wakeup from sleep
and then jumps to the resume location in OCMC which was
populated in one of the IPC registers as part of the suspend
sequence.

Code is based on work by Vaibhav Bedia.

Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 drivers/soc/ti/Kconfig               |   9 +
 drivers/soc/ti/Makefile              |   1 +
 drivers/soc/ti/pm33xx.c              | 337 +++++++++++++++++++++++++++++++++++
 include/linux/platform_data/pm33xx.h |   4 +-
 4 files changed, 349 insertions(+), 2 deletions(-)
 create mode 100644 drivers/soc/ti/pm33xx.c

diff --git a/drivers/soc/ti/Kconfig b/drivers/soc/ti/Kconfig
index 39e152abe6b9..92770d84a288 100644
--- a/drivers/soc/ti/Kconfig
+++ b/drivers/soc/ti/Kconfig
@@ -28,6 +28,15 @@ config KEYSTONE_NAVIGATOR_DMA
 
 	  If unsure, say N.
 
+config AMX3_PM
+	tristate "AMx3 Power Management"
+	depends on SOC_AM33XX || SOC_AM43XX
+	depends on WKUP_M3_IPC && TI_EMIF_SRAM && SRAM
+	help
+	  Enable power management on AM335x and AM437x. Required for suspend to mem
+	  and standby states on both AM335x and AM437x platforms and for deeper cpuidle
+	  c-states on AM335x.
+
 config WKUP_M3_IPC
 	tristate "TI AMx3 Wkup-M3 IPC Driver"
 	depends on WKUP_M3_RPROC
diff --git a/drivers/soc/ti/Makefile b/drivers/soc/ti/Makefile
index 7d572736c86e..d9bd4ba424be 100644
--- a/drivers/soc/ti/Makefile
+++ b/drivers/soc/ti/Makefile
@@ -4,5 +4,6 @@
 obj-$(CONFIG_KEYSTONE_NAVIGATOR_QMSS)	+= knav_qmss.o
 knav_qmss-y := knav_qmss_queue.o knav_qmss_acc.o
 obj-$(CONFIG_KEYSTONE_NAVIGATOR_DMA)	+= knav_dma.o
+obj-$(CONFIG_AMX3_PM)			+= pm33xx.o
 obj-$(CONFIG_WKUP_M3_IPC)		+= wkup_m3_ipc.o
 obj-$(CONFIG_TI_SCI_PM_DOMAINS)		+= ti_sci_pm_domains.o
diff --git a/drivers/soc/ti/pm33xx.c b/drivers/soc/ti/pm33xx.c
new file mode 100644
index 000000000000..cfeb8df7e82a
--- /dev/null
+++ b/drivers/soc/ti/pm33xx.c
@@ -0,0 +1,337 @@
+/*
+ * AM33XX Power Management Routines
+ *
+ * Copyright (C) 2012-2017 Texas Instruments Incorporated - http://www.ti.com/
+ *	Vaibhav Bedia, Dave Gerlach
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/cpu.h>
+#include <linux/err.h>
+#include <linux/genalloc.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_data/pm33xx.h>
+#include <linux/platform_device.h>
+#include <linux/sizes.h>
+#include <linux/sram.h>
+#include <linux/suspend.h>
+#include <linux/ti-emif-sram.h>
+#include <linux/wkup_m3_ipc.h>
+
+#include <asm/proc-fns.h>
+#include <asm/suspend.h>
+#include <asm/system_misc.h>
+
+#define AMX3_PM_SRAM_SYMBOL_OFFSET(sym) ((unsigned long)(sym) - \
+					 (unsigned long)pm_sram->do_wfi)
+
+static int (*am33xx_do_wfi_sram)(unsigned long unused);
+static phys_addr_t am33xx_do_wfi_sram_phys;
+
+static struct gen_pool *sram_pool, *sram_pool_data;
+static unsigned long ocmcram_location, ocmcram_location_data;
+
+static struct am33xx_pm_platform_data *pm_ops;
+static struct am33xx_pm_sram_addr *pm_sram;
+
+static struct wkup_m3_ipc *m3_ipc;
+
+static u32 sram_suspend_address(unsigned long addr)
+{
+	return ((unsigned long)am33xx_do_wfi_sram +
+		AMX3_PM_SRAM_SYMBOL_OFFSET(addr));
+}
+
+#ifdef CONFIG_SUSPEND
+static int am33xx_pm_suspend(suspend_state_t suspend_state)
+{
+	int i, ret = 0;
+
+	ret = pm_ops->soc_suspend((unsigned long)suspend_state,
+				  am33xx_do_wfi_sram);
+
+	if (ret) {
+		pr_err("PM: Kernel suspend failure\n");
+	} else {
+		i = m3_ipc->ops->request_pm_status(m3_ipc);
+
+		switch (i) {
+		case 0:
+			pr_info("PM: Successfully put all powerdomains to target state\n");
+			break;
+		case 1:
+			pr_err("PM: Could not transition all powerdomains to target state\n");
+			ret = -1;
+			break;
+		default:
+			pr_err("PM: CM3 returned unknown result = %d\n", i);
+			ret = -1;
+		}
+	}
+
+	return ret;
+}
+
+static int am33xx_pm_enter(suspend_state_t suspend_state)
+{
+	int ret = 0;
+
+	switch (suspend_state) {
+	case PM_SUSPEND_MEM:
+	case PM_SUSPEND_STANDBY:
+		ret = am33xx_pm_suspend(suspend_state);
+		break;
+	default:
+		ret = -EINVAL;
+	}
+
+	return ret;
+}
+
+static int am33xx_pm_begin(suspend_state_t state)
+{
+	int ret = -EINVAL;
+
+	switch (state) {
+	case PM_SUSPEND_MEM:
+		ret = m3_ipc->ops->prepare_low_power(m3_ipc, WKUP_M3_DEEPSLEEP);
+		break;
+	case PM_SUSPEND_STANDBY:
+		ret = m3_ipc->ops->prepare_low_power(m3_ipc, WKUP_M3_STANDBY);
+		break;
+	}
+
+	return ret;
+}
+
+static void am33xx_pm_end(void)
+{
+	m3_ipc->ops->finish_low_power(m3_ipc);
+}
+
+static int am33xx_pm_valid(suspend_state_t state)
+{
+	switch (state) {
+	case PM_SUSPEND_STANDBY:
+	case PM_SUSPEND_MEM:
+		return 1;
+	default:
+		return 0;
+	}
+}
+
+static const struct platform_suspend_ops am33xx_pm_ops = {
+	.begin		= am33xx_pm_begin,
+	.end		= am33xx_pm_end,
+	.enter		= am33xx_pm_enter,
+	.valid		= am33xx_pm_valid,
+};
+#endif /* CONFIG_SUSPEND */
+
+static void am33xx_pm_set_ipc_ops(void)
+{
+	u32 resume_address;
+	int temp;
+
+	temp = ti_emif_get_mem_type();
+	if (temp < 0) {
+		pr_err("PM: Cannot determine memory type, no PM available\n");
+		return;
+	}
+	m3_ipc->ops->set_mem_type(m3_ipc, temp);
+
+	/* Physical resume address to be used by ROM code */
+	resume_address = am33xx_do_wfi_sram_phys +
+			 *pm_sram->resume_offset + 0x4;
+
+	m3_ipc->ops->set_resume_address(m3_ipc, (void *)resume_address);
+}
+
+static void am33xx_pm_free_sram(void)
+{
+	gen_pool_free(sram_pool, ocmcram_location, *pm_sram->do_wfi_sz);
+	gen_pool_free(sram_pool_data, ocmcram_location_data,
+		      sizeof(struct am33xx_pm_ro_sram_data));
+}
+
+/*
+ * Push the minimal suspend-resume code to SRAM
+ */
+static int am33xx_prepare_push_sram_idle(void)
+{
+	struct device_node *np;
+
+	np = of_find_compatible_node(NULL, NULL, "ti,omap3-mpu");
+
+	if (!np) {
+		np = of_find_compatible_node(NULL, NULL, "ti,omap4-mpu");
+		if (!np) {
+			pr_warn("PM: %s: Unable to find device node for mpu\n",
+				__func__);
+			return -ENODEV;
+		}
+	}
+
+	sram_pool = of_gen_pool_get(np, "pm-sram", 0);
+	if (!sram_pool) {
+		pr_warn("PM: %s: Unable to get sram pool for ocmcram\n",
+			__func__);
+		return -ENODEV;
+	}
+
+	sram_pool_data = of_gen_pool_get(np, "pm-sram", 1);
+	if (!sram_pool_data) {
+		pr_warn("PM: %s: Unable to get sram data pool for ocmcram\n",
+			__func__);
+		return -ENODEV;
+	}
+
+	ocmcram_location = gen_pool_alloc(sram_pool, *pm_sram->do_wfi_sz);
+	if (!ocmcram_location) {
+		pr_warn("PM: %s: Unable to allocate memory from ocmcram\n",
+			__func__);
+		return -ENOMEM;
+	}
+
+	ocmcram_location_data = gen_pool_alloc(sram_pool_data,
+					       sizeof(struct emif_regs_amx3));
+	if (!ocmcram_location_data) {
+		pr_err("PM: Unable to allocate memory from ocmcram\n");
+		gen_pool_free(sram_pool, ocmcram_location, *pm_sram->do_wfi_sz);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+static int am33xx_push_sram_idle(void)
+{
+	struct am33xx_pm_ro_sram_data ro_sram_data;
+	int ret;
+	void *copy_addr;
+
+	ro_sram_data.amx3_pm_sram_data_virt = ocmcram_location_data;
+	ro_sram_data.amx3_pm_sram_data_phys =
+		gen_pool_virt_to_phys(sram_pool_data, ocmcram_location_data);
+
+	/* Save physical address to calculate resume offset during pm init */
+	am33xx_do_wfi_sram_phys = gen_pool_virt_to_phys(sram_pool,
+							ocmcram_location);
+
+	am33xx_do_wfi_sram = sram_exec_copy(sram_pool, (void *)ocmcram_location,
+					    pm_sram->do_wfi,
+					    *pm_sram->do_wfi_sz);
+	if (!am33xx_do_wfi_sram) {
+		pr_err("PM: %s: am33xx_do_wfi copy to sram failed\n", __func__);
+		return -ENODEV;
+	}
+
+	ret = ti_emif_copy_pm_function_table(sram_pool,
+			(void *)sram_suspend_address((unsigned long)pm_sram->emif_sram_table));
+	if (ret) {
+		pr_warn("PM: %s: EMIF function copy failed\n", __func__);
+		return -EPROBE_DEFER;
+	}
+
+	copy_addr = sram_exec_copy(sram_pool,
+			(void *)sram_suspend_address((unsigned long)pm_sram->ro_sram_data),
+			&ro_sram_data,
+			sizeof(ro_sram_data));
+	if (!copy_addr) {
+		pr_err("PM: %s: ro_sram_data copy to sram failed\n", __func__);
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int am33xx_pm_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	if (!of_machine_is_compatible("ti,am33xx") &&
+	    !of_machine_is_compatible("ti,am43"))
+		return -ENODEV;
+
+	pm_ops = dev->platform_data;
+	if (!pm_ops) {
+		pr_err("PM: Cannot get core PM ops!\n");
+		return -ENODEV;
+	}
+
+	pm_sram = pm_ops->get_sram_addrs();
+	if (!pm_sram) {
+		pr_err("PM: Cannot get PM asm function addresses!!\n");
+		return -ENODEV;
+	}
+
+	ret = am33xx_prepare_push_sram_idle();
+	if (ret)
+		return ret;
+
+	ret = am33xx_push_sram_idle();
+	if (ret)
+		goto err_free_sram;
+
+	m3_ipc = wkup_m3_ipc_get();
+	if (!m3_ipc) {
+		pr_err("PM: Cannot get wkup_m3_ipc handle\n");
+		ret = -EPROBE_DEFER;
+		goto err_free_sram;
+	}
+
+	am33xx_pm_set_ipc_ops();
+
+#ifdef CONFIG_SUSPEND
+	suspend_set_ops(&am33xx_pm_ops);
+#endif /* CONFIG_SUSPEND */
+
+	ret = pm_ops->init();
+	if (ret) {
+		pr_err("Unable to call core pm init!\n");
+		ret = -ENODEV;
+		goto err_put_wkup_m3_ipc;
+	}
+
+	return 0;
+
+err_put_wkup_m3_ipc:
+	wkup_m3_ipc_put(m3_ipc);
+err_free_sram:
+	am33xx_pm_free_sram();
+	return ret;
+}
+
+static int am33xx_pm_remove(struct platform_device *pdev)
+{
+	suspend_set_ops(NULL);
+	wkup_m3_ipc_put(m3_ipc);
+	am33xx_pm_free_sram();
+	return 0;
+}
+
+static struct platform_driver am33xx_pm_driver = {
+	.driver = {
+		.name   = "pm33xx",
+	},
+	.probe = am33xx_pm_probe,
+	.remove = am33xx_pm_remove,
+};
+module_platform_driver(am33xx_pm_driver);
+
+MODULE_ALIAS("platform:pm33xx");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("am33xx power management driver");
diff --git a/include/linux/platform_data/pm33xx.h b/include/linux/platform_data/pm33xx.h
index c191ab681093..b8d6c3a9b46c 100644
--- a/include/linux/platform_data/pm33xx.h
+++ b/include/linux/platform_data/pm33xx.h
@@ -38,12 +38,12 @@ struct am33xx_pm_sram_data {
 	u32 wfi_flags;
 	u32 l2_aux_ctrl_val;
 	u32 l2_prefetch_ctrl_val;
-};
+} __packed __aligned(8);
 
 struct am33xx_pm_ro_sram_data {
 	u32 amx3_pm_sram_data_virt;
 	u32 amx3_pm_sram_data_phys;
-};
+} __packed __aligned(8);
 
 extern inline void amx3_pm_asm_offsets(void)
 {
-- 
2.13.0

[PATCH v2 5/5] ARM: OMAP2+: Create dummy platform_device for pm33xx

[Dave Gerlach]

Add amx3_common_pm_init to create a dummy platform_device for
pm33xx so that our pm33xx module can probe and am335x and am437x
platforms to enable basic suspend to mem and standby support.

Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
---
 arch/arm/mach-omap2/common.h      |  7 +++++++
 arch/arm/mach-omap2/io.c          |  2 ++
 arch/arm/mach-omap2/pm33xx-core.c | 12 ++++++++++++
 3 files changed, 21 insertions(+)

diff --git a/arch/arm/mach-omap2/common.h b/arch/arm/mach-omap2/common.h
index 3089d3bfa19b..cd7724d040f8 100644
--- a/arch/arm/mach-omap2/common.h
+++ b/arch/arm/mach-omap2/common.h
@@ -77,6 +77,13 @@ static inline int omap4_pm_init_early(void)
 }
 #endif
 
+#if defined(CONFIG_PM) && (defined(CONFIG_SOC_AM33XX) || \
+	defined(CONFIG_SOC_AM43XX))
+void amx3_common_pm_init(void);
+#else
+static inline void amx3_common_pm_init(void) { }
+#endif
+
 extern void omap2_init_common_infrastructure(void);
 
 extern void omap_init_time(void);
diff --git a/arch/arm/mach-omap2/io.c b/arch/arm/mach-omap2/io.c
index 5aafb8449c40..1bbbf19c93ea 100644
--- a/arch/arm/mach-omap2/io.c
+++ b/arch/arm/mach-omap2/io.c
@@ -652,6 +652,7 @@ void __init am33xx_init_early(void)
 void __init am33xx_init_late(void)
 {
 	omap_common_late_init();
+	amx3_common_pm_init();
 }
 #endif
 
@@ -676,6 +677,7 @@ void __init am43xx_init_late(void)
 {
 	omap_common_late_init();
 	omap2_clk_enable_autoidle_all();
+	amx3_common_pm_init();
 }
 #endif
 
diff --git a/arch/arm/mach-omap2/pm33xx-core.c b/arch/arm/mach-omap2/pm33xx-core.c
index c84ffc4de2e9..b5d90841c27a 100644
--- a/arch/arm/mach-omap2/pm33xx-core.c
+++ b/arch/arm/mach-omap2/pm33xx-core.c
@@ -179,3 +179,15 @@ struct am33xx_pm_platform_data *am33xx_pm_get_pdata(void)
 	else
 		return NULL;
 }
+
+void __init amx3_common_pm_init(void)
+{
+	struct platform_device_info devinfo = { };
+	struct am33xx_pm_platform_data *pdata;
+
+	pdata = am33xx_pm_get_pdata();
+	devinfo.name = "pm33xx";
+	devinfo.data = pdata;
+	devinfo.size_data = sizeof(*pdata);
+	platform_device_register_full(&devinfo);
+}
-- 
2.13.0

Re: [PATCH v2 3/5] ARM: OMAP2+: pm33xx-core: Add platform code needed for PM

[Tony Lindgren]

Hi,

* Dave Gerlach <d-gerlach@ti.com> [170519 13:08]:
> In addition to this, to be able to share data structures between C and
> the sleep33xx and sleep43xx assembly code, we can automatically generate
> all of the C struct member offsets and sizes as macros by making use of
> the ARM asm-offsets file. In the same header that we define our data
> structures in we also define all the macros in an inline function and by
> adding a call to this in the asm_offsets file all macros are properly
> generated and available to the assembly code without cluttering up the
> asm-offsets file.
...
> diff --git a/arch/arm/kernel/asm-offsets.c b/arch/arm/kernel/asm-offsets.c
> index d728b5660e36..62253e7bfac4 100644
> --- a/arch/arm/kernel/asm-offsets.c
> +++ b/arch/arm/kernel/asm-offsets.c
> @@ -28,6 +28,7 @@
>  #include <asm/vdso_datapage.h>
>  #include <asm/hardware/cache-l2x0.h>
>  #include <linux/kbuild.h>
> +#include <linux/platform_data/pm33xx.h>
>  #include <linux/ti-emif-sram.h>
>  
>  /*
> @@ -187,6 +188,7 @@ int main(void)
>  #if defined(CONFIG_SOC_AM33XX) || defined(CONFIG_SOC_AM43XX)
>    BLANK();
>    ti_emif_offsets();
> +  amx3_pm_asm_offsets();
>  #endif
>  
>    return 0; 

Russell, care to take a look at the above if you're OK with it or
if you prefer to do it some other way?

Also please see thread "[PATCH v2 0/2] memory: Introduce ti-emif-sram
driver" for similar changes.

Thanks,

Tony

Re: [PATCH v2 4/5] soc: ti: Add pm33xx driver for basic suspend support

[Johan Hovold]

On Fri, May 19, 2017 at 03:04:37PM -0500, Dave Gerlach wrote:
> AM335x and AM437x support various low power modes as documented
> in section 8.1.4.3 of the AM335x Technical Reference Manual and
> section 6.4.3 of the AM437x Technical Reference Manual.
> 
> DeepSleep0 mode offers the lowest power mode with limited
> wakeup sources without a system reboot and is mapped as
> the suspend state in the kernel. In this state, MPU and
> PER domains are turned off with the internal RAM held in
> retention to facilitate the resume process. As part of
> the boot process, the assembly code is copied over to OCMCRAM
> so it can be executed to turn of the EMIF and put DDR into self
> refresh.
> 
> Both platforms have a Cortex-M3 (WKUP_M3) which assists the MPU
> in DeepSleep0 entry and exit. WKUP_M3 takes care
> of the clockdomain and powerdomain transitions based on the
> intended low power state. MPU needs to load the appropriate
> WKUP_M3 binary onto the WKUP_M3 memory space before it can
> leverage any of the PM features like DeepSleep. This loading
> is handled by the remoteproc driver wkup_m3_rproc.
> 
> Communication with the WKUP_M3 is handled by a wkup_m3_ipc
> driver that exposes the specific PM functionality to be used
> the PM code.

> +static void am33xx_pm_free_sram(void)
> +{
> +	gen_pool_free(sram_pool, ocmcram_location, *pm_sram->do_wfi_sz);
> +	gen_pool_free(sram_pool_data, ocmcram_location_data,
> +		      sizeof(struct am33xx_pm_ro_sram_data));
> +}
> +
> +/*
> + * Push the minimal suspend-resume code to SRAM
> + */
> +static int am33xx_prepare_push_sram_idle(void)
> +{
> +	struct device_node *np;
> +
> +	np = of_find_compatible_node(NULL, NULL, "ti,omap3-mpu");
> +

Stray newline.

> +	if (!np) {
> +		np = of_find_compatible_node(NULL, NULL, "ti,omap4-mpu");
> +		if (!np) {
> +			pr_warn("PM: %s: Unable to find device node for mpu\n",
> +				__func__);
> +			return -ENODEV;
> +		}
> +	}

You never put the reference to np you acquire here.

[snip] 

> +static int am33xx_push_sram_idle(void)
> +{
> +	struct am33xx_pm_ro_sram_data ro_sram_data;
> +	int ret;
> +	void *copy_addr;
> +
> +	ro_sram_data.amx3_pm_sram_data_virt = ocmcram_location_data;
> +	ro_sram_data.amx3_pm_sram_data_phys =
> +		gen_pool_virt_to_phys(sram_pool_data, ocmcram_location_data);
> +
> +	/* Save physical address to calculate resume offset during pm init */
> +	am33xx_do_wfi_sram_phys = gen_pool_virt_to_phys(sram_pool,
> +							ocmcram_location);
> +
> +	am33xx_do_wfi_sram = sram_exec_copy(sram_pool, (void *)ocmcram_location,
> +					    pm_sram->do_wfi,
> +					    *pm_sram->do_wfi_sz);
> +	if (!am33xx_do_wfi_sram) {
> +		pr_err("PM: %s: am33xx_do_wfi copy to sram failed\n", __func__);
> +		return -ENODEV;
> +	}
> +
> +	ret = ti_emif_copy_pm_function_table(sram_pool,
> +			(void *)sram_suspend_address((unsigned long)pm_sram->emif_sram_table));
> +	if (ret) {
> +		pr_warn("PM: %s: EMIF function copy failed\n", __func__);
> +		return -EPROBE_DEFER;
> +	}

Here's the dependency to the emif device I commented on earlier (and
below).

> +
> +	copy_addr = sram_exec_copy(sram_pool,
> +			(void *)sram_suspend_address((unsigned long)pm_sram->ro_sram_data),
> +			&ro_sram_data,
> +			sizeof(ro_sram_data));
> +	if (!copy_addr) {
> +		pr_err("PM: %s: ro_sram_data copy to sram failed\n", __func__);
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int am33xx_pm_probe(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	int ret;
> +
> +	if (!of_machine_is_compatible("ti,am33xx") &&
> +	    !of_machine_is_compatible("ti,am43"))
> +		return -ENODEV;
> +
> +	pm_ops = dev->platform_data;
> +	if (!pm_ops) {
> +		pr_err("PM: Cannot get core PM ops!\n");
> +		return -ENODEV;
> +	}
> +
> +	pm_sram = pm_ops->get_sram_addrs();
> +	if (!pm_sram) {
> +		pr_err("PM: Cannot get PM asm function addresses!!\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = am33xx_prepare_push_sram_idle();

Perhaps calling this one am33xx_pm_alloc_sram() would be more
descriptive (and match the release function)?

> +	if (ret)
> +		return ret;
> +
> +	ret = am33xx_push_sram_idle();
> +	if (ret)
> +		goto err_free_sram;

As I mentioned in my comments to the emif-sram driver, you may need to
create device link to the emif-sram device to prevent it from going away
under you here.

> +
> +	m3_ipc = wkup_m3_ipc_get();
> +	if (!m3_ipc) {
> +		pr_err("PM: Cannot get wkup_m3_ipc handle\n");

You shouldn't log this as an error when probe is being deferred. 

Why not use dev_err and friends for logging now that you have a struct
device?

And similarly to the emif-sram device, you may need to create a
device-link also to the ipc device to prevent its driver from being
unbound.

> +		ret = -EPROBE_DEFER;
> +		goto err_free_sram;
> +	}
> +
> +	am33xx_pm_set_ipc_ops();
> +
> +#ifdef CONFIG_SUSPEND
> +	suspend_set_ops(&am33xx_pm_ops);
> +#endif /* CONFIG_SUSPEND */

This renders a lockdep splash about a circular locking dependency when
suspending since we're taking the pm_mutex in suspend_set_ops here, and
during suspend we flush any deferred probes while already holding the
mutex:

 ======================================================
 WARNING: possible circular locking dependency detected
 4.12.0-rc7 #11 Not tainted
 ------------------------------------------------------
 bash/404 is trying to acquire lock:
  (deferred_probe_work){+.+.+.}, at: [<c014cf3c>] flush_work+0x30/0x27c
 
 but task is already holding lock:
  (pm_mutex){+.+...}, at: [<c01792dc>] pm_suspend+0x190/0xc94
 
 which lock already depends on the new lock.
 
 
 the existing dependency chain (in reverse order) is:
 
 -> #1 (pm_mutex){+.+...}:
        __mutex_lock+0x80/0x694
        mutex_lock_nested+0x2c/0x34
        suspend_set_ops+0x4c/0x128
        am33xx_pm_probe+0x1fc/0x3a8
        platform_drv_probe+0x5c/0xc0
        driver_probe_device+0x37c/0x490
        __device_attach_driver+0xac/0x128
        bus_for_each_drv+0x74/0xa8
        __device_attach+0xc4/0x154
        device_initial_probe+0x1c/0x20
        bus_probe_device+0x98/0xa0
        deferred_probe_work_func+0x4c/0xe4
        process_one_work+0x1f4/0x758
        worker_thread+0x1e0/0x514
        kthread+0x128/0x158
        ret_from_fork+0x14/0x24
 
 -> #0 (deferred_probe_work){+.+.+.}:
        lock_acquire+0x108/0x264
        flush_work+0x60/0x27c
        wait_for_device_probe+0x24/0xa4
        dpm_prepare+0xd0/0x91c
        dpm_suspend_start+0x1c/0x70
        suspend_devices_and_enter+0xc4/0xeac
        pm_suspend+0x890/0xc94
        state_store+0x80/0xdc
        kobj_attr_store+0x1c/0x28
        sysfs_kf_write+0x5c/0x60
        kernfs_fop_write+0x128/0x254
        __vfs_write+0x38/0x128
        vfs_write+0xb4/0x174
        SyS_write+0x54/0xb0
        ret_fast_syscall+0x0/0x1c

Johan

Re: [PATCH v2 5/5] ARM: OMAP2+: Create dummy platform_device for pm33xx

[Johan Hovold]

On Fri, May 19, 2017 at 03:04:38PM -0500, Dave Gerlach wrote:
> Add amx3_common_pm_init to create a dummy platform_device for
> pm33xx so that our pm33xx module can probe and am335x and am437x
> platforms to enable basic suspend to mem and standby support.
> 
> Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
> ---

> +void __init amx3_common_pm_init(void)
> +{
> +	struct platform_device_info devinfo = { };

You should make sure devinfo is zeroed, and also set .id to -1. 

> +	struct am33xx_pm_platform_data *pdata;
> +
> +	pdata = am33xx_pm_get_pdata();
> +	devinfo.name = "pm33xx";
> +	devinfo.data = pdata;
> +	devinfo.size_data = sizeof(*pdata);
> +	platform_device_register_full(&devinfo);
> +}

Johan

Re: [PATCH v2 3/5] ARM: OMAP2+: pm33xx-core: Add platform code needed for PM

[Johan Hovold]

On Fri, May 19, 2017 at 03:04:36PM -0500, Dave Gerlach wrote:
> Most of the PM code needed for am335x and am437x can be moved into a
> module under drivers but some core code must remain in mach-omap2 at the
> moment. This includes some internal clockdomain APIs and low-level ARM
> APIs which are also not exported for use by modules.
> 
> Implement a few functions that handle these low-level platform
> operations can be passed to the pm33xx module through the use of
> platform data.
> 
> In addition to this, to be able to share data structures between C and
> the sleep33xx and sleep43xx assembly code, we can automatically generate
> all of the C struct member offsets and sizes as macros by making use of
> the ARM asm-offsets file. In the same header that we define our data
> structures in we also define all the macros in an inline function and by
> adding a call to this in the asm_offsets file all macros are properly
> generated and available to the assembly code without cluttering up the
> asm-offsets file.
> 
> Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
> ---

> diff --git a/arch/arm/mach-omap2/pm.h b/arch/arm/mach-omap2/pm.h
> index b668719b9b25..2f9649b89053 100644
> --- a/arch/arm/mach-omap2/pm.h
> +++ b/arch/arm/mach-omap2/pm.h
> @@ -81,6 +81,11 @@ extern unsigned int omap3_do_wfi_sz;
>  /* ... and its pointer from SRAM after copy */
>  extern void (*omap3_do_wfi_sram)(void);
>  
> +struct am33xx_pm_platform_data *am33xx_pm_get_pdata(void);

This one is not used outside of pm33xx-core.c so can now be static, and
this declaration can be dropped.

> diff --git a/arch/arm/mach-omap2/pm33xx-core.c b/arch/arm/mach-omap2/pm33xx-core.c
> new file mode 100644
> index 000000000000..c84ffc4de2e9
> --- /dev/null
> +++ b/arch/arm/mach-omap2/pm33xx-core.c
> @@ -0,0 +1,181 @@
> +/*
> + * AM33XX Arch Power Management Routines
> + *
> + * Copyright (C) 2016-2017 Texas Instruments Incorporated - http://www.ti.com/
> + *	Dave Gerlach
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation version 2.
> + *
> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
> + * kind, whether express or implied; without even the implied warranty
> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#include <asm/smp_scu.h>

I get a compilation error here when compiling for am335x without
CONFIG_HAVE_ARM_SCU due to a missing errno.h include:

In file included from /home/johan/work/omicron/src/linux/arch/arm/mach-omap2/pm33xx-core.c:17:0:
/home/johan/work/omicron/src/linux/arch/arm/include/asm/smp_scu.h: In function 'scu_power_mode':
/home/johan/work/omicron/src/linux/arch/arm/include/asm/smp_scu.h:36:10: error: 'EINVAL' undeclared (first use in this function)
  return -EINVAL;
          ^
/home/johan/work/omicron/src/linux/arch/arm/include/asm/smp_scu.h:36:10: note: each undeclared identifier is reported only once for each function it appears in

This is arguably a bug in the header, which I'm submitting a fix for,
but you should include errno.h above anyway as you use its definitions
below as well.

> +#include <asm/suspend.h>
> +#include <linux/platform_data/pm33xx.h>

<snip>

> diff --git a/include/linux/platform_data/pm33xx.h b/include/linux/platform_data/pm33xx.h
> new file mode 100644
> index 000000000000..c191ab681093
> --- /dev/null
> +++ b/include/linux/platform_data/pm33xx.h
> @@ -0,0 +1,69 @@
> +/*
> + * TI pm33xx platform data
> + *
> + * Copyright (C) 2016-2017 Texas Instruments, Inc.
> + *	Dave Gerlach <d-gerlach@ti.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + */
> +
> +#ifndef _LINUX_PLATFORM_DATA_PM33XX_H
> +#define _LINUX_PLATFORM_DATA_PM33XX_H
> +

And here you should add a linux/types.h include to make this header
self-contained. Right now you are depending on the scu header to pull in
the types for pm33xx-core.c above.

> +#include <linux/kbuild.h>
> +
> +#ifndef __ASSEMBLER__
> +struct am33xx_pm_sram_addr {
> +	void (*do_wfi)(void);
> +	unsigned long *do_wfi_sz;
> +	unsigned long *resume_offset;
> +	unsigned long *emif_sram_table;
> +	unsigned long *ro_sram_data;
> +};
> +
> +struct am33xx_pm_platform_data {
> +	int	(*init)(void);
> +	int	(*soc_suspend)(unsigned int state, int (*fn)(unsigned long));
> +	struct  am33xx_pm_sram_addr *(*get_sram_addrs)(void);
> +};
> +
> +struct am33xx_pm_sram_data {
> +	u32 wfi_flags;
> +	u32 l2_aux_ctrl_val;
> +	u32 l2_prefetch_ctrl_val;
> +};
> +
> +struct am33xx_pm_ro_sram_data {
> +	u32 amx3_pm_sram_data_virt;
> +	u32 amx3_pm_sram_data_phys;
> +};

Johan

Re: [PATCH v2 4/5] soc: ti: Add pm33xx driver for basic suspend support

[Johan Hovold]

On Mon, Jul 03, 2017 at 06:54:19PM +0200, Johan Hovold wrote:
> On Fri, May 19, 2017 at 03:04:37PM -0500, Dave Gerlach wrote:

> > +static int am33xx_pm_probe(struct platform_device *pdev)

> > +#ifdef CONFIG_SUSPEND
> > +	suspend_set_ops(&am33xx_pm_ops);
> > +#endif /* CONFIG_SUSPEND */
> 
> This renders a lockdep splash about a circular locking dependency when
> suspending since we're taking the pm_mutex in suspend_set_ops here, and
> during suspend we flush any deferred probes while already holding the
> mutex:

Here's the full splat against 4.12:

 ======================================================
 WARNING: possible circular locking dependency detected
 4.12.0 #30 Not tainted
 ------------------------------------------------------
 bash/404 is trying to acquire lock:
  (deferred_probe_work){+.+.+.}, at: [<c014cf3c>] flush_work+0x30/0x27c
 
 but task is already holding lock:
  (pm_mutex){+.+...}, at: [<c01792dc>] pm_suspend+0x190/0xc94
 
 which lock already depends on the new lock.
 
 
 the existing dependency chain (in reverse order) is:
 
 -> #1 (pm_mutex){+.+...}:
        __mutex_lock+0x80/0x694
        mutex_lock_nested+0x2c/0x34
        suspend_set_ops+0x4c/0x128
        am33xx_pm_probe+0x1fc/0x3a8
        platform_drv_probe+0x5c/0xc0
        driver_probe_device+0x37c/0x490
        __device_attach_driver+0xac/0x128
        bus_for_each_drv+0x74/0xa8
        __device_attach+0xc4/0x154
        device_initial_probe+0x1c/0x20
        bus_probe_device+0x98/0xa0
        deferred_probe_work_func+0x4c/0xe4
        process_one_work+0x1f4/0x758
        worker_thread+0x1e0/0x514
        kthread+0x128/0x158
        ret_from_fork+0x14/0x24
 
 -> #0 (deferred_probe_work){+.+.+.}:
        lock_acquire+0x108/0x264
        flush_work+0x60/0x27c
        wait_for_device_probe+0x24/0xa4
        dpm_prepare+0xd0/0x91c
        dpm_suspend_start+0x1c/0x70
        suspend_devices_and_enter+0xc4/0xeac
        pm_suspend+0x890/0xc94
        state_store+0x80/0xdc
        kobj_attr_store+0x1c/0x28
        sysfs_kf_write+0x5c/0x60
        kernfs_fop_write+0x128/0x254
        __vfs_write+0x38/0x128
        vfs_write+0xb4/0x174
        SyS_write+0x54/0xb0
        ret_fast_syscall+0x0/0x1c
 
 other info that might help us debug this:
 
  Possible unsafe locking scenario:
 
        CPU0                    CPU1
        ----                    ----
   lock(pm_mutex);
                                lock(deferred_probe_work);
                                lock(pm_mutex);
   lock(deferred_probe_work);
 
  *** DEADLOCK ***
 
 4 locks held by bash/404:
  #0:  (sb_writers#6){.+.+.+}, at: [<c0244fac>] vfs_write+0x160/0x174
  #1:  (&of->mutex){+.+.+.}, at: [<c02bab64>] kernfs_fop_write+0xe4/0x254
  #2:  (s_active#99){.+.+.+}, at: [<c02bab6c>] kernfs_fop_write+0xec/0x254
  #3:  (pm_mutex){+.+...}, at: [<c01792dc>] pm_suspend+0x190/0xc94
 
 stack backtrace:
 CPU: 0 PID: 404 Comm: bash Not tainted 4.12.0 #30
 Hardware name: Generic AM33XX (Flattened Device Tree)
 [<c011192c>] (unwind_backtrace) from [<c010e104>] (show_stack+0x20/0x24)
 [<c010e104>] (show_stack) from [<c03f2da0>] (dump_stack+0x24/0x28)
 [<c03f2da0>] (dump_stack) from [<c016e070>] (print_circular_bug+0x20c/0x334)
 [<c016e070>] (print_circular_bug) from [<c0171660>] (__lock_acquire+0x1bf4/0x1c08)
 [<c0171660>] (__lock_acquire) from [<c0172000>] (lock_acquire+0x108/0x264)
 [<c0172000>] (lock_acquire) from [<c014cf6c>] (flush_work+0x60/0x27c)
 [<c014cf6c>] (flush_work) from [<c04a7524>] (wait_for_device_probe+0x24/0xa4)
 [<c04a7524>] (wait_for_device_probe) from [<c04bae20>] (dpm_prepare+0xd0/0x91c)
 [<c04bae20>] (dpm_prepare) from [<c04bb688>] (dpm_suspend_start+0x1c/0x70)
 [<c04bb688>] (dpm_suspend_start) from [<c0178364>] (suspend_devices_and_enter+0xc4/0xeac)
 [<c0178364>] (suspend_devices_and_enter) from [<c01799dc>] (pm_suspend+0x890/0xc94)
 [<c01799dc>] (pm_suspend) from [<c01773f0>] (state_store+0x80/0xdc)
 [<c01773f0>] (state_store) from [<c03f442c>] (kobj_attr_store+0x1c/0x28)
 [<c03f442c>] (kobj_attr_store) from [<c02bb858>] (sysfs_kf_write+0x5c/0x60)
 [<c02bb858>] (sysfs_kf_write) from [<c02baba8>] (kernfs_fop_write+0x128/0x254)
 [<c02baba8>] (kernfs_fop_write) from [<c0243830>] (__vfs_write+0x38/0x128)
 [<c0243830>] (__vfs_write) from [<c0244f00>] (vfs_write+0xb4/0x174)
 [<c0244f00>] (vfs_write) from [<c0245e08>] (SyS_write+0x54/0xb0)
 [<c0245e08>] (SyS_write) from [<c01092e0>] (ret_fast_syscall+0x0/0x1c)

Johan

Re: [PATCH v2 3/5] ARM: OMAP2+: pm33xx-core: Add platform code needed for PM

[Dave Gerlach]

On 07/04/2017 08:14 AM, Johan Hovold wrote:
> On Fri, May 19, 2017 at 03:04:36PM -0500, Dave Gerlach wrote:
>> Most of the PM code needed for am335x and am437x can be moved into a
>> module under drivers but some core code must remain in mach-omap2 at the
>> moment. This includes some internal clockdomain APIs and low-level ARM
>> APIs which are also not exported for use by modules.
>>
>> Implement a few functions that handle these low-level platform
>> operations can be passed to the pm33xx module through the use of
>> platform data.
>>
>> In addition to this, to be able to share data structures between C and
>> the sleep33xx and sleep43xx assembly code, we can automatically generate
>> all of the C struct member offsets and sizes as macros by making use of
>> the ARM asm-offsets file. In the same header that we define our data
>> structures in we also define all the macros in an inline function and by
>> adding a call to this in the asm_offsets file all macros are properly
>> generated and available to the assembly code without cluttering up the
>> asm-offsets file.
>>
>> Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
>> ---
> 
>> diff --git a/arch/arm/mach-omap2/pm.h b/arch/arm/mach-omap2/pm.h
>> index b668719b9b25..2f9649b89053 100644
>> --- a/arch/arm/mach-omap2/pm.h
>> +++ b/arch/arm/mach-omap2/pm.h
>> @@ -81,6 +81,11 @@ extern unsigned int omap3_do_wfi_sz;
>>  /* ... and its pointer from SRAM after copy */
>>  extern void (*omap3_do_wfi_sram)(void);
>>  
>> +struct am33xx_pm_platform_data *am33xx_pm_get_pdata(void);
> 
> This one is not used outside of pm33xx-core.c so can now be static, and
> this declaration can be dropped.

Yes you are correct, seems I missed this in a refactor.

> 
>> diff --git a/arch/arm/mach-omap2/pm33xx-core.c b/arch/arm/mach-omap2/pm33xx-core.c
>> new file mode 100644
>> index 000000000000..c84ffc4de2e9
>> --- /dev/null
>> +++ b/arch/arm/mach-omap2/pm33xx-core.c
>> @@ -0,0 +1,181 @@
>> +/*
>> + * AM33XX Arch Power Management Routines
>> + *
>> + * Copyright (C) 2016-2017 Texas Instruments Incorporated - http://www.ti.com/
>> + *	Dave Gerlach
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU General Public License as
>> + * published by the Free Software Foundation version 2.
>> + *
>> + * This program is distributed "as is" WITHOUT ANY WARRANTY of any
>> + * kind, whether express or implied; without even the implied warranty
>> + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + */
>> +
>> +#include <asm/smp_scu.h>
> 
> I get a compilation error here when compiling for am335x without
> CONFIG_HAVE_ARM_SCU due to a missing errno.h include:
> 
> In file included from /home/johan/work/omicron/src/linux/arch/arm/mach-omap2/pm33xx-core.c:17:0:
> /home/johan/work/omicron/src/linux/arch/arm/include/asm/smp_scu.h: In function 'scu_power_mode':
> /home/johan/work/omicron/src/linux/arch/arm/include/asm/smp_scu.h:36:10: error: 'EINVAL' undeclared (first use in this function)
>   return -EINVAL;
>           ^
> /home/johan/work/omicron/src/linux/arch/arm/include/asm/smp_scu.h:36:10: note: each undeclared identifier is reported only once for each function it appears in
> 
> This is arguably a bug in the header, which I'm submitting a fix for,
> but you should include errno.h above anyway as you use its definitions
> below as well.
> 

Yes ok, good catch.

>> +#include <asm/suspend.h>
>> +#include <linux/platform_data/pm33xx.h>
> 
> <snip>
> 
>> diff --git a/include/linux/platform_data/pm33xx.h b/include/linux/platform_data/pm33xx.h
>> new file mode 100644
>> index 000000000000..c191ab681093
>> --- /dev/null
>> +++ b/include/linux/platform_data/pm33xx.h
>> @@ -0,0 +1,69 @@
>> +/*
>> + * TI pm33xx platform data
>> + *
>> + * Copyright (C) 2016-2017 Texas Instruments, Inc.
>> + *	Dave Gerlach <d-gerlach@ti.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU General Public License for more details.
>> + */
>> +
>> +#ifndef _LINUX_PLATFORM_DATA_PM33XX_H
>> +#define _LINUX_PLATFORM_DATA_PM33XX_H
>> +
> 
> And here you should add a linux/types.h include to make this header
> self-contained. Right now you are depending on the scu header to pull in
> the types for pm33xx-core.c above.
> 

Ok will do. Thanks for the comments.

Regards,
Dave

>> +#include <linux/kbuild.h>
>> +
>> +#ifndef __ASSEMBLER__
>> +struct am33xx_pm_sram_addr {
>> +	void (*do_wfi)(void);
>> +	unsigned long *do_wfi_sz;
>> +	unsigned long *resume_offset;
>> +	unsigned long *emif_sram_table;
>> +	unsigned long *ro_sram_data;
>> +};
>> +
>> +struct am33xx_pm_platform_data {
>> +	int	(*init)(void);
>> +	int	(*soc_suspend)(unsigned int state, int (*fn)(unsigned long));
>> +	struct  am33xx_pm_sram_addr *(*get_sram_addrs)(void);
>> +};
>> +
>> +struct am33xx_pm_sram_data {
>> +	u32 wfi_flags;
>> +	u32 l2_aux_ctrl_val;
>> +	u32 l2_prefetch_ctrl_val;
>> +};
>> +
>> +struct am33xx_pm_ro_sram_data {
>> +	u32 amx3_pm_sram_data_virt;
>> +	u32 amx3_pm_sram_data_phys;
>> +};
> 
> Johan
> 

Re: [PATCH v2 4/5] soc: ti: Add pm33xx driver for basic suspend support

[Dave Gerlach]

On 07/03/2017 11:54 AM, Johan Hovold wrote:
> On Fri, May 19, 2017 at 03:04:37PM -0500, Dave Gerlach wrote:
>> AM335x and AM437x support various low power modes as documented
>> in section 8.1.4.3 of the AM335x Technical Reference Manual and
>> section 6.4.3 of the AM437x Technical Reference Manual.
>>
>> DeepSleep0 mode offers the lowest power mode with limited
>> wakeup sources without a system reboot and is mapped as
>> the suspend state in the kernel. In this state, MPU and
>> PER domains are turned off with the internal RAM held in
>> retention to facilitate the resume process. As part of
>> the boot process, the assembly code is copied over to OCMCRAM
>> so it can be executed to turn of the EMIF and put DDR into self
>> refresh.
>>
>> Both platforms have a Cortex-M3 (WKUP_M3) which assists the MPU
>> in DeepSleep0 entry and exit. WKUP_M3 takes care
>> of the clockdomain and powerdomain transitions based on the
>> intended low power state. MPU needs to load the appropriate
>> WKUP_M3 binary onto the WKUP_M3 memory space before it can
>> leverage any of the PM features like DeepSleep. This loading
>> is handled by the remoteproc driver wkup_m3_rproc.
>>
>> Communication with the WKUP_M3 is handled by a wkup_m3_ipc
>> driver that exposes the specific PM functionality to be used
>> the PM code.
> 
>> +static void am33xx_pm_free_sram(void)
>> +{
>> +	gen_pool_free(sram_pool, ocmcram_location, *pm_sram->do_wfi_sz);
>> +	gen_pool_free(sram_pool_data, ocmcram_location_data,
>> +		      sizeof(struct am33xx_pm_ro_sram_data));
>> +}
>> +
>> +/*
>> + * Push the minimal suspend-resume code to SRAM
>> + */
>> +static int am33xx_prepare_push_sram_idle(void)
>> +{
>> +	struct device_node *np;
>> +
>> +	np = of_find_compatible_node(NULL, NULL, "ti,omap3-mpu");
>> +
> 
> Stray newline.
> 

Yes thanks.

>> +	if (!np) {
>> +		np = of_find_compatible_node(NULL, NULL, "ti,omap4-mpu");
>> +		if (!np) {
>> +			pr_warn("PM: %s: Unable to find device node for mpu\n",
>> +				__func__);
>> +			return -ENODEV;
>> +		}
>> +	}
> 
> You never put the reference to np you acquire here.

Whoops, it seems I did not, will fix.

> 
> [snip] 
> 
>> +static int am33xx_push_sram_idle(void)
>> +{
>> +	struct am33xx_pm_ro_sram_data ro_sram_data;
>> +	int ret;
>> +	void *copy_addr;
>> +
>> +	ro_sram_data.amx3_pm_sram_data_virt = ocmcram_location_data;
>> +	ro_sram_data.amx3_pm_sram_data_phys =
>> +		gen_pool_virt_to_phys(sram_pool_data, ocmcram_location_data);
>> +
>> +	/* Save physical address to calculate resume offset during pm init */
>> +	am33xx_do_wfi_sram_phys = gen_pool_virt_to_phys(sram_pool,
>> +							ocmcram_location);
>> +
>> +	am33xx_do_wfi_sram = sram_exec_copy(sram_pool, (void *)ocmcram_location,
>> +					    pm_sram->do_wfi,
>> +					    *pm_sram->do_wfi_sz);
>> +	if (!am33xx_do_wfi_sram) {
>> +		pr_err("PM: %s: am33xx_do_wfi copy to sram failed\n", __func__);
>> +		return -ENODEV;
>> +	}
>> +
>> +	ret = ti_emif_copy_pm_function_table(sram_pool,
>> +			(void *)sram_suspend_address((unsigned long)pm_sram->emif_sram_table));
>> +	if (ret) {
>> +		pr_warn("PM: %s: EMIF function copy failed\n", __func__);
>> +		return -EPROBE_DEFER;
>> +	}
> 
> Here's the dependency to the emif device I commented on earlier (and
> below).
> 

I commented on this in the ti-emif-pm thread but we should be ok, we can't
remove that driver while pm33xx is loaded because of the direct call of exported
symbols, pm33xx holds a reference to ti-emif-pm until it is unloaded. I will
make sure the confirmation that these functions are valid is solid though.

>> +
>> +	copy_addr = sram_exec_copy(sram_pool,
>> +			(void *)sram_suspend_address((unsigned long)pm_sram->ro_sram_data),
>> +			&ro_sram_data,
>> +			sizeof(ro_sram_data));
>> +	if (!copy_addr) {
>> +		pr_err("PM: %s: ro_sram_data copy to sram failed\n", __func__);
>> +		return -ENODEV;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static int am33xx_pm_probe(struct platform_device *pdev)
>> +{
>> +	struct device *dev = &pdev->dev;
>> +	int ret;
>> +
>> +	if (!of_machine_is_compatible("ti,am33xx") &&
>> +	    !of_machine_is_compatible("ti,am43"))
>> +		return -ENODEV;
>> +
>> +	pm_ops = dev->platform_data;
>> +	if (!pm_ops) {
>> +		pr_err("PM: Cannot get core PM ops!\n");
>> +		return -ENODEV;
>> +	}
>> +
>> +	pm_sram = pm_ops->get_sram_addrs();
>> +	if (!pm_sram) {
>> +		pr_err("PM: Cannot get PM asm function addresses!!\n");
>> +		return -ENODEV;
>> +	}
>> +
>> +	ret = am33xx_prepare_push_sram_idle();
> 
> Perhaps calling this one am33xx_pm_alloc_sram() would be more
> descriptive (and match the release function)?

Not a bad point.

> 
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = am33xx_push_sram_idle();
>> +	if (ret)
>> +		goto err_free_sram;
> 
> As I mentioned in my comments to the emif-sram driver, you may need to
> create device link to the emif-sram device to prevent it from going away
> under you here.

Addressed in ti-emif-pm thread.

> 
>> +
>> +	m3_ipc = wkup_m3_ipc_get();
>> +	if (!m3_ipc) {
>> +		pr_err("PM: Cannot get wkup_m3_ipc handle\n");
> 
> You shouldn't log this as an error when probe is being deferred. 

Yes, good point, just noise.

> 
> Why not use dev_err and friends for logging now that you have a struct
> device?

I suppose I could now.

> 
> And similarly to the emif-sram device, you may need to create a
> device-link also to the ipc device to prevent its driver from being
> unbound.

As described in the ti-emif-pm thread for that driver, we also call exported
symbols directly from wkup_m3_ipc driver, so pm33xx cannot probe at all if
wkup_m3_ipc is not loaded, and wkup_m3_ipc cannot be removed once pm33xx has
been loaded on top.

> 
>> +		ret = -EPROBE_DEFER;
>> +		goto err_free_sram;
>> +	}
>> +
>> +	am33xx_pm_set_ipc_ops();
>> +
>> +#ifdef CONFIG_SUSPEND
>> +	suspend_set_ops(&am33xx_pm_ops);
>> +#endif /* CONFIG_SUSPEND */
> 
> This renders a lockdep splash about a circular locking dependency when
> suspending since we're taking the pm_mutex in suspend_set_ops here, and
> during suspend we flush any deferred probes while already holding the
> mutex:
> 
>  ======================================================
>  WARNING: possible circular locking dependency detected
>  4.12.0-rc7 #11 Not tainted
>  ------------------------------------------------------
>  bash/404 is trying to acquire lock:
>   (deferred_probe_work){+.+.+.}, at: [<c014cf3c>] flush_work+0x30/0x27c
>  
>  but task is already holding lock:
>   (pm_mutex){+.+...}, at: [<c01792dc>] pm_suspend+0x190/0xc94
>  
>  which lock already depends on the new lock.
>  
>  
>  the existing dependency chain (in reverse order) is:
>  
>  -> #1 (pm_mutex){+.+...}:
>         __mutex_lock+0x80/0x694
>         mutex_lock_nested+0x2c/0x34
>         suspend_set_ops+0x4c/0x128
>         am33xx_pm_probe+0x1fc/0x3a8
>         platform_drv_probe+0x5c/0xc0
>         driver_probe_device+0x37c/0x490
>         __device_attach_driver+0xac/0x128
>         bus_for_each_drv+0x74/0xa8
>         __device_attach+0xc4/0x154
>         device_initial_probe+0x1c/0x20
>         bus_probe_device+0x98/0xa0
>         deferred_probe_work_func+0x4c/0xe4
>         process_one_work+0x1f4/0x758
>         worker_thread+0x1e0/0x514
>         kthread+0x128/0x158
>         ret_from_fork+0x14/0x24
>  
>  -> #0 (deferred_probe_work){+.+.+.}:
>         lock_acquire+0x108/0x264
>         flush_work+0x60/0x27c
>         wait_for_device_probe+0x24/0xa4
>         dpm_prepare+0xd0/0x91c
>         dpm_suspend_start+0x1c/0x70
>         suspend_devices_and_enter+0xc4/0xeac
>         pm_suspend+0x890/0xc94
>         state_store+0x80/0xdc
>         kobj_attr_store+0x1c/0x28
>         sysfs_kf_write+0x5c/0x60
>         kernfs_fop_write+0x128/0x254
>         __vfs_write+0x38/0x128
>         vfs_write+0xb4/0x174
>         SyS_write+0x54/0xb0
>         ret_fast_syscall+0x0/0x1c
> 

Yes thanks, I have seen this before myself now. I will need to look closer into
eliminating this. I am not sure how it is happening, pm_suspend should not be
able to be called if suspend_set_ops has not completed, at which point it should
have released the mutex.

Regards,
Dave

> Johan
> 

Re: [PATCH v2 5/5] ARM: OMAP2+: Create dummy platform_device for pm33xx

[Dave Gerlach]

On 07/03/2017 11:58 AM, Johan Hovold wrote:
> On Fri, May 19, 2017 at 03:04:38PM -0500, Dave Gerlach wrote:
>> Add amx3_common_pm_init to create a dummy platform_device for
>> pm33xx so that our pm33xx module can probe and am335x and am437x
>> platforms to enable basic suspend to mem and standby support.
>>
>> Signed-off-by: Dave Gerlach <d-gerlach@ti.com>
>> ---
> 
>> +void __init amx3_common_pm_init(void)
>> +{
>> +	struct platform_device_info devinfo = { };
> 
> You should make sure devinfo is zeroed, and also set .id to -1. 

Thanks for the comment, will look into updating this for next version.

Regards,
Dave

> 
>> +	struct am33xx_pm_platform_data *pdata;
>> +
>> +	pdata = am33xx_pm_get_pdata();
>> +	devinfo.name = "pm33xx";
>> +	devinfo.data = pdata;
>> +	devinfo.size_data = sizeof(*pdata);
>> +	platform_device_register_full(&devinfo);
>> +}
> 
> Johan
> 

Re: [PATCH v2 4/5] soc: ti: Add pm33xx driver for basic suspend support

[Johan Hovold]

On Thu, Jul 06, 2017 at 02:08:07PM -0500, Dave Gerlach wrote:
> On 07/03/2017 11:54 AM, Johan Hovold wrote:
> > On Fri, May 19, 2017 at 03:04:37PM -0500, Dave Gerlach wrote:
> >> AM335x and AM437x support various low power modes as documented
> >> in section 8.1.4.3 of the AM335x Technical Reference Manual and
> >> section 6.4.3 of the AM437x Technical Reference Manual.
> >>
> >> DeepSleep0 mode offers the lowest power mode with limited
> >> wakeup sources without a system reboot and is mapped as
> >> the suspend state in the kernel. In this state, MPU and
> >> PER domains are turned off with the internal RAM held in
> >> retention to facilitate the resume process. As part of
> >> the boot process, the assembly code is copied over to OCMCRAM
> >> so it can be executed to turn of the EMIF and put DDR into self
> >> refresh.
> >>
> >> Both platforms have a Cortex-M3 (WKUP_M3) which assists the MPU
> >> in DeepSleep0 entry and exit. WKUP_M3 takes care
> >> of the clockdomain and powerdomain transitions based on the
> >> intended low power state. MPU needs to load the appropriate
> >> WKUP_M3 binary onto the WKUP_M3 memory space before it can
> >> leverage any of the PM features like DeepSleep. This loading
> >> is handled by the remoteproc driver wkup_m3_rproc.
> >>
> >> Communication with the WKUP_M3 is handled by a wkup_m3_ipc
> >> driver that exposes the specific PM functionality to be used
> >> the PM code.

> > And similarly to the emif-sram device, you may need to create a
> > device-link also to the ipc device to prevent its driver from being
> > unbound.
> 
> As described in the ti-emif-pm thread for that driver, we also call exported
> symbols directly from wkup_m3_ipc driver, so pm33xx cannot probe at all if
> wkup_m3_ipc is not loaded, and wkup_m3_ipc cannot be removed once pm33xx has
> been loaded on top.

As discussed in the other thread, the ipc driver can be unbound from its
device even if the module remains loaded.

> > 
> >> +		ret = -EPROBE_DEFER;
> >> +		goto err_free_sram;
> >> +	}
> >> +
> >> +	am33xx_pm_set_ipc_ops();
> >> +
> >> +#ifdef CONFIG_SUSPEND
> >> +	suspend_set_ops(&am33xx_pm_ops);
> >> +#endif /* CONFIG_SUSPEND */
> > 
> > This renders a lockdep splash about a circular locking dependency when
> > suspending since we're taking the pm_mutex in suspend_set_ops here, and
> > during suspend we flush any deferred probes while already holding the
> > mutex:
> > 
> >  ======================================================
> >  WARNING: possible circular locking dependency detected
> >  4.12.0-rc7 #11 Not tainted
> >  ------------------------------------------------------
> >  bash/404 is trying to acquire lock:
> >   (deferred_probe_work){+.+.+.}, at: [<c014cf3c>] flush_work+0x30/0x27c
> >  
> >  but task is already holding lock:
> >   (pm_mutex){+.+...}, at: [<c01792dc>] pm_suspend+0x190/0xc94
> >  
> >  which lock already depends on the new lock.
> >  
> >  
> >  the existing dependency chain (in reverse order) is:
> >  
> >  -> #1 (pm_mutex){+.+...}:
> >         __mutex_lock+0x80/0x694
> >         mutex_lock_nested+0x2c/0x34
> >         suspend_set_ops+0x4c/0x128
> >         am33xx_pm_probe+0x1fc/0x3a8
> >         platform_drv_probe+0x5c/0xc0
> >         driver_probe_device+0x37c/0x490
> >         __device_attach_driver+0xac/0x128
> >         bus_for_each_drv+0x74/0xa8
> >         __device_attach+0xc4/0x154
> >         device_initial_probe+0x1c/0x20
> >         bus_probe_device+0x98/0xa0
> >         deferred_probe_work_func+0x4c/0xe4
> >         process_one_work+0x1f4/0x758
> >         worker_thread+0x1e0/0x514
> >         kthread+0x128/0x158
> >         ret_from_fork+0x14/0x24
> >  
> >  -> #0 (deferred_probe_work){+.+.+.}:
> >         lock_acquire+0x108/0x264
> >         flush_work+0x60/0x27c
> >         wait_for_device_probe+0x24/0xa4
> >         dpm_prepare+0xd0/0x91c
> >         dpm_suspend_start+0x1c/0x70
> >         suspend_devices_and_enter+0xc4/0xeac
> >         pm_suspend+0x890/0xc94
> >         state_store+0x80/0xdc
> >         kobj_attr_store+0x1c/0x28
> >         sysfs_kf_write+0x5c/0x60
> >         kernfs_fop_write+0x128/0x254
> >         __vfs_write+0x38/0x128
> >         vfs_write+0xb4/0x174
> >         SyS_write+0x54/0xb0
> >         ret_fast_syscall+0x0/0x1c
> > 
> 
> Yes thanks, I have seen this before myself now. I will need to look closer into
> eliminating this. I am not sure how it is happening, pm_suspend should not be
> able to be called if suspend_set_ops has not completed, at which point it should
> have released the mutex.

So perhaps the deadlock cannot happen in practise then even if both
paths can indeed be taken (which triggers the lockdep warning).

Johan