* [PATCH 0/8] Apple M1 NVMe storage support
@ 2022-01-14 11:04 Mark Kettenis
2022-01-14 11:04 ` [PATCH 1/8] nvme: Split out PCI support Mark Kettenis
` (7 more replies)
0 siblings, 8 replies; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
This adds support for the (rather quirky) NVMe storage controller
integrated on Apple SoCs. This makes it possible to boot from the
NVMe storage that is present on all the M1 machines that Apple has
released so far.
The series has been designed to have as little impact on the existing
NVMe support as possible. It splits out the PCIe-specific bits in its
own file/driver and adds a platform driver for the Apple controller.
This platform driver handles all the quirkiness through driver ops.
The existing logic is retained when the driver ops aren't set. I've
tested this on a firefly-rk3399 board with:
nvme0 at pci1 dev 0 function 0 "Silicon Motion SM2260 NVMe" rev 0x03: msix, NVMe 1.2
nvme0: ADATA SX8000NP, firmware C2.2.1, serial 2H1220011170
(for those not familliar with OpenBSD, that's an ADATA SX8000NP NVMe
M.2 SSD with a Silicon Motion controller)
This patch series depends on the Apple M1 power management controller
support series:
https://patchwork.ozlabs.org/project/uboot/list/?series=280359
Mark Kettenis (8):
nvme: Split out PCI support
mailbox: apple: Add driver for Apple IOP mailbox
arm: apple: Add RTKit support
nvme: Introduce driver ops
nvme: Add shutdown function
power: domain: apple: Add reset support
nvme: apple: Add driver for Apple NVMe storage controller
configs: apple: Add NVMe boot target
arch/arm/Kconfig | 2 +
arch/arm/include/asm/arch-apple/rtkit.h | 11 +
arch/arm/mach-apple/Makefile | 1 +
arch/arm/mach-apple/rtkit.c | 231 +++++++++++++++++
configs/apple_m1_defconfig | 1 +
configs/clearfog_gt_8k_defconfig | 2 +-
configs/firefly-rk3399_defconfig | 2 +-
configs/khadas-vim3_android_ab_defconfig | 2 +-
configs/khadas-vim3_android_defconfig | 2 +-
configs/khadas-vim3_defconfig | 2 +-
configs/khadas-vim3l_android_ab_defconfig | 2 +-
configs/khadas-vim3l_android_defconfig | 2 +-
configs/khadas-vim3l_defconfig | 2 +-
configs/kontron_sl28_defconfig | 2 +-
configs/ls1012afrdm_qspi_defconfig | 2 +-
configs/ls1012afrdm_tfa_defconfig | 2 +-
.../ls1012afrwy_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1012afrwy_qspi_defconfig | 2 +-
configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1012afrwy_tfa_defconfig | 2 +-
configs/ls1012aqds_qspi_defconfig | 2 +-
configs/ls1012aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1012aqds_tfa_defconfig | 2 +-
configs/ls1012ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1012ardb_qspi_defconfig | 2 +-
configs/ls1012ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1012ardb_tfa_defconfig | 2 +-
configs/ls1021aiot_qspi_defconfig | 2 +-
configs/ls1021aiot_sdcard_defconfig | 2 +-
configs/ls1021aqds_ddr4_nor_defconfig | 2 +-
configs/ls1021aqds_ddr4_nor_lpuart_defconfig | 2 +-
configs/ls1021aqds_nand_defconfig | 2 +-
configs/ls1021aqds_nor_SECURE_BOOT_defconfig | 2 +-
configs/ls1021aqds_nor_defconfig | 2 +-
configs/ls1021aqds_nor_lpuart_defconfig | 2 +-
configs/ls1021aqds_qspi_defconfig | 2 +-
configs/ls1021aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1021aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1021atsn_qspi_defconfig | 2 +-
configs/ls1021atsn_sdcard_defconfig | 2 +-
configs/ls1021atwr_nor_SECURE_BOOT_defconfig | 2 +-
configs/ls1021atwr_nor_defconfig | 2 +-
configs/ls1021atwr_nor_lpuart_defconfig | 2 +-
configs/ls1021atwr_qspi_defconfig | 2 +-
configs/ls1021atwr_sdcard_ifc_defconfig | 2 +-
configs/ls1021atwr_sdcard_qspi_defconfig | 2 +-
configs/ls1028aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1028aqds_tfa_defconfig | 2 +-
configs/ls1028aqds_tfa_lpuart_defconfig | 2 +-
configs/ls1028ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1028ardb_tfa_defconfig | 2 +-
configs/ls1043aqds_defconfig | 2 +-
configs/ls1043aqds_lpuart_defconfig | 2 +-
configs/ls1043aqds_nand_defconfig | 2 +-
configs/ls1043aqds_nor_ddr3_defconfig | 2 +-
configs/ls1043aqds_qspi_defconfig | 2 +-
configs/ls1043aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1043aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1043aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1043aqds_tfa_defconfig | 2 +-
configs/ls1043ardb_SECURE_BOOT_defconfig | 2 +-
configs/ls1043ardb_defconfig | 2 +-
configs/ls1043ardb_nand_SECURE_BOOT_defconfig | 2 +-
configs/ls1043ardb_nand_defconfig | 2 +-
configs/ls1043ardb_sdcard_defconfig | 2 +-
configs/ls1043ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1043ardb_tfa_defconfig | 2 +-
configs/ls1046afrwy_tfa_defconfig | 2 +-
configs/ls1046aqds_SECURE_BOOT_defconfig | 2 +-
configs/ls1046aqds_defconfig | 2 +-
configs/ls1046aqds_lpuart_defconfig | 2 +-
configs/ls1046aqds_nand_defconfig | 2 +-
configs/ls1046aqds_qspi_defconfig | 2 +-
configs/ls1046aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1046aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1046aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1046aqds_tfa_defconfig | 2 +-
configs/ls1046ardb_emmc_defconfig | 2 +-
configs/ls1046ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1046ardb_qspi_defconfig | 2 +-
configs/ls1046ardb_qspi_spl_defconfig | 2 +-
configs/ls1046ardb_sdcard_defconfig | 2 +-
configs/ls1046ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1046ardb_tfa_defconfig | 2 +-
configs/ls1088aqds_defconfig | 2 +-
configs/ls1088aqds_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1088aqds_qspi_defconfig | 2 +-
configs/ls1088aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1088aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1088aqds_tfa_defconfig | 2 +-
configs/ls1088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1088ardb_qspi_defconfig | 2 +-
configs/ls1088ardb_sdcard_qspi_defconfig | 2 +-
configs/ls1088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1088ardb_tfa_defconfig | 2 +-
configs/ls2080aqds_SECURE_BOOT_defconfig | 2 +-
configs/ls2080aqds_defconfig | 2 +-
configs/ls2080aqds_nand_defconfig | 2 +-
configs/ls2080aqds_qspi_defconfig | 2 +-
configs/ls2080aqds_sdcard_defconfig | 2 +-
configs/ls2080ardb_SECURE_BOOT_defconfig | 2 +-
configs/ls2080ardb_defconfig | 2 +-
configs/ls2080ardb_nand_defconfig | 2 +-
configs/ls2081ardb_defconfig | 2 +-
configs/ls2088aqds_tfa_defconfig | 2 +-
configs/ls2088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls2088ardb_qspi_defconfig | 2 +-
configs/ls2088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls2088ardb_tfa_defconfig | 2 +-
configs/lx2160aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/lx2160aqds_tfa_defconfig | 2 +-
configs/lx2160ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/lx2160ardb_tfa_defconfig | 2 +-
configs/lx2160ardb_tfa_stmm_defconfig | 2 +-
configs/mvebu_crb_cn9130_defconfig | 2 +-
configs/mvebu_db_armada8k_defconfig | 2 +-
configs/mvebu_db_cn9130_defconfig | 2 +-
configs/mvebu_espressobin-88f3720_defconfig | 2 +-
configs/mvebu_mcbin-88f8040_defconfig | 2 +-
configs/mvebu_puzzle-m801-88f8040_defconfig | 2 +-
configs/nanopc-t4-rk3399_defconfig | 2 +-
configs/octeontx2_96xx_defconfig | 2 +-
configs/octeontx_81xx_defconfig | 2 +-
configs/octeontx_83xx_defconfig | 2 +-
configs/p3450-0000_defconfig | 2 +-
configs/pinebook-pro-rk3399_defconfig | 2 +-
configs/qemu-x86_64_defconfig | 2 +-
configs/qemu-x86_defconfig | 2 +-
configs/qemu_arm64_defconfig | 2 +-
configs/qemu_arm_defconfig | 2 +-
configs/rcar3_salvator-x_defconfig | 2 +-
configs/roc-pc-mezzanine-rk3399_defconfig | 2 +-
configs/rock-pi-4-rk3399_defconfig | 2 +-
configs/rock-pi-4c-rk3399_defconfig | 2 +-
configs/rock-pi-n10-rk3399pro_defconfig | 2 +-
configs/rock960-rk3399_defconfig | 2 +-
configs/rockpro64-rk3399_defconfig | 2 +-
configs/sandbox64_defconfig | 2 +-
configs/sandbox_defconfig | 2 +-
configs/sandbox_flattree_defconfig | 2 +-
configs/sandbox_noinst_defconfig | 2 +-
configs/sandbox_spl_defconfig | 2 +-
configs/sifive_unmatched_defconfig | 2 +-
configs/synquacer_developerbox_defconfig | 2 +-
configs/turris_mox_defconfig | 2 +-
configs/turris_omnia_defconfig | 2 +-
doc/develop/driver-model/nvme.rst | 1 +
drivers/mailbox/Kconfig | 11 +
drivers/mailbox/Makefile | 1 +
drivers/mailbox/apple-mbox.c | 92 +++++++
drivers/nvme/Kconfig | 21 +-
drivers/nvme/Makefile | 2 +
drivers/nvme/nvme.c | 86 ++-----
drivers/nvme/nvme.h | 37 +++
drivers/nvme/nvme_apple.c | 233 ++++++++++++++++++
drivers/nvme/nvme_pci.c | 49 ++++
drivers/power/domain/apple-pmgr.c | 73 +++++-
include/configs/apple.h | 7 +
include/linux/apple-mailbox.h | 19 ++
159 files changed, 957 insertions(+), 203 deletions(-)
create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
create mode 100644 arch/arm/mach-apple/rtkit.c
create mode 100644 drivers/mailbox/apple-mbox.c
create mode 100644 drivers/nvme/nvme_apple.c
create mode 100644 drivers/nvme/nvme_pci.c
create mode 100644 include/linux/apple-mailbox.h
--
2.34.1
^ permalink raw reply [flat|nested] 33+ messages in thread
* [PATCH 1/8] nvme: Split out PCI support
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox Mark Kettenis
` (6 subsequent siblings)
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
Apple SoCs have an integrated NVMe controller that isn't connected
over a PCIe bus. In preparation for adding support for this NVMe
controller, split out the PCI support into its own file. This file
is selected through a new CONFIG_NVME_PCI Kconfig option, so do
a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
configs/clearfog_gt_8k_defconfig | 2 +-
configs/firefly-rk3399_defconfig | 2 +-
configs/khadas-vim3_android_ab_defconfig | 2 +-
configs/khadas-vim3_android_defconfig | 2 +-
configs/khadas-vim3_defconfig | 2 +-
configs/khadas-vim3l_android_ab_defconfig | 2 +-
configs/khadas-vim3l_android_defconfig | 2 +-
configs/khadas-vim3l_defconfig | 2 +-
configs/kontron_sl28_defconfig | 2 +-
configs/ls1012afrdm_qspi_defconfig | 2 +-
configs/ls1012afrdm_tfa_defconfig | 2 +-
.../ls1012afrwy_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1012afrwy_qspi_defconfig | 2 +-
configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1012afrwy_tfa_defconfig | 2 +-
configs/ls1012aqds_qspi_defconfig | 2 +-
configs/ls1012aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1012aqds_tfa_defconfig | 2 +-
configs/ls1012ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1012ardb_qspi_defconfig | 2 +-
configs/ls1012ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1012ardb_tfa_defconfig | 2 +-
configs/ls1021aiot_qspi_defconfig | 2 +-
configs/ls1021aiot_sdcard_defconfig | 2 +-
configs/ls1021aqds_ddr4_nor_defconfig | 2 +-
configs/ls1021aqds_ddr4_nor_lpuart_defconfig | 2 +-
configs/ls1021aqds_nand_defconfig | 2 +-
configs/ls1021aqds_nor_SECURE_BOOT_defconfig | 2 +-
configs/ls1021aqds_nor_defconfig | 2 +-
configs/ls1021aqds_nor_lpuart_defconfig | 2 +-
configs/ls1021aqds_qspi_defconfig | 2 +-
configs/ls1021aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1021aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1021atsn_qspi_defconfig | 2 +-
configs/ls1021atsn_sdcard_defconfig | 2 +-
configs/ls1021atwr_nor_SECURE_BOOT_defconfig | 2 +-
configs/ls1021atwr_nor_defconfig | 2 +-
configs/ls1021atwr_nor_lpuart_defconfig | 2 +-
configs/ls1021atwr_qspi_defconfig | 2 +-
configs/ls1021atwr_sdcard_ifc_defconfig | 2 +-
configs/ls1021atwr_sdcard_qspi_defconfig | 2 +-
configs/ls1028aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1028aqds_tfa_defconfig | 2 +-
configs/ls1028aqds_tfa_lpuart_defconfig | 2 +-
configs/ls1028ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1028ardb_tfa_defconfig | 2 +-
configs/ls1043aqds_defconfig | 2 +-
configs/ls1043aqds_lpuart_defconfig | 2 +-
configs/ls1043aqds_nand_defconfig | 2 +-
configs/ls1043aqds_nor_ddr3_defconfig | 2 +-
configs/ls1043aqds_qspi_defconfig | 2 +-
configs/ls1043aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1043aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1043aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1043aqds_tfa_defconfig | 2 +-
configs/ls1043ardb_SECURE_BOOT_defconfig | 2 +-
configs/ls1043ardb_defconfig | 2 +-
configs/ls1043ardb_nand_SECURE_BOOT_defconfig | 2 +-
configs/ls1043ardb_nand_defconfig | 2 +-
configs/ls1043ardb_sdcard_defconfig | 2 +-
configs/ls1043ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1043ardb_tfa_defconfig | 2 +-
configs/ls1046afrwy_tfa_defconfig | 2 +-
configs/ls1046aqds_SECURE_BOOT_defconfig | 2 +-
configs/ls1046aqds_defconfig | 2 +-
configs/ls1046aqds_lpuart_defconfig | 2 +-
configs/ls1046aqds_nand_defconfig | 2 +-
configs/ls1046aqds_qspi_defconfig | 2 +-
configs/ls1046aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1046aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1046aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1046aqds_tfa_defconfig | 2 +-
configs/ls1046ardb_emmc_defconfig | 2 +-
configs/ls1046ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1046ardb_qspi_defconfig | 2 +-
configs/ls1046ardb_qspi_spl_defconfig | 2 +-
configs/ls1046ardb_sdcard_defconfig | 2 +-
configs/ls1046ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1046ardb_tfa_defconfig | 2 +-
configs/ls1088aqds_defconfig | 2 +-
configs/ls1088aqds_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1088aqds_qspi_defconfig | 2 +-
configs/ls1088aqds_sdcard_ifc_defconfig | 2 +-
configs/ls1088aqds_sdcard_qspi_defconfig | 2 +-
configs/ls1088aqds_tfa_defconfig | 2 +-
configs/ls1088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls1088ardb_qspi_defconfig | 2 +-
configs/ls1088ardb_sdcard_qspi_defconfig | 2 +-
configs/ls1088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls1088ardb_tfa_defconfig | 2 +-
configs/ls2080aqds_SECURE_BOOT_defconfig | 2 +-
configs/ls2080aqds_defconfig | 2 +-
configs/ls2080aqds_nand_defconfig | 2 +-
configs/ls2080aqds_qspi_defconfig | 2 +-
configs/ls2080aqds_sdcard_defconfig | 2 +-
configs/ls2080ardb_SECURE_BOOT_defconfig | 2 +-
configs/ls2080ardb_defconfig | 2 +-
configs/ls2080ardb_nand_defconfig | 2 +-
configs/ls2081ardb_defconfig | 2 +-
configs/ls2088aqds_tfa_defconfig | 2 +-
configs/ls2088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
configs/ls2088ardb_qspi_defconfig | 2 +-
configs/ls2088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/ls2088ardb_tfa_defconfig | 2 +-
configs/lx2160aqds_tfa_SECURE_BOOT_defconfig | 2 +-
configs/lx2160aqds_tfa_defconfig | 2 +-
configs/lx2160ardb_tfa_SECURE_BOOT_defconfig | 2 +-
configs/lx2160ardb_tfa_defconfig | 2 +-
configs/lx2160ardb_tfa_stmm_defconfig | 2 +-
configs/mvebu_crb_cn9130_defconfig | 2 +-
configs/mvebu_db_armada8k_defconfig | 2 +-
configs/mvebu_db_cn9130_defconfig | 2 +-
configs/mvebu_espressobin-88f3720_defconfig | 2 +-
configs/mvebu_mcbin-88f8040_defconfig | 2 +-
configs/mvebu_puzzle-m801-88f8040_defconfig | 2 +-
configs/nanopc-t4-rk3399_defconfig | 2 +-
configs/octeontx2_96xx_defconfig | 2 +-
configs/octeontx_81xx_defconfig | 2 +-
configs/octeontx_83xx_defconfig | 2 +-
configs/p3450-0000_defconfig | 2 +-
configs/pinebook-pro-rk3399_defconfig | 2 +-
configs/qemu-x86_64_defconfig | 2 +-
configs/qemu-x86_defconfig | 2 +-
configs/qemu_arm64_defconfig | 2 +-
configs/qemu_arm_defconfig | 2 +-
configs/rcar3_salvator-x_defconfig | 2 +-
configs/roc-pc-mezzanine-rk3399_defconfig | 2 +-
configs/rock-pi-4-rk3399_defconfig | 2 +-
configs/rock-pi-4c-rk3399_defconfig | 2 +-
configs/rock-pi-n10-rk3399pro_defconfig | 2 +-
configs/rock960-rk3399_defconfig | 2 +-
configs/rockpro64-rk3399_defconfig | 2 +-
configs/sandbox64_defconfig | 2 +-
configs/sandbox_defconfig | 2 +-
configs/sandbox_flattree_defconfig | 2 +-
configs/sandbox_noinst_defconfig | 2 +-
configs/sandbox_spl_defconfig | 2 +-
configs/sifive_unmatched_defconfig | 2 +-
configs/synquacer_developerbox_defconfig | 2 +-
configs/turris_mox_defconfig | 2 +-
configs/turris_omnia_defconfig | 2 +-
doc/develop/driver-model/nvme.rst | 1 +
drivers/nvme/Kconfig | 10 +++-
drivers/nvme/Makefile | 1 +
drivers/nvme/nvme.c | 38 ++------------
drivers/nvme/nvme.h | 3 ++
drivers/nvme/nvme_pci.c | 49 +++++++++++++++++++
147 files changed, 207 insertions(+), 177 deletions(-)
create mode 100644 drivers/nvme/nvme_pci.c
diff --git a/configs/clearfog_gt_8k_defconfig b/configs/clearfog_gt_8k_defconfig
index 6e344c9ce3..6dcd70fe12 100644
--- a/configs/clearfog_gt_8k_defconfig
+++ b/configs/clearfog_gt_8k_defconfig
@@ -56,7 +56,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MVEBU=y
CONFIG_PHY=y
diff --git a/configs/firefly-rk3399_defconfig b/configs/firefly-rk3399_defconfig
index d576b5c38d..fe1c019f1d 100644
--- a/configs/firefly-rk3399_defconfig
+++ b/configs/firefly-rk3399_defconfig
@@ -41,7 +41,7 @@ CONFIG_SF_DEFAULT_SPEED=20000000
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PMIC_RK8XX=y
CONFIG_REGULATOR_PWM=y
diff --git a/configs/khadas-vim3_android_ab_defconfig b/configs/khadas-vim3_android_ab_defconfig
index 88325c37db..7d3e2b8fdd 100644
--- a/configs/khadas-vim3_android_ab_defconfig
+++ b/configs/khadas-vim3_android_ab_defconfig
@@ -62,7 +62,7 @@ CONFIG_DM_MDIO=y
CONFIG_DM_MDIO_MUX=y
CONFIG_ETH_DESIGNWARE_MESON8B=y
CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MESON=y
CONFIG_MESON_G12A_USB_PHY=y
diff --git a/configs/khadas-vim3_android_defconfig b/configs/khadas-vim3_android_defconfig
index 9305a54b1b..3f5992d5b6 100644
--- a/configs/khadas-vim3_android_defconfig
+++ b/configs/khadas-vim3_android_defconfig
@@ -60,7 +60,7 @@ CONFIG_DM_MDIO=y
CONFIG_DM_MDIO_MUX=y
CONFIG_ETH_DESIGNWARE_MESON8B=y
CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MESON=y
CONFIG_MESON_G12A_USB_PHY=y
diff --git a/configs/khadas-vim3_defconfig b/configs/khadas-vim3_defconfig
index 65050efd3f..5e868ce2df 100644
--- a/configs/khadas-vim3_defconfig
+++ b/configs/khadas-vim3_defconfig
@@ -47,7 +47,7 @@ CONFIG_DM_MDIO=y
CONFIG_DM_MDIO_MUX=y
CONFIG_ETH_DESIGNWARE_MESON8B=y
CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MESON=y
CONFIG_MESON_G12A_USB_PHY=y
diff --git a/configs/khadas-vim3l_android_ab_defconfig b/configs/khadas-vim3l_android_ab_defconfig
index 608b54dca2..4f4c55c8ea 100644
--- a/configs/khadas-vim3l_android_ab_defconfig
+++ b/configs/khadas-vim3l_android_ab_defconfig
@@ -62,7 +62,7 @@ CONFIG_DM_MDIO=y
CONFIG_DM_MDIO_MUX=y
CONFIG_ETH_DESIGNWARE_MESON8B=y
CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MESON=y
CONFIG_MESON_G12A_USB_PHY=y
diff --git a/configs/khadas-vim3l_android_defconfig b/configs/khadas-vim3l_android_defconfig
index 5eed79b263..fdee62d801 100644
--- a/configs/khadas-vim3l_android_defconfig
+++ b/configs/khadas-vim3l_android_defconfig
@@ -60,7 +60,7 @@ CONFIG_DM_MDIO=y
CONFIG_DM_MDIO_MUX=y
CONFIG_ETH_DESIGNWARE_MESON8B=y
CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MESON=y
CONFIG_MESON_G12A_USB_PHY=y
diff --git a/configs/khadas-vim3l_defconfig b/configs/khadas-vim3l_defconfig
index b9162c9f1e..2b391646c5 100644
--- a/configs/khadas-vim3l_defconfig
+++ b/configs/khadas-vim3l_defconfig
@@ -47,7 +47,7 @@ CONFIG_DM_MDIO=y
CONFIG_DM_MDIO_MUX=y
CONFIG_ETH_DESIGNWARE_MESON8B=y
CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MESON=y
CONFIG_MESON_G12A_USB_PHY=y
diff --git a/configs/kontron_sl28_defconfig b/configs/kontron_sl28_defconfig
index 31a1083b0a..087dcb1d0e 100644
--- a/configs/kontron_sl28_defconfig
+++ b/configs/kontron_sl28_defconfig
@@ -83,7 +83,7 @@ CONFIG_DM_DSA=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MSCC_FELIX_SWITCH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_RTC_RV8803=y
diff --git a/configs/ls1012afrdm_qspi_defconfig b/configs/ls1012afrdm_qspi_defconfig
index ad3b7bff42..0b3964e9d8 100644
--- a/configs/ls1012afrdm_qspi_defconfig
+++ b/configs/ls1012afrdm_qspi_defconfig
@@ -50,7 +50,7 @@ CONFIG_SPI_FLASH_SPANSION=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012afrdm_tfa_defconfig b/configs/ls1012afrdm_tfa_defconfig
index 3317a0592b..93514082f2 100644
--- a/configs/ls1012afrdm_tfa_defconfig
+++ b/configs/ls1012afrdm_tfa_defconfig
@@ -50,7 +50,7 @@ CONFIG_SPI_FLASH_SPANSION=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012afrwy_qspi_SECURE_BOOT_defconfig b/configs/ls1012afrwy_qspi_SECURE_BOOT_defconfig
index cb8b288c39..8ecef720b0 100644
--- a/configs/ls1012afrwy_qspi_SECURE_BOOT_defconfig
+++ b/configs/ls1012afrwy_qspi_SECURE_BOOT_defconfig
@@ -49,7 +49,7 @@ CONFIG_SPI_FLASH_WINBOND=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012afrwy_qspi_defconfig b/configs/ls1012afrwy_qspi_defconfig
index 610f32cd99..097349e497 100644
--- a/configs/ls1012afrwy_qspi_defconfig
+++ b/configs/ls1012afrwy_qspi_defconfig
@@ -53,7 +53,7 @@ CONFIG_SPI_FLASH_WINBOND=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig b/configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig
index b3cb6f706d..48d12802f1 100644
--- a/configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig
@@ -49,7 +49,7 @@ CONFIG_SPI_FLASH_WINBOND=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012afrwy_tfa_defconfig b/configs/ls1012afrwy_tfa_defconfig
index afec07cf8e..5137b52fd8 100644
--- a/configs/ls1012afrwy_tfa_defconfig
+++ b/configs/ls1012afrwy_tfa_defconfig
@@ -53,7 +53,7 @@ CONFIG_SPI_FLASH_WINBOND=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012aqds_qspi_defconfig b/configs/ls1012aqds_qspi_defconfig
index 93e76d95f1..d10a4f558f 100644
--- a/configs/ls1012aqds_qspi_defconfig
+++ b/configs/ls1012aqds_qspi_defconfig
@@ -71,7 +71,7 @@ CONFIG_SPI_FLASH_SST=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012aqds_tfa_SECURE_BOOT_defconfig b/configs/ls1012aqds_tfa_SECURE_BOOT_defconfig
index 476ebd0415..14eeade305 100644
--- a/configs/ls1012aqds_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1012aqds_tfa_SECURE_BOOT_defconfig
@@ -63,7 +63,7 @@ CONFIG_SPI_FLASH_SST=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012aqds_tfa_defconfig b/configs/ls1012aqds_tfa_defconfig
index d4d9b8f76b..08850e5981 100644
--- a/configs/ls1012aqds_tfa_defconfig
+++ b/configs/ls1012aqds_tfa_defconfig
@@ -72,7 +72,7 @@ CONFIG_SPI_FLASH_SST=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012ardb_qspi_SECURE_BOOT_defconfig b/configs/ls1012ardb_qspi_SECURE_BOOT_defconfig
index 8767c87cc6..f495367808 100644
--- a/configs/ls1012ardb_qspi_SECURE_BOOT_defconfig
+++ b/configs/ls1012ardb_qspi_SECURE_BOOT_defconfig
@@ -51,7 +51,7 @@ CONFIG_DM_SPI_FLASH=y
CONFIG_SPI_FLASH_SPANSION=y
# CONFIG_SPI_FLASH_USE_4K_SECTORS is not set
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1012ardb_qspi_defconfig b/configs/ls1012ardb_qspi_defconfig
index fccbef70ea..262ef9bd03 100644
--- a/configs/ls1012ardb_qspi_defconfig
+++ b/configs/ls1012ardb_qspi_defconfig
@@ -56,7 +56,7 @@ CONFIG_SPI_FLASH_SPANSION=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012ardb_tfa_SECURE_BOOT_defconfig b/configs/ls1012ardb_tfa_SECURE_BOOT_defconfig
index bf35286848..1be4478bbf 100644
--- a/configs/ls1012ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1012ardb_tfa_SECURE_BOOT_defconfig
@@ -53,7 +53,7 @@ CONFIG_SPI_FLASH_SPANSION=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1012ardb_tfa_defconfig b/configs/ls1012ardb_tfa_defconfig
index 7a175d3c74..12e77de638 100644
--- a/configs/ls1012ardb_tfa_defconfig
+++ b/configs/ls1012ardb_tfa_defconfig
@@ -55,7 +55,7 @@ CONFIG_SPI_FLASH_SPANSION=y
CONFIG_FSL_PFE=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1021aiot_qspi_defconfig b/configs/ls1021aiot_qspi_defconfig
index 4705e34e1d..fca79db20a 100644
--- a/configs/ls1021aiot_qspi_defconfig
+++ b/configs/ls1021aiot_qspi_defconfig
@@ -53,7 +53,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aiot_sdcard_defconfig b/configs/ls1021aiot_sdcard_defconfig
index e387264d89..c099a3ccfa 100644
--- a/configs/ls1021aiot_sdcard_defconfig
+++ b/configs/ls1021aiot_sdcard_defconfig
@@ -70,7 +70,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_ddr4_nor_defconfig b/configs/ls1021aqds_ddr4_nor_defconfig
index 79ccc41a43..797d114245 100644
--- a/configs/ls1021aqds_ddr4_nor_defconfig
+++ b/configs/ls1021aqds_ddr4_nor_defconfig
@@ -74,7 +74,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_ddr4_nor_lpuart_defconfig b/configs/ls1021aqds_ddr4_nor_lpuart_defconfig
index 0a2a076332..b0c542c266 100644
--- a/configs/ls1021aqds_ddr4_nor_lpuart_defconfig
+++ b/configs/ls1021aqds_ddr4_nor_lpuart_defconfig
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_nand_defconfig b/configs/ls1021aqds_nand_defconfig
index 0b12f100d4..648182ae4c 100644
--- a/configs/ls1021aqds_nand_defconfig
+++ b/configs/ls1021aqds_nand_defconfig
@@ -97,7 +97,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_nor_SECURE_BOOT_defconfig b/configs/ls1021aqds_nor_SECURE_BOOT_defconfig
index 63930e4a13..e02b573e17 100644
--- a/configs/ls1021aqds_nor_SECURE_BOOT_defconfig
+++ b/configs/ls1021aqds_nor_SECURE_BOOT_defconfig
@@ -73,7 +73,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_nor_defconfig b/configs/ls1021aqds_nor_defconfig
index f4e2274fb8..a8da00a42b 100644
--- a/configs/ls1021aqds_nor_defconfig
+++ b/configs/ls1021aqds_nor_defconfig
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_nor_lpuart_defconfig b/configs/ls1021aqds_nor_lpuart_defconfig
index 0eacaa3354..8bea5fbfb9 100644
--- a/configs/ls1021aqds_nor_lpuart_defconfig
+++ b/configs/ls1021aqds_nor_lpuart_defconfig
@@ -76,7 +76,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_qspi_defconfig b/configs/ls1021aqds_qspi_defconfig
index f967619ef9..3a1b33dd70 100644
--- a/configs/ls1021aqds_qspi_defconfig
+++ b/configs/ls1021aqds_qspi_defconfig
@@ -67,7 +67,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_sdcard_ifc_defconfig b/configs/ls1021aqds_sdcard_ifc_defconfig
index fa190a7502..b5f3d9d170 100644
--- a/configs/ls1021aqds_sdcard_ifc_defconfig
+++ b/configs/ls1021aqds_sdcard_ifc_defconfig
@@ -93,7 +93,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021aqds_sdcard_qspi_defconfig b/configs/ls1021aqds_sdcard_qspi_defconfig
index 625dd086ab..bd3ae1a188 100644
--- a/configs/ls1021aqds_sdcard_qspi_defconfig
+++ b/configs/ls1021aqds_sdcard_qspi_defconfig
@@ -84,7 +84,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1021atsn_qspi_defconfig b/configs/ls1021atsn_qspi_defconfig
index 5c8cee3ea9..aaea7156c1 100644
--- a/configs/ls1021atsn_qspi_defconfig
+++ b/configs/ls1021atsn_qspi_defconfig
@@ -59,7 +59,7 @@ CONFIG_PHY_GIGE=y
CONFIG_MII=y
CONFIG_SJA1105=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_NS16550=y
diff --git a/configs/ls1021atsn_sdcard_defconfig b/configs/ls1021atsn_sdcard_defconfig
index a3b8f248c7..870fdb05bf 100644
--- a/configs/ls1021atsn_sdcard_defconfig
+++ b/configs/ls1021atsn_sdcard_defconfig
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
CONFIG_MII=y
CONFIG_SJA1105=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_NS16550=y
diff --git a/configs/ls1021atwr_nor_SECURE_BOOT_defconfig b/configs/ls1021atwr_nor_SECURE_BOOT_defconfig
index f61d29ce89..27b28cc1c9 100644
--- a/configs/ls1021atwr_nor_SECURE_BOOT_defconfig
+++ b/configs/ls1021atwr_nor_SECURE_BOOT_defconfig
@@ -62,7 +62,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x60940000
diff --git a/configs/ls1021atwr_nor_defconfig b/configs/ls1021atwr_nor_defconfig
index 02c25cda7d..1242f9c5aa 100644
--- a/configs/ls1021atwr_nor_defconfig
+++ b/configs/ls1021atwr_nor_defconfig
@@ -64,7 +64,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x60940000
diff --git a/configs/ls1021atwr_nor_lpuart_defconfig b/configs/ls1021atwr_nor_lpuart_defconfig
index 49d6dda6a5..2562f9077f 100644
--- a/configs/ls1021atwr_nor_lpuart_defconfig
+++ b/configs/ls1021atwr_nor_lpuart_defconfig
@@ -65,7 +65,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x60940000
diff --git a/configs/ls1021atwr_qspi_defconfig b/configs/ls1021atwr_qspi_defconfig
index 36a8aee0b1..602ce2e9e3 100644
--- a/configs/ls1021atwr_qspi_defconfig
+++ b/configs/ls1021atwr_qspi_defconfig
@@ -63,7 +63,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls1021atwr_sdcard_ifc_defconfig b/configs/ls1021atwr_sdcard_ifc_defconfig
index ede4c95dd1..cb454837bb 100644
--- a/configs/ls1021atwr_sdcard_ifc_defconfig
+++ b/configs/ls1021atwr_sdcard_ifc_defconfig
@@ -82,7 +82,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls1021atwr_sdcard_qspi_defconfig b/configs/ls1021atwr_sdcard_qspi_defconfig
index 1315043094..195ce83198 100644
--- a/configs/ls1021atwr_sdcard_qspi_defconfig
+++ b/configs/ls1021atwr_sdcard_qspi_defconfig
@@ -80,7 +80,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls1028aqds_tfa_SECURE_BOOT_defconfig b/configs/ls1028aqds_tfa_SECURE_BOOT_defconfig
index 2fb90735ab..64bd2bbc24 100644
--- a/configs/ls1028aqds_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1028aqds_tfa_SECURE_BOOT_defconfig
@@ -71,7 +71,7 @@ CONFIG_DM_DSA=y
CONFIG_E1000=y
CONFIG_MSCC_FELIX_SWITCH=y
CONFIG_MDIO_MUX_I2CREG=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1028aqds_tfa_defconfig b/configs/ls1028aqds_tfa_defconfig
index 0bdca833f1..ca827ac0b4 100644
--- a/configs/ls1028aqds_tfa_defconfig
+++ b/configs/ls1028aqds_tfa_defconfig
@@ -77,7 +77,7 @@ CONFIG_DM_DSA=y
CONFIG_E1000=y
CONFIG_MSCC_FELIX_SWITCH=y
CONFIG_MDIO_MUX_I2CREG=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1028aqds_tfa_lpuart_defconfig b/configs/ls1028aqds_tfa_lpuart_defconfig
index dbf6af99f5..f7933e383d 100644
--- a/configs/ls1028aqds_tfa_lpuart_defconfig
+++ b/configs/ls1028aqds_tfa_lpuart_defconfig
@@ -76,7 +76,7 @@ CONFIG_DM_MDIO_MUX=y
CONFIG_E1000=y
CONFIG_FSL_ENETC=y
CONFIG_MDIO_MUX_I2CREG=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1028ardb_tfa_SECURE_BOOT_defconfig b/configs/ls1028ardb_tfa_SECURE_BOOT_defconfig
index 5cb55d046d..f37cca2ad6 100644
--- a/configs/ls1028ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1028ardb_tfa_SECURE_BOOT_defconfig
@@ -66,7 +66,7 @@ CONFIG_DM_DSA=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MSCC_FELIX_SWITCH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1028ardb_tfa_defconfig b/configs/ls1028ardb_tfa_defconfig
index b58254dc65..b753824e9d 100644
--- a/configs/ls1028ardb_tfa_defconfig
+++ b/configs/ls1028ardb_tfa_defconfig
@@ -72,7 +72,7 @@ CONFIG_DM_DSA=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MSCC_FELIX_SWITCH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_defconfig b/configs/ls1043aqds_defconfig
index 1a484ff483..c30731e3df 100644
--- a/configs/ls1043aqds_defconfig
+++ b/configs/ls1043aqds_defconfig
@@ -79,7 +79,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_lpuart_defconfig b/configs/ls1043aqds_lpuart_defconfig
index dfd7fb881b..229f13cc40 100644
--- a/configs/ls1043aqds_lpuart_defconfig
+++ b/configs/ls1043aqds_lpuart_defconfig
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_nand_defconfig b/configs/ls1043aqds_nand_defconfig
index 9e87f0fd88..e6ea996d3c 100644
--- a/configs/ls1043aqds_nand_defconfig
+++ b/configs/ls1043aqds_nand_defconfig
@@ -100,7 +100,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_nor_ddr3_defconfig b/configs/ls1043aqds_nor_ddr3_defconfig
index bffe105220..4d7d14ece6 100644
--- a/configs/ls1043aqds_nor_ddr3_defconfig
+++ b/configs/ls1043aqds_nor_ddr3_defconfig
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_qspi_defconfig b/configs/ls1043aqds_qspi_defconfig
index e01324ccef..bbd5330c7d 100644
--- a/configs/ls1043aqds_qspi_defconfig
+++ b/configs/ls1043aqds_qspi_defconfig
@@ -74,7 +74,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_sdcard_ifc_defconfig b/configs/ls1043aqds_sdcard_ifc_defconfig
index b487b370b5..4ccbfb78db 100644
--- a/configs/ls1043aqds_sdcard_ifc_defconfig
+++ b/configs/ls1043aqds_sdcard_ifc_defconfig
@@ -98,7 +98,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_sdcard_qspi_defconfig b/configs/ls1043aqds_sdcard_qspi_defconfig
index 084d104ea4..a8cfbf09e8 100644
--- a/configs/ls1043aqds_sdcard_qspi_defconfig
+++ b/configs/ls1043aqds_sdcard_qspi_defconfig
@@ -91,7 +91,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_tfa_SECURE_BOOT_defconfig b/configs/ls1043aqds_tfa_SECURE_BOOT_defconfig
index 3e303c5311..f495098481 100644
--- a/configs/ls1043aqds_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1043aqds_tfa_SECURE_BOOT_defconfig
@@ -81,7 +81,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043aqds_tfa_defconfig b/configs/ls1043aqds_tfa_defconfig
index 8d02bed8e1..39368732a3 100644
--- a/configs/ls1043aqds_tfa_defconfig
+++ b/configs/ls1043aqds_tfa_defconfig
@@ -90,7 +90,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1043ardb_SECURE_BOOT_defconfig b/configs/ls1043ardb_SECURE_BOOT_defconfig
index 2d85f671cc..bba6e2eb46 100644
--- a/configs/ls1043ardb_SECURE_BOOT_defconfig
+++ b/configs/ls1043ardb_SECURE_BOOT_defconfig
@@ -64,7 +64,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x60940000
diff --git a/configs/ls1043ardb_defconfig b/configs/ls1043ardb_defconfig
index ca6958fdb4..e361c96049 100644
--- a/configs/ls1043ardb_defconfig
+++ b/configs/ls1043ardb_defconfig
@@ -67,7 +67,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x60940000
diff --git a/configs/ls1043ardb_nand_SECURE_BOOT_defconfig b/configs/ls1043ardb_nand_SECURE_BOOT_defconfig
index bbc775a4d9..6d421d5d28 100644
--- a/configs/ls1043ardb_nand_SECURE_BOOT_defconfig
+++ b/configs/ls1043ardb_nand_SECURE_BOOT_defconfig
@@ -79,7 +79,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FMAN_FW_IN_NAND=y
diff --git a/configs/ls1043ardb_nand_defconfig b/configs/ls1043ardb_nand_defconfig
index e8986db42b..7e0da803b2 100644
--- a/configs/ls1043ardb_nand_defconfig
+++ b/configs/ls1043ardb_nand_defconfig
@@ -87,7 +87,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FMAN_FW_IN_NAND=y
diff --git a/configs/ls1043ardb_sdcard_defconfig b/configs/ls1043ardb_sdcard_defconfig
index c399ed0891..933de4ed2a 100644
--- a/configs/ls1043ardb_sdcard_defconfig
+++ b/configs/ls1043ardb_sdcard_defconfig
@@ -85,7 +85,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x940000
diff --git a/configs/ls1043ardb_tfa_SECURE_BOOT_defconfig b/configs/ls1043ardb_tfa_SECURE_BOOT_defconfig
index 8ac9a06c72..14100c8429 100644
--- a/configs/ls1043ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1043ardb_tfa_SECURE_BOOT_defconfig
@@ -65,7 +65,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x940000
diff --git a/configs/ls1043ardb_tfa_defconfig b/configs/ls1043ardb_tfa_defconfig
index 99c125731f..6281f841a4 100644
--- a/configs/ls1043ardb_tfa_defconfig
+++ b/configs/ls1043ardb_tfa_defconfig
@@ -71,7 +71,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_SYS_QE_FW_ADDR=0x940000
diff --git a/configs/ls1046afrwy_tfa_defconfig b/configs/ls1046afrwy_tfa_defconfig
index af07b1cd46..d587826d44 100644
--- a/configs/ls1046afrwy_tfa_defconfig
+++ b/configs/ls1046afrwy_tfa_defconfig
@@ -58,7 +58,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_SECURE_BOOT_defconfig b/configs/ls1046aqds_SECURE_BOOT_defconfig
index 55426320d8..19c483ed12 100644
--- a/configs/ls1046aqds_SECURE_BOOT_defconfig
+++ b/configs/ls1046aqds_SECURE_BOOT_defconfig
@@ -76,7 +76,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_defconfig b/configs/ls1046aqds_defconfig
index f136aeafae..c98d2d5995 100644
--- a/configs/ls1046aqds_defconfig
+++ b/configs/ls1046aqds_defconfig
@@ -79,7 +79,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_lpuart_defconfig b/configs/ls1046aqds_lpuart_defconfig
index 6d448905ef..07d8474e5a 100644
--- a/configs/ls1046aqds_lpuart_defconfig
+++ b/configs/ls1046aqds_lpuart_defconfig
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_nand_defconfig b/configs/ls1046aqds_nand_defconfig
index e4cd4a2f60..9b1eb76b20 100644
--- a/configs/ls1046aqds_nand_defconfig
+++ b/configs/ls1046aqds_nand_defconfig
@@ -99,7 +99,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_qspi_defconfig b/configs/ls1046aqds_qspi_defconfig
index fe295c51e4..6ce6abac58 100644
--- a/configs/ls1046aqds_qspi_defconfig
+++ b/configs/ls1046aqds_qspi_defconfig
@@ -75,7 +75,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_sdcard_ifc_defconfig b/configs/ls1046aqds_sdcard_ifc_defconfig
index fe1fe6f162..2ebe445ac6 100644
--- a/configs/ls1046aqds_sdcard_ifc_defconfig
+++ b/configs/ls1046aqds_sdcard_ifc_defconfig
@@ -99,7 +99,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_sdcard_qspi_defconfig b/configs/ls1046aqds_sdcard_qspi_defconfig
index 18f560e947..f85af10f73 100644
--- a/configs/ls1046aqds_sdcard_qspi_defconfig
+++ b/configs/ls1046aqds_sdcard_qspi_defconfig
@@ -93,7 +93,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_tfa_SECURE_BOOT_defconfig b/configs/ls1046aqds_tfa_SECURE_BOOT_defconfig
index b84f8772c6..3f0613d3d7 100644
--- a/configs/ls1046aqds_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1046aqds_tfa_SECURE_BOOT_defconfig
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046aqds_tfa_defconfig b/configs/ls1046aqds_tfa_defconfig
index f706dd6179..ae78bf7ea0 100644
--- a/configs/ls1046aqds_tfa_defconfig
+++ b/configs/ls1046aqds_tfa_defconfig
@@ -90,7 +90,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1046ardb_emmc_defconfig b/configs/ls1046ardb_emmc_defconfig
index 46301fb183..5c288a9e83 100644
--- a/configs/ls1046ardb_emmc_defconfig
+++ b/configs/ls1046ardb_emmc_defconfig
@@ -83,7 +83,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1046ardb_qspi_SECURE_BOOT_defconfig b/configs/ls1046ardb_qspi_SECURE_BOOT_defconfig
index a4696f1a16..5b6d9aa24b 100644
--- a/configs/ls1046ardb_qspi_SECURE_BOOT_defconfig
+++ b/configs/ls1046ardb_qspi_SECURE_BOOT_defconfig
@@ -66,7 +66,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1046ardb_qspi_defconfig b/configs/ls1046ardb_qspi_defconfig
index 1837833a8d..fd3ccc7063 100644
--- a/configs/ls1046ardb_qspi_defconfig
+++ b/configs/ls1046ardb_qspi_defconfig
@@ -70,7 +70,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1046ardb_qspi_spl_defconfig b/configs/ls1046ardb_qspi_spl_defconfig
index 777b7678a3..08e1fb2ca0 100644
--- a/configs/ls1046ardb_qspi_spl_defconfig
+++ b/configs/ls1046ardb_qspi_spl_defconfig
@@ -88,7 +88,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1046ardb_sdcard_defconfig b/configs/ls1046ardb_sdcard_defconfig
index ac8f2bcb1b..6e3694c42f 100644
--- a/configs/ls1046ardb_sdcard_defconfig
+++ b/configs/ls1046ardb_sdcard_defconfig
@@ -82,7 +82,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1046ardb_tfa_SECURE_BOOT_defconfig b/configs/ls1046ardb_tfa_SECURE_BOOT_defconfig
index bd1dc05c58..eccd109b9a 100644
--- a/configs/ls1046ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1046ardb_tfa_SECURE_BOOT_defconfig
@@ -62,7 +62,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1046ardb_tfa_defconfig b/configs/ls1046ardb_tfa_defconfig
index 149091af4a..ebcf6777be 100644
--- a/configs/ls1046ardb_tfa_defconfig
+++ b/configs/ls1046ardb_tfa_defconfig
@@ -68,7 +68,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_FMAN_ENET=y
CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_EP=y
diff --git a/configs/ls1088aqds_defconfig b/configs/ls1088aqds_defconfig
index a2b2a34e3f..8bc243e36b 100644
--- a/configs/ls1088aqds_defconfig
+++ b/configs/ls1088aqds_defconfig
@@ -78,7 +78,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1088aqds_qspi_SECURE_BOOT_defconfig b/configs/ls1088aqds_qspi_SECURE_BOOT_defconfig
index dbcd6e2ea2..bc9243567c 100644
--- a/configs/ls1088aqds_qspi_SECURE_BOOT_defconfig
+++ b/configs/ls1088aqds_qspi_SECURE_BOOT_defconfig
@@ -74,7 +74,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1088aqds_qspi_defconfig b/configs/ls1088aqds_qspi_defconfig
index 7033c92897..05747b0952 100644
--- a/configs/ls1088aqds_qspi_defconfig
+++ b/configs/ls1088aqds_qspi_defconfig
@@ -77,7 +77,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1088aqds_sdcard_ifc_defconfig b/configs/ls1088aqds_sdcard_ifc_defconfig
index e5372317c3..07dfbae2d8 100644
--- a/configs/ls1088aqds_sdcard_ifc_defconfig
+++ b/configs/ls1088aqds_sdcard_ifc_defconfig
@@ -88,7 +88,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1088aqds_sdcard_qspi_defconfig b/configs/ls1088aqds_sdcard_qspi_defconfig
index 9df9845296..dcf50ff803 100644
--- a/configs/ls1088aqds_sdcard_qspi_defconfig
+++ b/configs/ls1088aqds_sdcard_qspi_defconfig
@@ -87,7 +87,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls1088aqds_tfa_defconfig b/configs/ls1088aqds_tfa_defconfig
index aac8486bc3..4c2e9420e1 100644
--- a/configs/ls1088aqds_tfa_defconfig
+++ b/configs/ls1088aqds_tfa_defconfig
@@ -99,7 +99,7 @@ CONFIG_E1000=y
CONFIG_MII=y
CONFIG_MDIO_MUX_I2CREG=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1088ardb_qspi_SECURE_BOOT_defconfig b/configs/ls1088ardb_qspi_SECURE_BOOT_defconfig
index ed0943b6ec..c4c1a39790 100644
--- a/configs/ls1088ardb_qspi_SECURE_BOOT_defconfig
+++ b/configs/ls1088ardb_qspi_SECURE_BOOT_defconfig
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls1088ardb_qspi_defconfig b/configs/ls1088ardb_qspi_defconfig
index 486a20dbc5..2f74b87597 100644
--- a/configs/ls1088ardb_qspi_defconfig
+++ b/configs/ls1088ardb_qspi_defconfig
@@ -78,7 +78,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls1088ardb_sdcard_qspi_defconfig b/configs/ls1088ardb_sdcard_qspi_defconfig
index d54ff504a8..89e2bcf138 100644
--- a/configs/ls1088ardb_sdcard_qspi_defconfig
+++ b/configs/ls1088ardb_sdcard_qspi_defconfig
@@ -88,7 +88,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls1088ardb_tfa_SECURE_BOOT_defconfig b/configs/ls1088ardb_tfa_SECURE_BOOT_defconfig
index dd547f797b..e2b5116c4f 100644
--- a/configs/ls1088ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls1088ardb_tfa_SECURE_BOOT_defconfig
@@ -76,7 +76,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls1088ardb_tfa_defconfig b/configs/ls1088ardb_tfa_defconfig
index fc2d1b475c..8f54aa8a12 100644
--- a/configs/ls1088ardb_tfa_defconfig
+++ b/configs/ls1088ardb_tfa_defconfig
@@ -82,7 +82,7 @@ CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls2080aqds_SECURE_BOOT_defconfig b/configs/ls2080aqds_SECURE_BOOT_defconfig
index 0ad25aca76..13bbfad1e5 100644
--- a/configs/ls2080aqds_SECURE_BOOT_defconfig
+++ b/configs/ls2080aqds_SECURE_BOOT_defconfig
@@ -66,7 +66,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080aqds_defconfig b/configs/ls2080aqds_defconfig
index 30cdad6fbe..0586bd337a 100644
--- a/configs/ls2080aqds_defconfig
+++ b/configs/ls2080aqds_defconfig
@@ -69,7 +69,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080aqds_nand_defconfig b/configs/ls2080aqds_nand_defconfig
index 47f09b6288..cc74c5a686 100644
--- a/configs/ls2080aqds_nand_defconfig
+++ b/configs/ls2080aqds_nand_defconfig
@@ -77,7 +77,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080aqds_qspi_defconfig b/configs/ls2080aqds_qspi_defconfig
index bbc9f3d113..8df52491a7 100644
--- a/configs/ls2080aqds_qspi_defconfig
+++ b/configs/ls2080aqds_qspi_defconfig
@@ -66,7 +66,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080aqds_sdcard_defconfig b/configs/ls2080aqds_sdcard_defconfig
index c7a6e2a851..65a0b81f15 100644
--- a/configs/ls2080aqds_sdcard_defconfig
+++ b/configs/ls2080aqds_sdcard_defconfig
@@ -72,7 +72,7 @@ CONFIG_PHY_VITESSE=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080ardb_SECURE_BOOT_defconfig b/configs/ls2080ardb_SECURE_BOOT_defconfig
index 15dec62dd4..33a129b20b 100644
--- a/configs/ls2080ardb_SECURE_BOOT_defconfig
+++ b/configs/ls2080ardb_SECURE_BOOT_defconfig
@@ -66,7 +66,7 @@ CONFIG_PHY_CORTINA=y
CONFIG_CORTINA_FW_ADDR=0x580980000
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080ardb_defconfig b/configs/ls2080ardb_defconfig
index 909c10b631..7461713d03 100644
--- a/configs/ls2080ardb_defconfig
+++ b/configs/ls2080ardb_defconfig
@@ -69,7 +69,7 @@ CONFIG_PHY_CORTINA=y
CONFIG_CORTINA_FW_ADDR=0x580980000
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2080ardb_nand_defconfig b/configs/ls2080ardb_nand_defconfig
index f40ecb7e68..ad301b8810 100644
--- a/configs/ls2080ardb_nand_defconfig
+++ b/configs/ls2080ardb_nand_defconfig
@@ -78,7 +78,7 @@ CONFIG_PHY_CORTINA=y
CONFIG_CORTINA_FW_ADDR=0x980000
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2081ardb_defconfig b/configs/ls2081ardb_defconfig
index ed0b1b7622..e0c7e7b882 100644
--- a/configs/ls2081ardb_defconfig
+++ b/configs/ls2081ardb_defconfig
@@ -62,7 +62,7 @@ CONFIG_PHY_CORTINA=y
CONFIG_CORTINA_FW_ADDR=0x980000
CONFIG_E1000=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_DM_PCI_COMPAT=y
CONFIG_PCIE_LAYERSCAPE_RC=y
diff --git a/configs/ls2088aqds_tfa_defconfig b/configs/ls2088aqds_tfa_defconfig
index 61e0fdf113..69cb3d7c40 100644
--- a/configs/ls2088aqds_tfa_defconfig
+++ b/configs/ls2088aqds_tfa_defconfig
@@ -87,7 +87,7 @@ CONFIG_E1000=y
CONFIG_MII=y
CONFIG_MDIO_MUX_I2CREG=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls2088ardb_qspi_SECURE_BOOT_defconfig b/configs/ls2088ardb_qspi_SECURE_BOOT_defconfig
index c49d163346..037ee97a33 100644
--- a/configs/ls2088ardb_qspi_SECURE_BOOT_defconfig
+++ b/configs/ls2088ardb_qspi_SECURE_BOOT_defconfig
@@ -65,7 +65,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls2088ardb_qspi_defconfig b/configs/ls2088ardb_qspi_defconfig
index b8c7c78b8e..b08bcd44e8 100644
--- a/configs/ls2088ardb_qspi_defconfig
+++ b/configs/ls2088ardb_qspi_defconfig
@@ -72,7 +72,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_SCSI=y
diff --git a/configs/ls2088ardb_tfa_SECURE_BOOT_defconfig b/configs/ls2088ardb_tfa_SECURE_BOOT_defconfig
index 3cc9168290..590f30828f 100644
--- a/configs/ls2088ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/ls2088ardb_tfa_SECURE_BOOT_defconfig
@@ -77,7 +77,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/ls2088ardb_tfa_defconfig b/configs/ls2088ardb_tfa_defconfig
index a28b45b129..dc1abed910 100644
--- a/configs/ls2088ardb_tfa_defconfig
+++ b/configs/ls2088ardb_tfa_defconfig
@@ -85,7 +85,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_DM_RTC=y
diff --git a/configs/lx2160aqds_tfa_SECURE_BOOT_defconfig b/configs/lx2160aqds_tfa_SECURE_BOOT_defconfig
index a9a0a27888..d6f4275b9c 100644
--- a/configs/lx2160aqds_tfa_SECURE_BOOT_defconfig
+++ b/configs/lx2160aqds_tfa_SECURE_BOOT_defconfig
@@ -80,7 +80,7 @@ CONFIG_E1000=y
CONFIG_MII=y
CONFIG_MDIO_MUX_I2CREG=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_GEN4=y
diff --git a/configs/lx2160aqds_tfa_defconfig b/configs/lx2160aqds_tfa_defconfig
index de8dc553d1..920dee7916 100644
--- a/configs/lx2160aqds_tfa_defconfig
+++ b/configs/lx2160aqds_tfa_defconfig
@@ -87,7 +87,7 @@ CONFIG_E1000=y
CONFIG_MII=y
CONFIG_MDIO_MUX_I2CREG=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_GEN4=y
diff --git a/configs/lx2160ardb_tfa_SECURE_BOOT_defconfig b/configs/lx2160ardb_tfa_SECURE_BOOT_defconfig
index ac42c2508c..a82372d567 100644
--- a/configs/lx2160ardb_tfa_SECURE_BOOT_defconfig
+++ b/configs/lx2160ardb_tfa_SECURE_BOOT_defconfig
@@ -72,7 +72,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_GEN4=y
diff --git a/configs/lx2160ardb_tfa_defconfig b/configs/lx2160ardb_tfa_defconfig
index 59955eebbe..31920f4430 100644
--- a/configs/lx2160ardb_tfa_defconfig
+++ b/configs/lx2160ardb_tfa_defconfig
@@ -81,7 +81,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_GEN4=y
diff --git a/configs/lx2160ardb_tfa_stmm_defconfig b/configs/lx2160ardb_tfa_stmm_defconfig
index 149e82bed3..35bd6a26da 100644
--- a/configs/lx2160ardb_tfa_stmm_defconfig
+++ b/configs/lx2160ardb_tfa_stmm_defconfig
@@ -81,7 +81,7 @@ CONFIG_DM_MDIO=y
CONFIG_E1000=y
CONFIG_MII=y
CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_LAYERSCAPE_RC=y
CONFIG_PCIE_LAYERSCAPE_GEN4=y
diff --git a/configs/mvebu_crb_cn9130_defconfig b/configs/mvebu_crb_cn9130_defconfig
index 039fd8b3dc..f1215fa359 100644
--- a/configs/mvebu_crb_cn9130_defconfig
+++ b/configs/mvebu_crb_cn9130_defconfig
@@ -62,7 +62,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MVEBU=y
CONFIG_PHY=y
diff --git a/configs/mvebu_db_armada8k_defconfig b/configs/mvebu_db_armada8k_defconfig
index e6168a76a6..622d687e5d 100644
--- a/configs/mvebu_db_armada8k_defconfig
+++ b/configs/mvebu_db_armada8k_defconfig
@@ -51,7 +51,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MVEBU=y
CONFIG_PHY=y
diff --git a/configs/mvebu_db_cn9130_defconfig b/configs/mvebu_db_cn9130_defconfig
index 42d7038c5a..1789838691 100644
--- a/configs/mvebu_db_cn9130_defconfig
+++ b/configs/mvebu_db_cn9130_defconfig
@@ -67,7 +67,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MVEBU=y
CONFIG_PHY=y
diff --git a/configs/mvebu_espressobin-88f3720_defconfig b/configs/mvebu_espressobin-88f3720_defconfig
index 3a69954fcd..cb3ddcf2a9 100644
--- a/configs/mvebu_espressobin-88f3720_defconfig
+++ b/configs/mvebu_espressobin-88f3720_defconfig
@@ -73,7 +73,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVNETA=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_AARDVARK=y
CONFIG_PHY=y
diff --git a/configs/mvebu_mcbin-88f8040_defconfig b/configs/mvebu_mcbin-88f8040_defconfig
index 2aa06f80a0..f45ce91f6f 100644
--- a/configs/mvebu_mcbin-88f8040_defconfig
+++ b/configs/mvebu_mcbin-88f8040_defconfig
@@ -56,7 +56,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MVEBU=y
CONFIG_PHY=y
diff --git a/configs/mvebu_puzzle-m801-88f8040_defconfig b/configs/mvebu_puzzle-m801-88f8040_defconfig
index 053b2f4c96..af01e6176a 100644
--- a/configs/mvebu_puzzle-m801-88f8040_defconfig
+++ b/configs/mvebu_puzzle-m801-88f8040_defconfig
@@ -60,7 +60,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_E1000=y
CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_MVEBU=y
CONFIG_PHY=y
diff --git a/configs/nanopc-t4-rk3399_defconfig b/configs/nanopc-t4-rk3399_defconfig
index f31668c5c2..3b3da3870b 100644
--- a/configs/nanopc-t4-rk3399_defconfig
+++ b/configs/nanopc-t4-rk3399_defconfig
@@ -37,7 +37,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PMIC_RK8XX=y
CONFIG_REGULATOR_PWM=y
diff --git a/configs/octeontx2_96xx_defconfig b/configs/octeontx2_96xx_defconfig
index 1298bfe309..1ce892d963 100644
--- a/configs/octeontx2_96xx_defconfig
+++ b/configs/octeontx2_96xx_defconfig
@@ -99,7 +99,7 @@ CONFIG_E1000_SPI=y
CONFIG_CMD_E1000=y
CONFIG_NET_OCTEONTX2=y
CONFIG_OCTEONTX_SMI=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_REGION_MULTI_ENTRY=y
CONFIG_PCI_SRIOV=y
diff --git a/configs/octeontx_81xx_defconfig b/configs/octeontx_81xx_defconfig
index ba8cc97ab8..ddb9007bf1 100644
--- a/configs/octeontx_81xx_defconfig
+++ b/configs/octeontx_81xx_defconfig
@@ -99,7 +99,7 @@ CONFIG_E1000_SPI=y
CONFIG_CMD_E1000=y
CONFIG_NET_OCTEONTX=y
CONFIG_OCTEONTX_SMI=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_REGION_MULTI_ENTRY=y
CONFIG_PCI_SRIOV=y
diff --git a/configs/octeontx_83xx_defconfig b/configs/octeontx_83xx_defconfig
index 26759341c5..b8ebc2812e 100644
--- a/configs/octeontx_83xx_defconfig
+++ b/configs/octeontx_83xx_defconfig
@@ -96,7 +96,7 @@ CONFIG_E1000_SPI=y
CONFIG_CMD_E1000=y
CONFIG_NET_OCTEONTX=y
CONFIG_OCTEONTX_SMI=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_REGION_MULTI_ENTRY=y
CONFIG_PCI_SRIOV=y
diff --git a/configs/p3450-0000_defconfig b/configs/p3450-0000_defconfig
index e4265d6321..46f4cd0110 100644
--- a/configs/p3450-0000_defconfig
+++ b/configs/p3450-0000_defconfig
@@ -43,7 +43,7 @@ CONFIG_SYS_I2C_TEGRA=y
CONFIG_SF_DEFAULT_SPEED=24000000
CONFIG_SPI_FLASH_MACRONIX=y
CONFIG_RTL8169=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_TEGRA=y
CONFIG_SYS_NS16550=y
diff --git a/configs/pinebook-pro-rk3399_defconfig b/configs/pinebook-pro-rk3399_defconfig
index 81aedb28e3..d7378f5eb3 100644
--- a/configs/pinebook-pro-rk3399_defconfig
+++ b/configs/pinebook-pro-rk3399_defconfig
@@ -56,7 +56,7 @@ CONFIG_SF_DEFAULT_SPEED=20000000
CONFIG_SPI_FLASH_GIGADEVICE=y
CONFIG_SPI_FLASH_WINBOND=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/qemu-x86_64_defconfig b/configs/qemu-x86_64_defconfig
index 2dfb48b383..0d9071850a 100644
--- a/configs/qemu-x86_64_defconfig
+++ b/configs/qemu-x86_64_defconfig
@@ -56,7 +56,7 @@ CONFIG_TFTP_TSIZE=y
CONFIG_REGMAP=y
CONFIG_SYSCON=y
CONFIG_CPU=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_SPL_DM_RTC=y
CONFIG_SPI=y
CONFIG_USB_KEYBOARD=y
diff --git a/configs/qemu-x86_defconfig b/configs/qemu-x86_defconfig
index 6be7ce0c6e..ea86d044a7 100644
--- a/configs/qemu-x86_defconfig
+++ b/configs/qemu-x86_defconfig
@@ -39,7 +39,7 @@ CONFIG_TFTP_TSIZE=y
CONFIG_REGMAP=y
CONFIG_SYSCON=y
CONFIG_CPU=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_SPI=y
CONFIG_USB_KEYBOARD=y
CONFIG_FRAMEBUFFER_SET_VESA_MODE=y
diff --git a/configs/qemu_arm64_defconfig b/configs/qemu_arm64_defconfig
index 5e41ea980f..ca5534cd3a 100644
--- a/configs/qemu_arm64_defconfig
+++ b/configs/qemu_arm64_defconfig
@@ -46,7 +46,7 @@ CONFIG_FLASH_CFI_MTD=y
CONFIG_SYS_FLASH_CFI=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_SCSI=y
diff --git a/configs/qemu_arm_defconfig b/configs/qemu_arm_defconfig
index 4197cb0f92..42f85c8176 100644
--- a/configs/qemu_arm_defconfig
+++ b/configs/qemu_arm_defconfig
@@ -48,7 +48,7 @@ CONFIG_FLASH_CFI_MTD=y
CONFIG_SYS_FLASH_CFI=y
CONFIG_DM_ETH=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_ECAM_GENERIC=y
CONFIG_SCSI=y
diff --git a/configs/rcar3_salvator-x_defconfig b/configs/rcar3_salvator-x_defconfig
index 57a2c01932..e0bd1de73e 100644
--- a/configs/rcar3_salvator-x_defconfig
+++ b/configs/rcar3_salvator-x_defconfig
@@ -79,7 +79,7 @@ CONFIG_PHY_MICREL=y
CONFIG_PHY_MICREL_KSZ90X1=y
CONFIG_DM_ETH=y
CONFIG_RENESAS_RAVB=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_REGION_MULTI_ENTRY=y
CONFIG_PCI_RCAR_GEN3=y
diff --git a/configs/roc-pc-mezzanine-rk3399_defconfig b/configs/roc-pc-mezzanine-rk3399_defconfig
index ca2fb9e13f..b79200fd3b 100644
--- a/configs/roc-pc-mezzanine-rk3399_defconfig
+++ b/configs/roc-pc-mezzanine-rk3399_defconfig
@@ -49,7 +49,7 @@ CONFIG_SPI_FLASH_WINBOND=y
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/rock-pi-4-rk3399_defconfig b/configs/rock-pi-4-rk3399_defconfig
index 032b908669..73f2824f0a 100644
--- a/configs/rock-pi-4-rk3399_defconfig
+++ b/configs/rock-pi-4-rk3399_defconfig
@@ -41,7 +41,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/rock-pi-4c-rk3399_defconfig b/configs/rock-pi-4c-rk3399_defconfig
index 6f5e8666b0..ce4d7761a2 100644
--- a/configs/rock-pi-4c-rk3399_defconfig
+++ b/configs/rock-pi-4c-rk3399_defconfig
@@ -41,7 +41,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/rock-pi-n10-rk3399pro_defconfig b/configs/rock-pi-n10-rk3399pro_defconfig
index bd8b1201ef..c066d9160a 100644
--- a/configs/rock-pi-n10-rk3399pro_defconfig
+++ b/configs/rock-pi-n10-rk3399pro_defconfig
@@ -42,7 +42,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/rock960-rk3399_defconfig b/configs/rock960-rk3399_defconfig
index e46f07e74d..d95da51891 100644
--- a/configs/rock960-rk3399_defconfig
+++ b/configs/rock960-rk3399_defconfig
@@ -42,7 +42,7 @@ CONFIG_MMC_SDHCI=y
CONFIG_MMC_SDHCI_SDMA=y
CONFIG_MMC_SDHCI_ROCKCHIP=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/rockpro64-rk3399_defconfig b/configs/rockpro64-rk3399_defconfig
index 637c5c2466..d5e98a4f73 100644
--- a/configs/rockpro64-rk3399_defconfig
+++ b/configs/rockpro64-rk3399_defconfig
@@ -48,7 +48,7 @@ CONFIG_SPI_FLASH_GIGADEVICE=y
CONFIG_DM_ETH=y
CONFIG_ETH_DESIGNWARE=y
CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
diff --git a/configs/sandbox64_defconfig b/configs/sandbox64_defconfig
index d849989cf6..46f30b55e4 100644
--- a/configs/sandbox64_defconfig
+++ b/configs/sandbox64_defconfig
@@ -158,7 +158,7 @@ CONFIG_SPI_FLASH_STMICRO=y
CONFIG_SPI_FLASH_SST=y
CONFIG_SPI_FLASH_WINBOND=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_SANDBOX=y
CONFIG_PHY=y
diff --git a/configs/sandbox_defconfig b/configs/sandbox_defconfig
index 4f413582fb..7ab4e42802 100644
--- a/configs/sandbox_defconfig
+++ b/configs/sandbox_defconfig
@@ -200,7 +200,7 @@ CONFIG_SPI_FLASH_WINBOND=y
CONFIG_MULTIPLEXER=y
CONFIG_MUX_MMIO=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_REGION_MULTI_ENTRY=y
CONFIG_PCI_SANDBOX=y
diff --git a/configs/sandbox_flattree_defconfig b/configs/sandbox_flattree_defconfig
index 4d5a73fce0..325e36203f 100644
--- a/configs/sandbox_flattree_defconfig
+++ b/configs/sandbox_flattree_defconfig
@@ -134,7 +134,7 @@ CONFIG_SPI_FLASH_STMICRO=y
CONFIG_SPI_FLASH_SST=y
CONFIG_SPI_FLASH_WINBOND=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_REGION_MULTI_ENTRY=y
CONFIG_PCI_SANDBOX=y
diff --git a/configs/sandbox_noinst_defconfig b/configs/sandbox_noinst_defconfig
index 490368e768..ff107e823e 100644
--- a/configs/sandbox_noinst_defconfig
+++ b/configs/sandbox_noinst_defconfig
@@ -154,7 +154,7 @@ CONFIG_SPI_FLASH_STMICRO=y
CONFIG_SPI_FLASH_SST=y
CONFIG_SPI_FLASH_WINBOND=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_SANDBOX=y
CONFIG_PHY=y
diff --git a/configs/sandbox_spl_defconfig b/configs/sandbox_spl_defconfig
index f1a54ace9b..5d60da0edf 100644
--- a/configs/sandbox_spl_defconfig
+++ b/configs/sandbox_spl_defconfig
@@ -156,7 +156,7 @@ CONFIG_SPI_FLASH_STMICRO=y
CONFIG_SPI_FLASH_SST=y
CONFIG_SPI_FLASH_WINBOND=y
CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_SANDBOX=y
CONFIG_PHY=y
diff --git a/configs/sifive_unmatched_defconfig b/configs/sifive_unmatched_defconfig
index 299580894c..86f8c7856e 100644
--- a/configs/sifive_unmatched_defconfig
+++ b/configs/sifive_unmatched_defconfig
@@ -44,7 +44,7 @@ CONFIG_SPL_CLK=y
CONFIG_SYS_I2C_EEPROM_ADDR=0x54
CONFIG_SPI_FLASH_ISSI=y
CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCIE_DW_SIFIVE=y
CONFIG_DM_RESET=y
diff --git a/configs/synquacer_developerbox_defconfig b/configs/synquacer_developerbox_defconfig
index da57dc288f..fe12c74374 100644
--- a/configs/synquacer_developerbox_defconfig
+++ b/configs/synquacer_developerbox_defconfig
@@ -69,7 +69,7 @@ CONFIG_PHY_GIGE=y
CONFIG_RGMII=y
CONFIG_MII=y
CONFIG_SNI_NETSEC=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_DM_RTC=y
CONFIG_RTC_PCF8563=y
CONFIG_SCSI=y
diff --git a/configs/turris_mox_defconfig b/configs/turris_mox_defconfig
index 3cae32f69b..d5ab77b273 100644
--- a/configs/turris_mox_defconfig
+++ b/configs/turris_mox_defconfig
@@ -79,7 +79,7 @@ CONFIG_SPI_FLASH_MTD=y
CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_MVNETA=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_AARDVARK=y
CONFIG_PHY=y
diff --git a/configs/turris_omnia_defconfig b/configs/turris_omnia_defconfig
index 9d121b7982..cf3c5c3248 100644
--- a/configs/turris_omnia_defconfig
+++ b/configs/turris_omnia_defconfig
@@ -77,7 +77,7 @@ CONFIG_PHY_MARVELL=y
CONFIG_PHY_GIGE=y
CONFIG_MVNETA=y
CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
CONFIG_PCI=y
CONFIG_PCI_MVEBU=y
CONFIG_DM_RTC=y
diff --git a/doc/develop/driver-model/nvme.rst b/doc/develop/driver-model/nvme.rst
index 736c0a063d..fd0c0f00d2 100644
--- a/doc/develop/driver-model/nvme.rst
+++ b/doc/develop/driver-model/nvme.rst
@@ -40,6 +40,7 @@ It only support basic block read/write functions in the NVMe driver.
Config options
--------------
CONFIG_NVME Enable NVMe device support
+CONFIG_NVME_PCI Enable PCIe NVMe device support
CONFIG_CMD_NVME Enable basic NVMe commands
Usage in U-Boot
diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
index 1f6d1f5648..78da444c8b 100644
--- a/drivers/nvme/Kconfig
+++ b/drivers/nvme/Kconfig
@@ -4,8 +4,16 @@
config NVME
bool "NVM Express device support"
- depends on BLK && PCI
+ depends on BLK
select HAVE_BLOCK_DEVICE
help
This option enables support for NVM Express devices.
It supports basic functions of NVMe (read/write).
+
+config NVME_PCI
+ bool "NVM Express PCI device support"
+ depends on PCI
+ select NVME
+ help
+ This option enables support for NVM Express PCI
+ devices.
diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
index 64f102b208..fad9724e17 100644
--- a/drivers/nvme/Makefile
+++ b/drivers/nvme/Makefile
@@ -3,3 +3,4 @@
# Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_PCI) += nvme_pci.o
diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index 3c529a2fce..be518ec20b 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -12,7 +12,6 @@
#include <log.h>
#include <malloc.h>
#include <memalign.h>
-#include <pci.h>
#include <time.h>
#include <dm/device-internal.h>
#include <linux/compat.h>
@@ -698,7 +697,6 @@ static int nvme_blk_probe(struct udevice *udev)
struct blk_desc *desc = dev_get_uclass_plat(udev);
struct nvme_ns *ns = dev_get_priv(udev);
u8 flbas;
- struct pci_child_plat *pplat;
struct nvme_id_ns *id;
id = memalign(ndev->page_size, sizeof(struct nvme_id_ns));
@@ -723,8 +721,7 @@ static int nvme_blk_probe(struct udevice *udev)
desc->log2blksz = ns->lba_shift;
desc->blksz = 1 << ns->lba_shift;
desc->bdev = udev;
- pplat = dev_get_parent_plat(udev->parent);
- sprintf(desc->vendor, "0x%.4x", pplat->vendor);
+ memcpy(desc->vendor, ndev->vendor, sizeof(ndev->vendor));
memcpy(desc->product, ndev->serial, sizeof(ndev->serial));
memcpy(desc->revision, ndev->firmware_rev, sizeof(ndev->firmware_rev));
@@ -818,27 +815,13 @@ U_BOOT_DRIVER(nvme_blk) = {
.priv_auto = sizeof(struct nvme_ns),
};
-static int nvme_bind(struct udevice *udev)
+int nvme_init(struct udevice *udev)
{
- static int ndev_num;
- char name[20];
-
- sprintf(name, "nvme#%d", ndev_num++);
-
- return device_set_name(udev, name);
-}
-
-static int nvme_probe(struct udevice *udev)
-{
- int ret;
struct nvme_dev *ndev = dev_get_priv(udev);
struct nvme_id_ns *id;
-
- ndev->instance = trailing_strtol(udev->name);
+ int ret;
INIT_LIST_HEAD(&ndev->namespaces);
- ndev->bar = dm_pci_map_bar(udev, PCI_BASE_ADDRESS_0,
- PCI_REGION_MEM);
if (readl(&ndev->bar->csts) == -1) {
ret = -ENODEV;
printf("Error: %s: Out of memory!\n", udev->name);
@@ -922,18 +905,3 @@ free_queue:
free_nvme:
return ret;
}
-
-U_BOOT_DRIVER(nvme) = {
- .name = "nvme",
- .id = UCLASS_NVME,
- .bind = nvme_bind,
- .probe = nvme_probe,
- .priv_auto = sizeof(struct nvme_dev),
-};
-
-struct pci_device_id nvme_supported[] = {
- { PCI_DEVICE_CLASS(PCI_CLASS_STORAGE_EXPRESS, ~0) },
- {}
-};
-
-U_BOOT_PCI_DEVICE(nvme, nvme_supported);
diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index c6aae4da5d..8e9ae3c7f6 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -608,6 +608,7 @@ struct nvme_dev {
u32 ctrl_config;
struct nvme_bar __iomem *bar;
struct list_head namespaces;
+ char vendor[8];
char serial[20];
char model[40];
char firmware_rev[8];
@@ -635,4 +636,6 @@ struct nvme_ns {
u8 flbas;
};
+int nvme_init(struct udevice *udev);
+
#endif /* __DRIVER_NVME_H__ */
diff --git a/drivers/nvme/nvme_pci.c b/drivers/nvme/nvme_pci.c
new file mode 100644
index 0000000000..5f60fb884f
--- /dev/null
+++ b/drivers/nvme/nvme_pci.c
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2017 NXP Semiconductors
+ * Copyright (C) 2017 Bin Meng <bmeng.cn@gmail.com>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <pci.h>
+#include "nvme.h"
+
+static int nvme_bind(struct udevice *udev)
+{
+ static int ndev_num;
+ char name[20];
+
+ sprintf(name, "nvme#%d", ndev_num++);
+
+ return device_set_name(udev, name);
+}
+
+static int nvme_probe(struct udevice *udev)
+{
+ struct nvme_dev *ndev = dev_get_priv(udev);
+ struct pci_child_plat *pplat;
+
+ pplat = dev_get_parent_plat(udev);
+ sprintf(ndev->vendor, "0x%.4x", pplat->vendor);
+
+ ndev->instance = trailing_strtol(udev->name);
+ ndev->bar = dm_pci_map_bar(udev, PCI_BASE_ADDRESS_0,
+ PCI_REGION_MEM);
+ return nvme_init(udev);
+}
+
+U_BOOT_DRIVER(nvme) = {
+ .name = "nvme",
+ .id = UCLASS_NVME,
+ .bind = nvme_bind,
+ .probe = nvme_probe,
+ .priv_auto = sizeof(struct nvme_dev),
+};
+
+struct pci_device_id nvme_supported[] = {
+ { PCI_DEVICE_CLASS(PCI_CLASS_STORAGE_EXPRESS, ~0) },
+ {}
+};
+
+U_BOOT_PCI_DEVICE(nvme, nvme_supported);
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
2022-01-14 11:04 ` [PATCH 1/8] nvme: Split out PCI support Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 3/8] arm: apple: Add RTKit support Mark Kettenis
` (5 subsequent siblings)
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
This mailbox driver provides a communication channel with the
Apple IOP controllers found on Apple SoCs. These IOP controllers
are used to implement various functions such as the System
Manegement Controller (SMC) and NVMe storage. It allows sending
and receiving a 96-bit message over a single channel.
The header file with the struct used for mailbox messages is taken
straight from Linux.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
Signed-off-by: Sven Peter <sven@svenpeter.dev>
---
arch/arm/Kconfig | 1 +
drivers/mailbox/Kconfig | 11 +++++
drivers/mailbox/Makefile | 1 +
drivers/mailbox/apple-mbox.c | 92 +++++++++++++++++++++++++++++++++++
include/linux/apple-mailbox.h | 19 ++++++++
5 files changed, 124 insertions(+)
create mode 100644 drivers/mailbox/apple-mbox.c
create mode 100644 include/linux/apple-mailbox.h
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 44954977b6..79eec3aa06 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -934,6 +934,7 @@ config ARCH_APPLE
select DM
select DM_GPIO
select DM_KEYBOARD
+ select DM_MAILBOX
select DM_SERIAL
select DM_USB
select DM_VIDEO
diff --git a/drivers/mailbox/Kconfig b/drivers/mailbox/Kconfig
index dd4b0ac0c3..73db2af0b8 100644
--- a/drivers/mailbox/Kconfig
+++ b/drivers/mailbox/Kconfig
@@ -10,6 +10,17 @@ config DM_MAILBOX
the basis of a variety of inter-process/inter-CPU communication
protocols.
+config APPLE_MBOX
+ bool "Enable Apple IOP controller support"
+ depends on DM_MAILBOX && ARCH_APPLE
+ default y
+ help
+ Enable support for the mailboxes that provide a comminucation
+ channel with Apple IOP controllers integrated on Apple SoCs.
+ These IOP controllers are used to implement various functions
+ such as the System Management Controller (SMC) and NVMe and this
+ driver is required to get that functionality up and running.
+
config SANDBOX_MBOX
bool "Enable the sandbox mailbox test driver"
depends on DM_MAILBOX && SANDBOX
diff --git a/drivers/mailbox/Makefile b/drivers/mailbox/Makefile
index d2ace8cd21..59e8d0de93 100644
--- a/drivers/mailbox/Makefile
+++ b/drivers/mailbox/Makefile
@@ -4,6 +4,7 @@
#
obj-$(CONFIG_$(SPL_)DM_MAILBOX) += mailbox-uclass.o
+obj-$(CONFIG_APPLE_MBOX) += apple-mbox.o
obj-$(CONFIG_SANDBOX_MBOX) += sandbox-mbox.o
obj-$(CONFIG_SANDBOX_MBOX) += sandbox-mbox-test.o
obj-$(CONFIG_STM32_IPCC) += stm32-ipcc.o
diff --git a/drivers/mailbox/apple-mbox.c b/drivers/mailbox/apple-mbox.c
new file mode 100644
index 0000000000..30c8e2f03f
--- /dev/null
+++ b/drivers/mailbox/apple-mbox.c
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2021 Mark Kettenis <kettenis@openbsd.org>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox-uclass.h>
+#include <asm/io.h>
+#include <linux/apple-mailbox.h>
+#include <linux/delay.h>
+
+#define REG_A2I_STAT 0x110
+#define REG_A2I_STAT_EMPTY BIT(17)
+#define REG_A2I_STAT_FULL BIT(16)
+#define REG_I2A_STAT 0x114
+#define REG_I2A_STAT_EMPTY BIT(17)
+#define REG_I2A_STAT_FULL BIT(16)
+#define REG_A2I_MSG0 0x800
+#define REG_A2I_MSG1 0x808
+#define REG_I2A_MSG0 0x830
+#define REG_I2A_MSG1 0x838
+
+struct apple_mbox_priv {
+ void *base;
+};
+
+static int apple_mbox_of_xlate(struct mbox_chan *chan,
+ struct ofnode_phandle_args *args)
+{
+ if (args->args_count != 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int apple_mbox_send(struct mbox_chan *chan, const void *data)
+{
+ struct apple_mbox_priv *priv = dev_get_priv(chan->dev);
+ const struct apple_mbox_msg *msg = data;
+
+ writeq(msg->msg0, priv->base + REG_A2I_MSG0);
+ writeq(msg->msg1, priv->base + REG_A2I_MSG1);
+ while (readl(priv->base + REG_A2I_STAT) & REG_A2I_STAT_FULL)
+ udelay(1);
+
+ return 0;
+}
+
+static int apple_mbox_recv(struct mbox_chan *chan, void *data)
+{
+ struct apple_mbox_priv *priv = dev_get_priv(chan->dev);
+ struct apple_mbox_msg *msg = data;
+
+ if (readl(priv->base + REG_I2A_STAT) & REG_I2A_STAT_EMPTY)
+ return -ENODATA;
+
+ msg->msg0 = readq(priv->base + REG_I2A_MSG0);
+ msg->msg1 = readq(priv->base + REG_I2A_MSG1);
+ return 0;
+}
+
+struct mbox_ops apple_mbox_ops = {
+ .of_xlate = apple_mbox_of_xlate,
+ .send = apple_mbox_send,
+ .recv = apple_mbox_recv,
+};
+
+static int apple_mbox_probe(struct udevice *dev)
+{
+ struct apple_mbox_priv *priv = dev_get_priv(dev);
+
+ priv->base = dev_read_addr_ptr(dev);
+ if (!priv->base)
+ return -EINVAL;
+
+ return 0;
+}
+
+static const struct udevice_id apple_mbox_of_match[] = {
+ { .compatible = "apple,asc-mailbox-v4" },
+ { /* sentinel */ }
+};
+
+U_BOOT_DRIVER(apple_mbox) = {
+ .name = "apple-mbox",
+ .id = UCLASS_MAILBOX,
+ .of_match = apple_mbox_of_match,
+ .probe = apple_mbox_probe,
+ .priv_auto = sizeof(struct apple_mbox_priv),
+ .ops = &apple_mbox_ops,
+};
diff --git a/include/linux/apple-mailbox.h b/include/linux/apple-mailbox.h
new file mode 100644
index 0000000000..720fbb7029
--- /dev/null
+++ b/include/linux/apple-mailbox.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
+/*
+ * Apple mailbox message format
+ *
+ * Copyright (C) 2021 The Asahi Linux Contributors
+ */
+
+#ifndef _LINUX_APPLE_MAILBOX_H_
+#define _LINUX_APPLE_MAILBOX_H_
+
+#include <linux/types.h>
+
+/* encodes a single 96bit message sent over the single channel */
+struct apple_mbox_msg {
+ u64 msg0;
+ u32 msg1;
+};
+
+#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 3/8] arm: apple: Add RTKit support
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
2022-01-14 11:04 ` [PATCH 1/8] nvme: Split out PCI support Mark Kettenis
2022-01-14 11:04 ` [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 4/8] nvme: Introduce driver ops Mark Kettenis
` (4 subsequent siblings)
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
Most Apple IOPs run a firmware that is based on what Apple calls
RTKit. RTKit implements a common mailbox protocol. This code
provides an implementation of the AP side of this protocol,
providing a function to initialize RTKit-based firmwares as well
as a function to do a clean shutdown of this firmware.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
arch/arm/include/asm/arch-apple/rtkit.h | 11 ++
arch/arm/mach-apple/Makefile | 1 +
arch/arm/mach-apple/rtkit.c | 231 ++++++++++++++++++++++++
3 files changed, 243 insertions(+)
create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
create mode 100644 arch/arm/mach-apple/rtkit.c
diff --git a/arch/arm/include/asm/arch-apple/rtkit.h b/arch/arm/include/asm/arch-apple/rtkit.h
new file mode 100644
index 0000000000..51f77f298c
--- /dev/null
+++ b/arch/arm/include/asm/arch-apple/rtkit.h
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
+ */
+
+#define APPLE_RTKIT_PWR_STATE_SLEEP 0x01
+#define APPLE_RTKIT_PWR_STATE_QUIESCED 0x10
+#define APPLE_RTKIT_PWR_STATE_ON 0x20
+
+int apple_rtkit_init(struct mbox_chan *);
+int apple_rtkit_shutdown(struct mbox_chan *, int);
diff --git a/arch/arm/mach-apple/Makefile b/arch/arm/mach-apple/Makefile
index e74a8c9df1..52f30a777b 100644
--- a/arch/arm/mach-apple/Makefile
+++ b/arch/arm/mach-apple/Makefile
@@ -2,3 +2,4 @@
obj-y += board.o
obj-y += lowlevel_init.o
+obj-y += rtkit.o
diff --git a/arch/arm/mach-apple/rtkit.c b/arch/arm/mach-apple/rtkit.c
new file mode 100644
index 0000000000..79048a801f
--- /dev/null
+++ b/arch/arm/mach-apple/rtkit.c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
+ * (C) Copyright 2021 Copyright The Asahi Linux Contributors
+ */
+
+#include <common.h>
+#include <mailbox.h>
+#include <malloc.h>
+
+#include <asm/arch-apple/rtkit.h>
+#include <linux/apple-mailbox.h>
+#include <linux/bitfield.h>
+
+#define APPLE_RTKIT_EP_MGMT 0
+#define APPLE_RTKIT_EP_CRASHLOG 1
+#define APPLE_RTKIT_EP_SYSLOG 2
+#define APPLE_RTKIT_EP_DEBUG 3
+#define APPLE_RTKIT_EP_IOREPORT 4
+
+/* Messages for management endpoint. */
+#define APPLE_RTKIT_MGMT_TYPE GENMASK(59, 52)
+
+#define APPLE_RTKIT_MGMT_PWR_STATE GENMASK(15, 0)
+
+#define APPLE_RTKIT_MGMT_HELLO 1
+#define APPLE_RTKIT_MGMT_HELLO_REPLY 2
+#define APPLE_RTKIT_MGMT_HELLO_MINVER GENMASK(15, 0)
+#define APPLE_RTKIT_MGMT_HELLO_MAXVER GENMASK(31, 16)
+
+#define APPLE_RTKIT_MGMT_STARTEP 5
+#define APPLE_RTKIT_MGMT_STARTEP_EP GENMASK(39, 32)
+#define APPLE_RTKIT_MGMT_STARTEP_FLAG BIT(1)
+
+#define APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE 6
+#define APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE_ACK 7
+
+#define APPLE_RTKIT_MGMT_EPMAP 8
+#define APPLE_RTKIT_MGMT_EPMAP_LAST BIT(51)
+#define APPLE_RTKIT_MGMT_EPMAP_BASE GENMASK(34, 32)
+#define APPLE_RTKIT_MGMT_EPMAP_BITMAP GENMASK(31, 0)
+
+#define APPLE_RTKIT_MGMT_EPMAP_REPLY 8
+#define APPLE_RTKIT_MGMT_EPMAP_REPLY_MORE BIT(0)
+
+#define APPLE_RTKIT_MIN_SUPPORTED_VERSION 11
+#define APPLE_RTKIT_MAX_SUPPORTED_VERSION 12
+
+/* Messages for internal endpoints. */
+#define APPLE_RTKIT_BUFFER_REQUEST 1
+#define APPLE_RTKIT_BUFFER_REQUEST_SIZE GENMASK(51, 44)
+#define APPLE_RTKIT_BUFFER_REQUEST_IOVA GENMASK(41, 0)
+
+int apple_rtkit_init(struct mbox_chan *chan)
+{
+ struct apple_mbox_msg msg;
+ int endpoints[256];
+ int nendpoints = 0;
+ int endpoint;
+ int min_ver, max_ver, want_ver;
+ int msgtype, pwrstate;
+ u64 reply;
+ u32 bitmap, base;
+ int i, ret;
+
+ /* Wakup the IOP. */
+ msg.msg0 = FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE) |
+ FIELD_PREP(APPLE_RTKIT_MGMT_PWR_STATE, APPLE_RTKIT_PWR_STATE_ON);
+ msg.msg1 = APPLE_RTKIT_EP_MGMT;
+ ret = mbox_send(chan, &msg);
+ if (ret < 0)
+ return ret;
+
+ /* Wait for protocol version negotiation message. */
+ ret = mbox_recv(chan, &msg, 10000);
+ if (ret < 0)
+ return ret;
+
+ endpoint = msg.msg1;
+ msgtype = FIELD_GET(APPLE_RTKIT_MGMT_TYPE, msg.msg0);
+ if (endpoint != APPLE_RTKIT_EP_MGMT) {
+ printf("%s: unexpected endpoint %d\n", __func__, endpoint);
+ return -EINVAL;
+ }
+ if (msgtype != APPLE_RTKIT_MGMT_HELLO) {
+ printf("%s: unexpected message type %d\n", __func__, msgtype);
+ return -EINVAL;
+ }
+
+ min_ver = FIELD_GET(APPLE_RTKIT_MGMT_HELLO_MINVER, msg.msg0);
+ max_ver = FIELD_GET(APPLE_RTKIT_MGMT_HELLO_MAXVER, msg.msg0);
+ want_ver = min(APPLE_RTKIT_MAX_SUPPORTED_VERSION, max_ver);
+
+ if (min_ver > APPLE_RTKIT_MAX_SUPPORTED_VERSION) {
+ printf("%s: firmware min version %d is too new\n",
+ __func__, min_ver);
+ return -ENOTSUPP;
+ }
+
+ if (max_ver < APPLE_RTKIT_MIN_SUPPORTED_VERSION) {
+ printf("%s: firmware max version %d is too old\n",
+ __func__, max_ver);
+ return -ENOTSUPP;
+ }
+
+ /* Ack version. */
+ msg.msg0 = FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, APPLE_RTKIT_MGMT_HELLO_REPLY) |
+ FIELD_PREP(APPLE_RTKIT_MGMT_HELLO_MINVER, want_ver) |
+ FIELD_PREP(APPLE_RTKIT_MGMT_HELLO_MAXVER, want_ver);
+ msg.msg1 = APPLE_RTKIT_EP_MGMT;
+ ret = mbox_send(chan, &msg);
+ if (ret < 0)
+ return ret;
+
+wait_epmap:
+ /* Wait for endpoint map message. */
+ ret = mbox_recv(chan, &msg, 10000);
+ if (ret < 0)
+ return ret;
+
+ endpoint = msg.msg1;
+ msgtype = FIELD_GET(APPLE_RTKIT_MGMT_TYPE, msg.msg0);
+ if (endpoint != APPLE_RTKIT_EP_MGMT) {
+ printf("%s: unexpected endpoint %d\n", __func__, endpoint);
+ return -EINVAL;
+ }
+ if (msgtype != APPLE_RTKIT_MGMT_EPMAP) {
+ printf("%s: unexpected message type %d\n", __func__, msgtype);
+ return -EINVAL;
+ }
+
+ bitmap = FIELD_GET(APPLE_RTKIT_MGMT_EPMAP_BITMAP, msg.msg0);
+ base = FIELD_GET(APPLE_RTKIT_MGMT_EPMAP_BASE, msg.msg0);
+ for (i = 0; i < 32; i++) {
+ if (bitmap & (1U << i))
+ endpoints[nendpoints++] = base * 32 + i;
+ }
+
+ /* Ack endpoint map. */
+ reply = FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, APPLE_RTKIT_MGMT_EPMAP_REPLY) |
+ FIELD_PREP(APPLE_RTKIT_MGMT_EPMAP_BASE, base);
+ if (msg.msg0 & APPLE_RTKIT_MGMT_EPMAP_LAST)
+ reply |= APPLE_RTKIT_MGMT_EPMAP_LAST;
+ else
+ reply |= APPLE_RTKIT_MGMT_EPMAP_REPLY_MORE;
+ msg.msg0 = reply;
+ msg.msg1 = APPLE_RTKIT_EP_MGMT;
+ ret = mbox_send(chan, &msg);
+ if (ret < 0)
+ return ret;
+
+ if (reply & APPLE_RTKIT_MGMT_EPMAP_REPLY_MORE)
+ goto wait_epmap;
+
+ for (i = 0; i < nendpoints; i++) {
+ /* Don't start the syslog endpoint since we can't
+ easily handle its messages in U-Boot. */
+ if (endpoints[i] == APPLE_RTKIT_EP_SYSLOG)
+ continue;
+
+ /* Request endpoint. */
+ msg.msg0 = FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, APPLE_RTKIT_MGMT_STARTEP) |
+ FIELD_PREP(APPLE_RTKIT_MGMT_STARTEP_EP, endpoints[i]) |
+ APPLE_RTKIT_MGMT_STARTEP_FLAG;
+ msg.msg1 = APPLE_RTKIT_EP_MGMT;
+ ret = mbox_send(chan, &msg);
+ if (ret < 0)
+ return ret;
+ }
+
+ pwrstate = APPLE_RTKIT_PWR_STATE_SLEEP;
+ while (pwrstate != APPLE_RTKIT_PWR_STATE_ON) {
+ ret = mbox_recv(chan, &msg, 100000);
+ if (ret < 0)
+ return ret;
+
+ endpoint = msg.msg1;
+ msgtype = FIELD_GET(APPLE_RTKIT_MGMT_TYPE, msg.msg0);
+
+ if (endpoint == APPLE_RTKIT_EP_CRASHLOG ||
+ endpoint == APPLE_RTKIT_EP_SYSLOG ||
+ endpoint == APPLE_RTKIT_EP_IOREPORT) {
+ u64 addr = FIELD_GET(APPLE_RTKIT_BUFFER_REQUEST_IOVA, msg.msg0);
+ u64 size = FIELD_GET(APPLE_RTKIT_BUFFER_REQUEST_SIZE, msg.msg0);
+
+ if (msgtype == APPLE_RTKIT_BUFFER_REQUEST && addr != 0)
+ continue;
+
+ msg.msg0 = FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, APPLE_RTKIT_BUFFER_REQUEST) |
+ FIELD_PREP(APPLE_RTKIT_BUFFER_REQUEST_SIZE, size) |
+ FIELD_PREP(APPLE_RTKIT_BUFFER_REQUEST_IOVA, addr);
+ msg.msg1 = endpoint;
+ ret = mbox_send(chan, &msg);
+ if (ret < 0)
+ return ret;
+ continue;
+ }
+
+ if (endpoint != APPLE_RTKIT_EP_MGMT) {
+ printf("%s: unexpected endpoint %d\n", __func__, endpoint);
+ return -EINVAL;
+ }
+ if (msgtype != APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE_ACK) {
+ printf("%s: unexpected message type %d\n", __func__, msgtype);
+ return -EINVAL;
+ }
+
+ pwrstate = FIELD_GET(APPLE_RTKIT_MGMT_PWR_STATE, msg.msg0);
+ }
+
+ return 0;
+}
+
+int apple_rtkit_shutdown(struct mbox_chan *chan, int pwrstate)
+{
+ struct apple_mbox_msg msg;
+ int ret;
+
+ msg.msg0 = FIELD_PREP(APPLE_RTKIT_MGMT_TYPE, APPLE_RTKIT_MGMT_SET_IOP_PWR_STATE) |
+ FIELD_PREP(APPLE_RTKIT_MGMT_PWR_STATE, pwrstate);
+ msg.msg1 = APPLE_RTKIT_EP_MGMT;
+ ret = mbox_send(chan, &msg);
+ if (ret < 0)
+ return ret;
+
+ ret = mbox_recv(chan, &msg, 100000);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 4/8] nvme: Introduce driver ops
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
` (2 preceding siblings ...)
2022-01-14 11:04 ` [PATCH 3/8] arm: apple: Add RTKit support Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 5/8] nvme: Add shutdown function Mark Kettenis
` (3 subsequent siblings)
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
The NVMe storage controller integrated on Apple SoCs deviates
from the NVMe standard in two aspects. It uses a "linear"
submission queue and it integrates an NVMMU that needs to be
programmed for each NVMe command. Introduce driver ops such
that we can set up the linear submission queue and program the
NVMMU in the driver for this strange beast.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
2 files changed, 51 insertions(+), 27 deletions(-)
diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index be518ec20b..e2d0f9c668 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -27,33 +27,6 @@
#define IO_TIMEOUT 30
#define MAX_PRP_POOL 512
-enum nvme_queue_id {
- NVME_ADMIN_Q,
- NVME_IO_Q,
- NVME_Q_NUM,
-};
-
-/*
- * An NVM Express queue. Each device has at least two (one for admin
- * commands and one for I/O commands).
- */
-struct nvme_queue {
- struct nvme_dev *dev;
- struct nvme_command *sq_cmds;
- struct nvme_completion *cqes;
- wait_queue_head_t sq_full;
- u32 __iomem *q_db;
- u16 q_depth;
- s16 cq_vector;
- u16 sq_head;
- u16 sq_tail;
- u16 cq_head;
- u16 qid;
- u8 cq_phase;
- u8 cqe_seen;
- unsigned long cmdid_data[];
-};
-
static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
{
u32 bit = enabled ? NVME_CSTS_RDY : 0;
@@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
*/
static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
{
+ struct nvme_ops *ops;
u16 tail = nvmeq->sq_tail;
memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
(ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
+ ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+ if (ops && ops->submit_cmd) {
+ ops->submit_cmd(nvmeq, cmd);
+ return;
+ }
+
if (++tail == nvmeq->q_depth)
tail = 0;
writel(tail, nvmeq->q_db);
@@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
struct nvme_command *cmd,
u32 *result, unsigned timeout)
{
+ struct nvme_ops *ops;
u16 head = nvmeq->cq_head;
u16 phase = nvmeq->cq_phase;
u16 status;
@@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
return -ETIMEDOUT;
}
+ ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+ if (ops && ops->complete_cmd)
+ ops->complete_cmd(nvmeq, cmd);
+
status >>= 1;
if (status) {
printf("ERROR: status = %x, phase = %d, head = %d\n",
@@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
int qid, int depth)
{
+ struct nvme_ops *ops;
struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
if (!nvmeq)
return NULL;
@@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
dev->queue_count++;
dev->queues[qid] = nvmeq;
+ ops = (struct nvme_ops *)dev->udev->driver->ops;
+ if (ops && ops->alloc_queue)
+ ops->alloc_queue(nvmeq);
+
return nvmeq;
free_queue:
@@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
struct nvme_id_ns *id;
int ret;
+ ndev->udev = udev;
INIT_LIST_HEAD(&ndev->namespaces);
if (readl(&ndev->bar->csts) == -1) {
ret = -ENODEV;
diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index 8e9ae3c7f6..57803b43fd 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -596,6 +596,7 @@ enum {
/* Represents an NVM Express device. Each nvme_dev is a PCI function. */
struct nvme_dev {
+ struct udevice *udev;
struct list_head node;
struct nvme_queue **queues;
u32 __iomem *dbs;
@@ -622,6 +623,32 @@ struct nvme_dev {
u32 nn;
};
+enum nvme_queue_id {
+ NVME_ADMIN_Q,
+ NVME_IO_Q,
+ NVME_Q_NUM,
+};
+
+/*
+ * An NVM Express queue. Each device has at least two (one for admin
+ * commands and one for I/O commands).
+ */
+struct nvme_queue {
+ struct nvme_dev *dev;
+ struct nvme_command *sq_cmds;
+ struct nvme_completion *cqes;
+ u32 __iomem *q_db;
+ u16 q_depth;
+ s16 cq_vector;
+ u16 sq_head;
+ u16 sq_tail;
+ u16 cq_head;
+ u16 qid;
+ u8 cq_phase;
+ u8 cqe_seen;
+ unsigned long cmdid_data[];
+};
+
/*
* An NVM Express namespace is equivalent to a SCSI LUN.
* Each namespace is operated as an independent "device".
@@ -636,6 +663,12 @@ struct nvme_ns {
u8 flbas;
};
+struct nvme_ops {
+ int (*alloc_queue)(struct nvme_queue *);
+ void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
+ void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);
+};
+
int nvme_init(struct udevice *udev);
#endif /* __DRIVER_NVME_H__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 5/8] nvme: Add shutdown function
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
` (3 preceding siblings ...)
2022-01-14 11:04 ` [PATCH 4/8] nvme: Introduce driver ops Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 6/8] power: domain: apple: Add reset support Mark Kettenis
` (2 subsequent siblings)
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
Add a function to disable the NVMe controller. This will be used
to let the driver for the NVMe storage integrated on Apple SoCs
shutdown the NVMe controller such we can shutdown the NVMe
IOP controller in a clean way afterwards before handing control
to the OS.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
drivers/nvme/nvme.c | 7 +++++++
drivers/nvme/nvme.h | 1 +
2 files changed, 8 insertions(+)
diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
index e2d0f9c668..3b1a5c718f 100644
--- a/drivers/nvme/nvme.c
+++ b/drivers/nvme/nvme.c
@@ -896,3 +896,10 @@ free_queue:
free_nvme:
return ret;
}
+
+int nvme_shutdown(struct udevice *udev)
+{
+ struct nvme_dev *ndev = dev_get_priv(udev);
+
+ return nvme_disable_ctrl(ndev);
+}
diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
index 57803b43fd..6e39af8831 100644
--- a/drivers/nvme/nvme.h
+++ b/drivers/nvme/nvme.h
@@ -670,5 +670,6 @@ struct nvme_ops {
};
int nvme_init(struct udevice *udev);
+int nvme_shutdown(struct udevice *udev);
#endif /* __DRIVER_NVME_H__ */
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 6/8] power: domain: apple: Add reset support
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
` (4 preceding siblings ...)
2022-01-14 11:04 ` [PATCH 5/8] nvme: Add shutdown function Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller Mark Kettenis
2022-01-14 11:04 ` [PATCH 8/8] configs: apple: Add NVMe boot target Mark Kettenis
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
The power management controller found on Apple SoCs als provides
a way to reset all devices within a power domain. This is needed
to cleanly shutdown the NVMe controller before we hand over
control to the OS.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
arch/arm/Kconfig | 1 +
drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
2 files changed, 73 insertions(+), 1 deletion(-)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 79eec3aa06..04b4a20211 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -935,6 +935,7 @@ config ARCH_APPLE
select DM_GPIO
select DM_KEYBOARD
select DM_MAILBOX
+ select DM_RESET
select DM_SERIAL
select DM_USB
select DM_VIDEO
diff --git a/drivers/power/domain/apple-pmgr.c b/drivers/power/domain/apple-pmgr.c
index d25f136b9d..4d06e76ff5 100644
--- a/drivers/power/domain/apple-pmgr.c
+++ b/drivers/power/domain/apple-pmgr.c
@@ -6,14 +6,22 @@
#include <common.h>
#include <asm/io.h>
#include <dm.h>
+#include <dm/device-internal.h>
#include <linux/err.h>
#include <linux/bitfield.h>
#include <power-domain-uclass.h>
+#include <reset-uclass.h>
#include <regmap.h>
#include <syscon.h>
-#define APPLE_PMGR_PS_TARGET GENMASK(3, 0)
+#define APPLE_PMGR_RESET BIT(31)
+#define APPLE_PMGR_DEV_DISABLE BIT(10)
+#define APPLE_PMGR_WAS_CLKGATED BIT(9)
+#define APPLE_PMGR_WAS_PWRGATED BIT(8)
#define APPLE_PMGR_PS_ACTUAL GENMASK(7, 4)
+#define APPLE_PMGR_PS_TARGET GENMASK(3, 0)
+
+#define APPLE_PMGR_FLAGS (APPLE_PMGR_WAS_CLKGATED | APPLE_PMGR_WAS_PWRGATED)
#define APPLE_PMGR_PS_ACTIVE 0xf
#define APPLE_PMGR_PS_PWRGATE 0x0
@@ -25,6 +33,65 @@ struct apple_pmgr_priv {
u32 offset; /* offset within regmap for this domain */
};
+static int apple_reset_of_xlate(struct reset_ctl *reset_ctl,
+ struct ofnode_phandle_args *args)
+{
+ if (args->args_count != 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int apple_reset_request(struct reset_ctl *reset_ctl)
+{
+ return 0;
+}
+
+static int apple_reset_free(struct reset_ctl *reset_ctl)
+{
+ return 0;
+}
+
+static int apple_reset_assert(struct reset_ctl *reset_ctl)
+{
+ struct apple_pmgr_priv *priv = dev_get_priv(reset_ctl->dev->parent);
+
+ regmap_update_bits(priv->regmap, priv->offset,
+ APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE,
+ APPLE_PMGR_DEV_DISABLE);
+ regmap_update_bits(priv->regmap, priv->offset,
+ APPLE_PMGR_FLAGS | APPLE_PMGR_RESET,
+ APPLE_PMGR_RESET);
+
+ return 0;
+}
+
+static int apple_reset_deassert(struct reset_ctl *reset_ctl)
+{
+ struct apple_pmgr_priv *priv = dev_get_priv(reset_ctl->dev->parent);
+
+ regmap_update_bits(priv->regmap, priv->offset,
+ APPLE_PMGR_FLAGS | APPLE_PMGR_RESET, 0);
+ regmap_update_bits(priv->regmap, priv->offset,
+ APPLE_PMGR_FLAGS | APPLE_PMGR_DEV_DISABLE, 0);
+
+ return 0;
+}
+
+struct reset_ops apple_reset_ops = {
+ .of_xlate = apple_reset_of_xlate,
+ .request = apple_reset_request,
+ .rfree = apple_reset_free,
+ .rst_assert = apple_reset_assert,
+ .rst_deassert = apple_reset_deassert,
+};
+
+static struct driver apple_reset_driver = {
+ .name = "apple_reset",
+ .id = UCLASS_RESET,
+ .ops = &apple_reset_ops,
+};
+
static int apple_pmgr_request(struct power_domain *power_domain)
{
return 0;
@@ -78,6 +145,7 @@ static const struct udevice_id apple_pmgr_ids[] = {
static int apple_pmgr_probe(struct udevice *dev)
{
struct apple_pmgr_priv *priv = dev_get_priv(dev);
+ struct udevice *child;
int ret;
ret = dev_power_domain_on(dev);
@@ -92,6 +160,9 @@ static int apple_pmgr_probe(struct udevice *dev)
if (ret < 0)
return ret;
+ device_bind(dev, &apple_reset_driver, "apple_reset", NULL,
+ dev_ofnode(dev), &child);
+
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
` (5 preceding siblings ...)
2022-01-14 11:04 ` [PATCH 6/8] power: domain: apple: Add reset support Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 8/8] configs: apple: Add NVMe boot target Mark Kettenis
7 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
Add a driver for the NVMe storage controller integrated on
Apple SoCs. This NVMe controller isn't PCI based and deviates
from the NVMe standard in its implementation of the command
submission queue and the integration of an NVMMU that needs
to be managed. This commit tweaks the core NVMe code to
support the linear command submission queue implemented by
this controller. But setting up the submission queue and
managing the NVMMU controller is handled by implementing
the driver ops that were added in an earlier commit.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
Tested-on: firefly-rk3399
Tested-by: Mark Kettenis <kettenis@openbsd.org>
---
configs/apple_m1_defconfig | 1 +
drivers/nvme/Kconfig | 11 ++
drivers/nvme/Makefile | 1 +
drivers/nvme/nvme_apple.c | 233 +++++++++++++++++++++++++++++++++++++
4 files changed, 246 insertions(+)
create mode 100644 drivers/nvme/nvme_apple.c
diff --git a/configs/apple_m1_defconfig b/configs/apple_m1_defconfig
index cb235e4e7d..1528217b17 100644
--- a/configs/apple_m1_defconfig
+++ b/configs/apple_m1_defconfig
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
# CONFIG_NET is not set
# CONFIG_MMC is not set
CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_DWC3=y
CONFIG_USB_KEYBOARD=y
diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
index 78da444c8b..0cb465160b 100644
--- a/drivers/nvme/Kconfig
+++ b/drivers/nvme/Kconfig
@@ -10,6 +10,17 @@ config NVME
This option enables support for NVM Express devices.
It supports basic functions of NVMe (read/write).
+config NVME_APPLE
+ bool "Apple NVMe controller support"
+ select NVME
+ help
+ This option enables support for the NVMe storage
+ controller integrated on Apple SoCs. This controller
+ isn't PCI-based based and deviates from the NVMe
+ standard implementation in its implementation of
+ the command submission queue and the integration
+ of an NVMMU that needs to be managed.
+
config NVME_PCI
bool "NVM Express PCI device support"
depends on PCI
diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
index fad9724e17..fa7b619446 100644
--- a/drivers/nvme/Makefile
+++ b/drivers/nvme/Makefile
@@ -3,4 +3,5 @@
# Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
obj-$(CONFIG_NVME_PCI) += nvme_pci.o
diff --git a/drivers/nvme/nvme_apple.c b/drivers/nvme/nvme_apple.c
new file mode 100644
index 0000000000..b0dc8492f0
--- /dev/null
+++ b/drivers/nvme/nvme_apple.c
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox.h>
+#include <mapmem.h>
+#include "nvme.h"
+#include <reset.h>
+
+#include <asm/io.h>
+#include <asm/arch-apple/rtkit.h>
+#include <linux/iopoll.h>
+
+#undef readl_poll_timeout
+#define readl_poll_timeout readl_poll_sleep_timeout
+
+#define REG_CPU_CTRL 0x0044
+#define REG_CPU_CTRL_RUN BIT(4)
+
+#define ANS_MAX_PEND_CMDS_CTRL 0x01210
+#define ANS_MAX_QUEUE_DEPTH 64
+#define ANS_BOOT_STATUS 0x01300
+#define ANS_BOOT_STATUS_OK 0xde71ce55
+#define ANS_MODESEL 0x01304
+#define ANS_UNKNOWN_CTRL 0x24008
+#define ANS_PRP_NULL_CHECK (1 << 11)
+#define ANS_LINEAR_SQ_CTRL 0x24908
+#define ANS_LINEAR_SQ_CTRL_EN (1 << 0)
+#define ANS_ASQ_DB 0x2490c
+#define ANS_IOSQ_DB 0x24910
+#define ANS_NVMMU_NUM 0x28100
+#define ANS_NVMMU_BASE_ASQ 0x28108
+#define ANS_NVMMU_BASE_IOSQ 0x28110
+#define ANS_NVMMU_TCB_INVAL 0x28118
+#define ANS_NVMMU_TCB_STAT 0x28120
+
+#define ANS_NVMMU_TCB_SIZE 0x4000
+#define ANS_NVMMU_TCB_PITCH 0x80
+
+struct ans_nvmmu_tcb {
+ u8 opcode;
+ u8 flags;
+ u8 command_id;
+ u8 pad0;
+ u32 prpl_len;
+ u8 pad1[16];
+ u64 prp1;
+ u64 prp2;
+};
+
+#define ANS_NVMMU_TCB_WRITE BIT(0)
+#define ANS_NVMMU_TCB_READ BIT(1)
+
+struct apple_nvme_priv {
+ struct nvme_dev ndev;
+ void *base;
+ void *asc;
+ struct reset_ctl_bulk resets;
+ struct mbox_chan chan;
+ struct ans_nvmmu_tcb *tcbs[NVME_Q_NUM];
+ u32 __iomem *q_db[NVME_Q_NUM];
+};
+
+static int apple_nvme_alloc_queue(struct nvme_queue *nvmeq)
+{
+ struct apple_nvme_priv *priv =
+ container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+ struct nvme_dev *dev = nvmeq->dev;
+
+ switch (nvmeq->qid) {
+ case NVME_ADMIN_Q:
+ case NVME_IO_Q:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ priv->tcbs[nvmeq->qid] = (void *)memalign(4096, ANS_NVMMU_TCB_SIZE);
+ memset((void *)priv->tcbs[nvmeq->qid], 0, ANS_NVMMU_TCB_SIZE);
+
+ switch (nvmeq->qid) {
+ case NVME_ADMIN_Q:
+ priv->q_db[nvmeq->qid] =
+ ((void __iomem *)dev->bar) + ANS_ASQ_DB;
+ nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
+ ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_ASQ);
+ break;
+ case NVME_IO_Q:
+ priv->q_db[nvmeq->qid] =
+ ((void __iomem *)dev->bar) + ANS_IOSQ_DB;
+ nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
+ ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_IOSQ);
+ break;
+ }
+
+ return 0;
+}
+
+static void apple_nvme_submit_cmd(struct nvme_queue *nvmeq,
+ struct nvme_command *cmd)
+{
+ struct apple_nvme_priv *priv =
+ container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+ struct ans_nvmmu_tcb *tcb;
+ u16 tail = nvmeq->sq_tail;
+
+ tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
+ memset(tcb, 0, sizeof(*tcb));
+ tcb->opcode = cmd->common.opcode;
+ tcb->flags = ANS_NVMMU_TCB_WRITE | ANS_NVMMU_TCB_READ;
+ tcb->command_id = tail;
+ tcb->prpl_len = cmd->rw.length;
+ tcb->prp1 = cmd->common.prp1;
+ tcb->prp2 = cmd->common.prp2;
+
+ writel(tail, priv->q_db[nvmeq->qid]);
+}
+
+static void apple_nvme_complete_cmd(struct nvme_queue *nvmeq,
+ struct nvme_command *cmd)
+{
+ struct apple_nvme_priv *priv =
+ container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+ struct ans_nvmmu_tcb *tcb;
+ u16 tail = nvmeq->sq_tail;
+
+ tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
+ memset(tcb, 0, sizeof(*tcb));
+ writel(tail, ((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_INVAL);
+ readl(((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_STAT);
+
+ if (++tail == nvmeq->q_depth)
+ tail = 0;
+ nvmeq->sq_tail = tail;
+}
+
+static int apple_nvme_probe(struct udevice *dev)
+{
+ struct apple_nvme_priv *priv = dev_get_priv(dev);
+ fdt_addr_t addr;
+ u32 ctrl, stat;
+ int ret;
+
+ priv->base = dev_read_addr_ptr(dev);
+ if (!priv->base)
+ return -EINVAL;
+
+ addr = dev_read_addr_index(dev, 1);
+ if (addr == FDT_ADDR_T_NONE)
+ return -EINVAL;
+ priv->asc = map_sysmem(addr, 0);
+
+ ret = reset_get_bulk(dev, &priv->resets);
+ if (ret < 0)
+ return ret;
+
+ ret = mbox_get_by_index(dev, 0, &priv->chan);
+ if (ret < 0)
+ return ret;
+
+ ctrl = readl(priv->asc + REG_CPU_CTRL);
+ writel(ctrl | REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
+
+ ret = apple_rtkit_init(&priv->chan);
+ if (ret < 0)
+ return ret;
+
+ ret = readl_poll_timeout(priv->base + ANS_BOOT_STATUS, stat,
+ (stat == ANS_BOOT_STATUS_OK), 100, 500000);
+ if (ret < 0) {
+ printf("%s: NVMe firmware didn't boot\n", __func__);
+ return -ETIMEDOUT;
+ }
+
+ writel(ANS_LINEAR_SQ_CTRL_EN, priv->base + ANS_LINEAR_SQ_CTRL);
+ writel(((ANS_MAX_QUEUE_DEPTH << 16) | ANS_MAX_QUEUE_DEPTH),
+ priv->base + ANS_MAX_PEND_CMDS_CTRL);
+
+ writel(readl(priv->base + ANS_UNKNOWN_CTRL) & ~ANS_PRP_NULL_CHECK,
+ priv->base + ANS_UNKNOWN_CTRL);
+
+ strcpy(priv->ndev.vendor, "Apple");
+
+ writel((ANS_NVMMU_TCB_SIZE / ANS_NVMMU_TCB_PITCH) - 1,
+ priv->base + ANS_NVMMU_NUM);
+ writel(0, priv->base + ANS_MODESEL);
+
+ priv->ndev.bar = priv->base;
+ return nvme_init(dev);
+}
+
+static int apple_nvme_remove(struct udevice *dev)
+{
+ struct apple_nvme_priv *priv = dev_get_priv(dev);
+ u32 ctrl;
+
+ nvme_shutdown(dev);
+
+ apple_rtkit_shutdown(&priv->chan, APPLE_RTKIT_PWR_STATE_SLEEP);
+
+ ctrl = readl(priv->asc + REG_CPU_CTRL);
+ writel(ctrl & ~REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
+
+ reset_assert_bulk(&priv->resets);
+ reset_deassert_bulk(&priv->resets);
+
+ return 0;
+}
+
+static const struct nvme_ops apple_nvme_ops = {
+ .alloc_queue = apple_nvme_alloc_queue,
+ .submit_cmd = apple_nvme_submit_cmd,
+ .complete_cmd = apple_nvme_complete_cmd,
+};
+
+static const struct udevice_id apple_nvme_ids[] = {
+ { .compatible = "apple,nvme-ans2" },
+ { /* sentinel */ }
+};
+
+U_BOOT_DRIVER(apple_nvme) = {
+ .name = "apple_nvme",
+ .id = UCLASS_NVME,
+ .of_match = apple_nvme_ids,
+ .priv_auto = sizeof(struct apple_nvme_priv),
+ .probe = apple_nvme_probe,
+ .remove = apple_nvme_remove,
+ .ops = &apple_nvme_ops,
+ .flags = DM_FLAG_OS_PREPARE,
+};
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH 8/8] configs: apple: Add NVMe boot target
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
` (6 preceding siblings ...)
2022-01-14 11:04 ` [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller Mark Kettenis
@ 2022-01-14 11:04 ` Mark Kettenis
7 siblings, 0 replies; 33+ messages in thread
From: Mark Kettenis @ 2022-01-14 11:04 UTC (permalink / raw)
To: u-boot; +Cc: sjg, jh80.chung, trini, sven, marcan, bmeng.cn, Mark Kettenis
Add a boot target for NVMe such that we can boot from NVMe.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
---
include/configs/apple.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/include/configs/apple.h b/include/configs/apple.h
index 3e5fb495f1..47faad8150 100644
--- a/include/configs/apple.h
+++ b/include/configs/apple.h
@@ -13,6 +13,12 @@
"fdt_addr_r=0x960100000\0" \
"kernel_addr_r=0x960200000\0"
+#if CONFIG_IS_ENABLED(CMD_NVME)
+ #define BOOT_TARGET_NVME(func) func(NVME, nvme, 0)
+#else
+ #define BOOT_TARGET_NVME(func)
+#endif
+
#if CONFIG_IS_ENABLED(CMD_USB)
#define BOOT_TARGET_USB(func) func(USB, usb, 0)
#else
@@ -20,6 +26,7 @@
#endif
#define BOOT_TARGET_DEVICES(func) \
+ BOOT_TARGET_NVME(func) \
BOOT_TARGET_USB(func)
#include <config_distro_bootcmd.h>
--
2.34.1
^ permalink raw reply related [flat|nested] 33+ messages in thread
* Re: [PATCH 1/8] nvme: Split out PCI support
2022-01-14 11:04 ` [PATCH 1/8] nvme: Split out PCI support Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
2022-01-22 12:47 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
Hi Mark,
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> Apple SoCs have an integrated NVMe controller that isn't connected
> over a PCIe bus. In preparation for adding support for this NVMe
> controller, split out the PCI support into its own file. This file
> is selected through a new CONFIG_NVME_PCI Kconfig option, so do
> a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> ---
> configs/clearfog_gt_8k_defconfig | 2 +-
> configs/firefly-rk3399_defconfig | 2 +-
> configs/khadas-vim3_android_ab_defconfig | 2 +-
> configs/khadas-vim3_android_defconfig | 2 +-
> configs/khadas-vim3_defconfig | 2 +-
> configs/khadas-vim3l_android_ab_defconfig | 2 +-
> configs/khadas-vim3l_android_defconfig | 2 +-
> configs/khadas-vim3l_defconfig | 2 +-
> configs/kontron_sl28_defconfig | 2 +-
> configs/ls1012afrdm_qspi_defconfig | 2 +-
> configs/ls1012afrdm_tfa_defconfig | 2 +-
> .../ls1012afrwy_qspi_SECURE_BOOT_defconfig | 2 +-
> configs/ls1012afrwy_qspi_defconfig | 2 +-
> configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1012afrwy_tfa_defconfig | 2 +-
> configs/ls1012aqds_qspi_defconfig | 2 +-
> configs/ls1012aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1012aqds_tfa_defconfig | 2 +-
> configs/ls1012ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> configs/ls1012ardb_qspi_defconfig | 2 +-
> configs/ls1012ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1012ardb_tfa_defconfig | 2 +-
> configs/ls1021aiot_qspi_defconfig | 2 +-
> configs/ls1021aiot_sdcard_defconfig | 2 +-
> configs/ls1021aqds_ddr4_nor_defconfig | 2 +-
> configs/ls1021aqds_ddr4_nor_lpuart_defconfig | 2 +-
> configs/ls1021aqds_nand_defconfig | 2 +-
> configs/ls1021aqds_nor_SECURE_BOOT_defconfig | 2 +-
> configs/ls1021aqds_nor_defconfig | 2 +-
> configs/ls1021aqds_nor_lpuart_defconfig | 2 +-
> configs/ls1021aqds_qspi_defconfig | 2 +-
> configs/ls1021aqds_sdcard_ifc_defconfig | 2 +-
> configs/ls1021aqds_sdcard_qspi_defconfig | 2 +-
> configs/ls1021atsn_qspi_defconfig | 2 +-
> configs/ls1021atsn_sdcard_defconfig | 2 +-
> configs/ls1021atwr_nor_SECURE_BOOT_defconfig | 2 +-
> configs/ls1021atwr_nor_defconfig | 2 +-
> configs/ls1021atwr_nor_lpuart_defconfig | 2 +-
> configs/ls1021atwr_qspi_defconfig | 2 +-
> configs/ls1021atwr_sdcard_ifc_defconfig | 2 +-
> configs/ls1021atwr_sdcard_qspi_defconfig | 2 +-
> configs/ls1028aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1028aqds_tfa_defconfig | 2 +-
> configs/ls1028aqds_tfa_lpuart_defconfig | 2 +-
> configs/ls1028ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1028ardb_tfa_defconfig | 2 +-
> configs/ls1043aqds_defconfig | 2 +-
> configs/ls1043aqds_lpuart_defconfig | 2 +-
> configs/ls1043aqds_nand_defconfig | 2 +-
> configs/ls1043aqds_nor_ddr3_defconfig | 2 +-
> configs/ls1043aqds_qspi_defconfig | 2 +-
> configs/ls1043aqds_sdcard_ifc_defconfig | 2 +-
> configs/ls1043aqds_sdcard_qspi_defconfig | 2 +-
> configs/ls1043aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1043aqds_tfa_defconfig | 2 +-
> configs/ls1043ardb_SECURE_BOOT_defconfig | 2 +-
> configs/ls1043ardb_defconfig | 2 +-
> configs/ls1043ardb_nand_SECURE_BOOT_defconfig | 2 +-
> configs/ls1043ardb_nand_defconfig | 2 +-
> configs/ls1043ardb_sdcard_defconfig | 2 +-
> configs/ls1043ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1043ardb_tfa_defconfig | 2 +-
> configs/ls1046afrwy_tfa_defconfig | 2 +-
> configs/ls1046aqds_SECURE_BOOT_defconfig | 2 +-
> configs/ls1046aqds_defconfig | 2 +-
> configs/ls1046aqds_lpuart_defconfig | 2 +-
> configs/ls1046aqds_nand_defconfig | 2 +-
> configs/ls1046aqds_qspi_defconfig | 2 +-
> configs/ls1046aqds_sdcard_ifc_defconfig | 2 +-
> configs/ls1046aqds_sdcard_qspi_defconfig | 2 +-
> configs/ls1046aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1046aqds_tfa_defconfig | 2 +-
> configs/ls1046ardb_emmc_defconfig | 2 +-
> configs/ls1046ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> configs/ls1046ardb_qspi_defconfig | 2 +-
> configs/ls1046ardb_qspi_spl_defconfig | 2 +-
> configs/ls1046ardb_sdcard_defconfig | 2 +-
> configs/ls1046ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1046ardb_tfa_defconfig | 2 +-
> configs/ls1088aqds_defconfig | 2 +-
> configs/ls1088aqds_qspi_SECURE_BOOT_defconfig | 2 +-
> configs/ls1088aqds_qspi_defconfig | 2 +-
> configs/ls1088aqds_sdcard_ifc_defconfig | 2 +-
> configs/ls1088aqds_sdcard_qspi_defconfig | 2 +-
> configs/ls1088aqds_tfa_defconfig | 2 +-
> configs/ls1088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> configs/ls1088ardb_qspi_defconfig | 2 +-
> configs/ls1088ardb_sdcard_qspi_defconfig | 2 +-
> configs/ls1088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls1088ardb_tfa_defconfig | 2 +-
> configs/ls2080aqds_SECURE_BOOT_defconfig | 2 +-
> configs/ls2080aqds_defconfig | 2 +-
> configs/ls2080aqds_nand_defconfig | 2 +-
> configs/ls2080aqds_qspi_defconfig | 2 +-
> configs/ls2080aqds_sdcard_defconfig | 2 +-
> configs/ls2080ardb_SECURE_BOOT_defconfig | 2 +-
> configs/ls2080ardb_defconfig | 2 +-
> configs/ls2080ardb_nand_defconfig | 2 +-
> configs/ls2081ardb_defconfig | 2 +-
> configs/ls2088aqds_tfa_defconfig | 2 +-
> configs/ls2088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> configs/ls2088ardb_qspi_defconfig | 2 +-
> configs/ls2088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/ls2088ardb_tfa_defconfig | 2 +-
> configs/lx2160aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/lx2160aqds_tfa_defconfig | 2 +-
> configs/lx2160ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> configs/lx2160ardb_tfa_defconfig | 2 +-
> configs/lx2160ardb_tfa_stmm_defconfig | 2 +-
> configs/mvebu_crb_cn9130_defconfig | 2 +-
> configs/mvebu_db_armada8k_defconfig | 2 +-
> configs/mvebu_db_cn9130_defconfig | 2 +-
> configs/mvebu_espressobin-88f3720_defconfig | 2 +-
> configs/mvebu_mcbin-88f8040_defconfig | 2 +-
> configs/mvebu_puzzle-m801-88f8040_defconfig | 2 +-
> configs/nanopc-t4-rk3399_defconfig | 2 +-
> configs/octeontx2_96xx_defconfig | 2 +-
> configs/octeontx_81xx_defconfig | 2 +-
> configs/octeontx_83xx_defconfig | 2 +-
> configs/p3450-0000_defconfig | 2 +-
> configs/pinebook-pro-rk3399_defconfig | 2 +-
> configs/qemu-x86_64_defconfig | 2 +-
> configs/qemu-x86_defconfig | 2 +-
> configs/qemu_arm64_defconfig | 2 +-
> configs/qemu_arm_defconfig | 2 +-
> configs/rcar3_salvator-x_defconfig | 2 +-
> configs/roc-pc-mezzanine-rk3399_defconfig | 2 +-
> configs/rock-pi-4-rk3399_defconfig | 2 +-
> configs/rock-pi-4c-rk3399_defconfig | 2 +-
> configs/rock-pi-n10-rk3399pro_defconfig | 2 +-
> configs/rock960-rk3399_defconfig | 2 +-
> configs/rockpro64-rk3399_defconfig | 2 +-
> configs/sandbox64_defconfig | 2 +-
> configs/sandbox_defconfig | 2 +-
> configs/sandbox_flattree_defconfig | 2 +-
> configs/sandbox_noinst_defconfig | 2 +-
> configs/sandbox_spl_defconfig | 2 +-
> configs/sifive_unmatched_defconfig | 2 +-
> configs/synquacer_developerbox_defconfig | 2 +-
> configs/turris_mox_defconfig | 2 +-
> configs/turris_omnia_defconfig | 2 +-
> doc/develop/driver-model/nvme.rst | 1 +
> drivers/nvme/Kconfig | 10 +++-
> drivers/nvme/Makefile | 1 +
> drivers/nvme/nvme.c | 38 ++------------
> drivers/nvme/nvme.h | 3 ++
> drivers/nvme/nvme_pci.c | 49 +++++++++++++++++++
> 147 files changed, 207 insertions(+), 177 deletions(-)
> create mode 100644 drivers/nvme/nvme_pci.c
>
Shouldn't the vendor addition go in another patch?
Otherwise:
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox
2022-01-14 11:04 ` [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
2022-01-22 13:54 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> This mailbox driver provides a communication channel with the
> Apple IOP controllers found on Apple SoCs. These IOP controllers
> are used to implement various functions such as the System
> Manegement Controller (SMC) and NVMe storage. It allows sending
> and receiving a 96-bit message over a single channel.
>
> The header file with the struct used for mailbox messages is taken
> straight from Linux.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> Signed-off-by: Sven Peter <sven@svenpeter.dev>
> ---
> arch/arm/Kconfig | 1 +
> drivers/mailbox/Kconfig | 11 +++++
> drivers/mailbox/Makefile | 1 +
> drivers/mailbox/apple-mbox.c | 92 +++++++++++++++++++++++++++++++++++
> include/linux/apple-mailbox.h | 19 ++++++++
> 5 files changed, 124 insertions(+)
> create mode 100644 drivers/mailbox/apple-mbox.c
> create mode 100644 include/linux/apple-mailbox.h
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
with nit below
[..]
> diff --git a/include/linux/apple-mailbox.h b/include/linux/apple-mailbox.h
> new file mode 100644
> index 0000000000..720fbb7029
> --- /dev/null
> +++ b/include/linux/apple-mailbox.h
> @@ -0,0 +1,19 @@
> +/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
> +/*
> + * Apple mailbox message format
> + *
> + * Copyright (C) 2021 The Asahi Linux Contributors
> + */
> +
> +#ifndef _LINUX_APPLE_MAILBOX_H_
> +#define _LINUX_APPLE_MAILBOX_H_
Drop the _LINUX
> +
> +#include <linux/types.h>
> +
> +/* encodes a single 96bit message sent over the single channel */
> +struct apple_mbox_msg {
> + u64 msg0;
> + u32 msg1;
> +};
> +
> +#endif
> --
> 2.34.1
>
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 3/8] arm: apple: Add RTKit support
2022-01-14 11:04 ` [PATCH 3/8] arm: apple: Add RTKit support Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
2022-01-22 13:59 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
Hi Mark,
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> Most Apple IOPs run a firmware that is based on what Apple calls
> RTKit. RTKit implements a common mailbox protocol. This code
> provides an implementation of the AP side of this protocol,
> providing a function to initialize RTKit-based firmwares as well
> as a function to do a clean shutdown of this firmware.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> ---
> arch/arm/include/asm/arch-apple/rtkit.h | 11 ++
> arch/arm/mach-apple/Makefile | 1 +
> arch/arm/mach-apple/rtkit.c | 231 ++++++++++++++++++++++++
> 3 files changed, 243 insertions(+)
> create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
> create mode 100644 arch/arm/mach-apple/rtkit.c
This should be a driver.
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 4/8] nvme: Introduce driver ops
2022-01-14 11:04 ` [PATCH 4/8] nvme: Introduce driver ops Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
2022-01-22 13:33 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
Hi Mark,
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> The NVMe storage controller integrated on Apple SoCs deviates
> from the NVMe standard in two aspects. It uses a "linear"
> submission queue and it integrates an NVMMU that needs to be
> programmed for each NVMe command. Introduce driver ops such
> that we can set up the linear submission queue and program the
> NVMMU in the driver for this strange beast.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> ---
> drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
> drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
> 2 files changed, 51 insertions(+), 27 deletions(-)
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
But please see below
>
> diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
> index be518ec20b..e2d0f9c668 100644
> --- a/drivers/nvme/nvme.c
> +++ b/drivers/nvme/nvme.c
> @@ -27,33 +27,6 @@
> #define IO_TIMEOUT 30
> #define MAX_PRP_POOL 512
>
> -enum nvme_queue_id {
> - NVME_ADMIN_Q,
> - NVME_IO_Q,
> - NVME_Q_NUM,
> -};
> -
> -/*
> - * An NVM Express queue. Each device has at least two (one for admin
> - * commands and one for I/O commands).
> - */
> -struct nvme_queue {
> - struct nvme_dev *dev;
> - struct nvme_command *sq_cmds;
> - struct nvme_completion *cqes;
> - wait_queue_head_t sq_full;
> - u32 __iomem *q_db;
> - u16 q_depth;
> - s16 cq_vector;
> - u16 sq_head;
> - u16 sq_tail;
> - u16 cq_head;
> - u16 qid;
> - u8 cq_phase;
> - u8 cqe_seen;
> - unsigned long cmdid_data[];
> -};
> -
> static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
> {
> u32 bit = enabled ? NVME_CSTS_RDY : 0;
> @@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
> */
> static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
> {
> + struct nvme_ops *ops;
> u16 tail = nvmeq->sq_tail;
>
> memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
> flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
> (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
>
> + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
> + if (ops && ops->submit_cmd) {
> + ops->submit_cmd(nvmeq, cmd);
> + return;
> + }
> +
> if (++tail == nvmeq->q_depth)
> tail = 0;
> writel(tail, nvmeq->q_db);
> @@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
> struct nvme_command *cmd,
> u32 *result, unsigned timeout)
> {
> + struct nvme_ops *ops;
> u16 head = nvmeq->cq_head;
> u16 phase = nvmeq->cq_phase;
> u16 status;
> @@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
> return -ETIMEDOUT;
> }
>
> + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
> + if (ops && ops->complete_cmd)
> + ops->complete_cmd(nvmeq, cmd);
> +
> status >>= 1;
> if (status) {
> printf("ERROR: status = %x, phase = %d, head = %d\n",
> @@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
> static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
> int qid, int depth)
> {
> + struct nvme_ops *ops;
> struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
> if (!nvmeq)
> return NULL;
> @@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
> dev->queue_count++;
> dev->queues[qid] = nvmeq;
>
> + ops = (struct nvme_ops *)dev->udev->driver->ops;
> + if (ops && ops->alloc_queue)
> + ops->alloc_queue(nvmeq);
> +
> return nvmeq;
>
> free_queue:
> @@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
> struct nvme_id_ns *id;
> int ret;
>
> + ndev->udev = udev;
> INIT_LIST_HEAD(&ndev->namespaces);
> if (readl(&ndev->bar->csts) == -1) {
> ret = -ENODEV;
> diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
> index 8e9ae3c7f6..57803b43fd 100644
> --- a/drivers/nvme/nvme.h
> +++ b/drivers/nvme/nvme.h
> @@ -596,6 +596,7 @@ enum {
>
> /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
> struct nvme_dev {
> + struct udevice *udev;
> struct list_head node;
> struct nvme_queue **queues;
> u32 __iomem *dbs;
> @@ -622,6 +623,32 @@ struct nvme_dev {
> u32 nn;
> };
>
> +enum nvme_queue_id {
comment
> + NVME_ADMIN_Q,
> + NVME_IO_Q,
> + NVME_Q_NUM,
> +};
> +
> +/*
> + * An NVM Express queue. Each device has at least two (one for admin
> + * commands and one for I/O commands).
> + */
> +struct nvme_queue {
> + struct nvme_dev *dev;
> + struct nvme_command *sq_cmds;
> + struct nvme_completion *cqes;
> + u32 __iomem *q_db;
> + u16 q_depth;
> + s16 cq_vector;
> + u16 sq_head;
> + u16 sq_tail;
> + u16 cq_head;
> + u16 qid;
> + u8 cq_phase;
> + u8 cqe_seen;
> + unsigned long cmdid_data[];
> +};
> +
> /*
> * An NVM Express namespace is equivalent to a SCSI LUN.
> * Each namespace is operated as an independent "device".
> @@ -636,6 +663,12 @@ struct nvme_ns {
> u8 flbas;
> };
>
> +struct nvme_ops {
> + int (*alloc_queue)(struct nvme_queue *);
> + void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
> + void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);
each of these needs a comment
> +};
> +
> int nvme_init(struct udevice *udev);
(for later, this should be dev, not udev)
>
> #endif /* __DRIVER_NVME_H__ */
> --
> 2.34.1
>
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 5/8] nvme: Add shutdown function
2022-01-14 11:04 ` [PATCH 5/8] nvme: Add shutdown function Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
0 siblings, 0 replies; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
Hi Mark,
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> Add a function to disable the NVMe controller. This will be used
> to let the driver for the NVMe storage integrated on Apple SoCs
> shutdown the NVMe controller such we can shutdown the NVMe
> IOP controller in a clean way afterwards before handing control
> to the OS.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> ---
> drivers/nvme/nvme.c | 7 +++++++
> drivers/nvme/nvme.h | 1 +
> 2 files changed, 8 insertions(+)
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
with change below
>
> diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
> index e2d0f9c668..3b1a5c718f 100644
> --- a/drivers/nvme/nvme.c
> +++ b/drivers/nvme/nvme.c
> @@ -896,3 +896,10 @@ free_queue:
> free_nvme:
> return ret;
> }
> +
> +int nvme_shutdown(struct udevice *udev)
> +{
> + struct nvme_dev *ndev = dev_get_priv(udev);
> +
> + return nvme_disable_ctrl(ndev);
> +}
> diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
> index 57803b43fd..6e39af8831 100644
> --- a/drivers/nvme/nvme.h
> +++ b/drivers/nvme/nvme.h
> @@ -670,5 +670,6 @@ struct nvme_ops {
> };
>
> int nvme_init(struct udevice *udev);
> +int nvme_shutdown(struct udevice *udev);
Please add a comment
>
> #endif /* __DRIVER_NVME_H__ */
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 6/8] power: domain: apple: Add reset support
2022-01-14 11:04 ` [PATCH 6/8] power: domain: apple: Add reset support Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
2022-01-22 14:11 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
Hi Mark,
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> The power management controller found on Apple SoCs als provides
> a way to reset all devices within a power domain. This is needed
> to cleanly shutdown the NVMe controller before we hand over
> control to the OS.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> ---
> arch/arm/Kconfig | 1 +
> drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
> 2 files changed, 73 insertions(+), 1 deletion(-)
This should use devicetree instead of device_bind() and be a reset
driver in drivers/reset
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller
2022-01-14 11:04 ` [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller Mark Kettenis
@ 2022-01-22 1:40 ` Simon Glass
2022-01-22 14:45 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 1:40 UTC (permalink / raw)
To: Mark Kettenis
Cc: U-Boot Mailing List, Jaehoon Chung, Tom Rini, sven, marcan, Bin Meng
Hi Mark,
On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
>
> Add a driver for the NVMe storage controller integrated on
> Apple SoCs. This NVMe controller isn't PCI based and deviates
> from the NVMe standard in its implementation of the command
> submission queue and the integration of an NVMMU that needs
> to be managed. This commit tweaks the core NVMe code to
> support the linear command submission queue implemented by
> this controller. But setting up the submission queue and
> managing the NVMMU controller is handled by implementing
> the driver ops that were added in an earlier commit.
>
> Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> Tested-on: firefly-rk3399
> Tested-by: Mark Kettenis <kettenis@openbsd.org>
> ---
> configs/apple_m1_defconfig | 1 +
> drivers/nvme/Kconfig | 11 ++
> drivers/nvme/Makefile | 1 +
> drivers/nvme/nvme_apple.c | 233 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 246 insertions(+)
> create mode 100644 drivers/nvme/nvme_apple.c
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
>
> diff --git a/configs/apple_m1_defconfig b/configs/apple_m1_defconfig
> index cb235e4e7d..1528217b17 100644
> --- a/configs/apple_m1_defconfig
> +++ b/configs/apple_m1_defconfig
> @@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
> # CONFIG_NET is not set
> # CONFIG_MMC is not set
> CONFIG_DEBUG_UART_ANNOUNCE=y
> +CONFIG_NVME_APPLE=y
> CONFIG_USB_XHCI_HCD=y
> CONFIG_USB_XHCI_DWC3=y
> CONFIG_USB_KEYBOARD=y
> diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
> index 78da444c8b..0cb465160b 100644
> --- a/drivers/nvme/Kconfig
> +++ b/drivers/nvme/Kconfig
> @@ -10,6 +10,17 @@ config NVME
> This option enables support for NVM Express devices.
> It supports basic functions of NVMe (read/write).
>
> +config NVME_APPLE
> + bool "Apple NVMe controller support"
> + select NVME
> + help
> + This option enables support for the NVMe storage
> + controller integrated on Apple SoCs. This controller
> + isn't PCI-based based and deviates from the NVMe
> + standard implementation in its implementation of
> + the command submission queue and the integration
> + of an NVMMU that needs to be managed.
> +
> config NVME_PCI
> bool "NVM Express PCI device support"
> depends on PCI
> diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
> index fad9724e17..fa7b619446 100644
> --- a/drivers/nvme/Makefile
> +++ b/drivers/nvme/Makefile
> @@ -3,4 +3,5 @@
> # Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
>
> obj-y += nvme-uclass.o nvme.o nvme_show.o
> +obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
> obj-$(CONFIG_NVME_PCI) += nvme_pci.o
> diff --git a/drivers/nvme/nvme_apple.c b/drivers/nvme/nvme_apple.c
> new file mode 100644
> index 0000000000..b0dc8492f0
> --- /dev/null
> +++ b/drivers/nvme/nvme_apple.c
> @@ -0,0 +1,233 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
> + */
> +
> +#include <common.h>
> +#include <dm.h>
> +#include <mailbox.h>
> +#include <mapmem.h>
> +#include "nvme.h"
> +#include <reset.h>
> +
> +#include <asm/io.h>
> +#include <asm/arch-apple/rtkit.h>
asm/arch/ should work
> +#include <linux/iopoll.h>
> +
> +#undef readl_poll_timeout
> +#define readl_poll_timeout readl_poll_sleep_timeout
Do we need this?
> +
> +#define REG_CPU_CTRL 0x0044
> +#define REG_CPU_CTRL_RUN BIT(4)
> +
> +#define ANS_MAX_PEND_CMDS_CTRL 0x01210
> +#define ANS_MAX_QUEUE_DEPTH 64
> +#define ANS_BOOT_STATUS 0x01300
> +#define ANS_BOOT_STATUS_OK 0xde71ce55
> +#define ANS_MODESEL 0x01304
> +#define ANS_UNKNOWN_CTRL 0x24008
> +#define ANS_PRP_NULL_CHECK (1 << 11)
> +#define ANS_LINEAR_SQ_CTRL 0x24908
> +#define ANS_LINEAR_SQ_CTRL_EN (1 << 0)
> +#define ANS_ASQ_DB 0x2490c
> +#define ANS_IOSQ_DB 0x24910
> +#define ANS_NVMMU_NUM 0x28100
> +#define ANS_NVMMU_BASE_ASQ 0x28108
> +#define ANS_NVMMU_BASE_IOSQ 0x28110
> +#define ANS_NVMMU_TCB_INVAL 0x28118
> +#define ANS_NVMMU_TCB_STAT 0x28120
> +
> +#define ANS_NVMMU_TCB_SIZE 0x4000
> +#define ANS_NVMMU_TCB_PITCH 0x80
> +
> +struct ans_nvmmu_tcb {
> + u8 opcode;
> + u8 flags;
> + u8 command_id;
> + u8 pad0;
> + u32 prpl_len;
> + u8 pad1[16];
> + u64 prp1;
> + u64 prp2;
Needs comments
> +};
> +
> +#define ANS_NVMMU_TCB_WRITE BIT(0)
> +#define ANS_NVMMU_TCB_READ BIT(1)
> +
> +struct apple_nvme_priv {
> + struct nvme_dev ndev;
> + void *base;
> + void *asc;
> + struct reset_ctl_bulk resets;
> + struct mbox_chan chan;
> + struct ans_nvmmu_tcb *tcbs[NVME_Q_NUM];
> + u32 __iomem *q_db[NVME_Q_NUM];
Needs comments
> +};
> +
> +static int apple_nvme_alloc_queue(struct nvme_queue *nvmeq)
> +{
> + struct apple_nvme_priv *priv =
> + container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
> + struct nvme_dev *dev = nvmeq->dev;
> +
> + switch (nvmeq->qid) {
> + case NVME_ADMIN_Q:
> + case NVME_IO_Q:
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + priv->tcbs[nvmeq->qid] = (void *)memalign(4096, ANS_NVMMU_TCB_SIZE);
> + memset((void *)priv->tcbs[nvmeq->qid], 0, ANS_NVMMU_TCB_SIZE);
> +
> + switch (nvmeq->qid) {
> + case NVME_ADMIN_Q:
> + priv->q_db[nvmeq->qid] =
> + ((void __iomem *)dev->bar) + ANS_ASQ_DB;
> + nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
> + ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_ASQ);
> + break;
> + case NVME_IO_Q:
> + priv->q_db[nvmeq->qid] =
> + ((void __iomem *)dev->bar) + ANS_IOSQ_DB;
> + nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
> + ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_IOSQ);
> + break;
> + }
> +
> + return 0;
> +}
> +
> +static void apple_nvme_submit_cmd(struct nvme_queue *nvmeq,
> + struct nvme_command *cmd)
> +{
> + struct apple_nvme_priv *priv =
> + container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
> + struct ans_nvmmu_tcb *tcb;
> + u16 tail = nvmeq->sq_tail;
> +
> + tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
> + memset(tcb, 0, sizeof(*tcb));
> + tcb->opcode = cmd->common.opcode;
> + tcb->flags = ANS_NVMMU_TCB_WRITE | ANS_NVMMU_TCB_READ;
> + tcb->command_id = tail;
> + tcb->prpl_len = cmd->rw.length;
> + tcb->prp1 = cmd->common.prp1;
> + tcb->prp2 = cmd->common.prp2;
> +
> + writel(tail, priv->q_db[nvmeq->qid]);
> +}
> +
> +static void apple_nvme_complete_cmd(struct nvme_queue *nvmeq,
> + struct nvme_command *cmd)
> +{
> + struct apple_nvme_priv *priv =
> + container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
> + struct ans_nvmmu_tcb *tcb;
> + u16 tail = nvmeq->sq_tail;
> +
> + tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
> + memset(tcb, 0, sizeof(*tcb));
> + writel(tail, ((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_INVAL);
> + readl(((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_STAT);
> +
> + if (++tail == nvmeq->q_depth)
> + tail = 0;
> + nvmeq->sq_tail = tail;
> +}
> +
> +static int apple_nvme_probe(struct udevice *dev)
> +{
> + struct apple_nvme_priv *priv = dev_get_priv(dev);
> + fdt_addr_t addr;
> + u32 ctrl, stat;
> + int ret;
> +
> + priv->base = dev_read_addr_ptr(dev);
> + if (!priv->base)
> + return -EINVAL;
> +
> + addr = dev_read_addr_index(dev, 1);
> + if (addr == FDT_ADDR_T_NONE)
> + return -EINVAL;
> + priv->asc = map_sysmem(addr, 0);
> +
> + ret = reset_get_bulk(dev, &priv->resets);
> + if (ret < 0)
> + return ret;
> +
> + ret = mbox_get_by_index(dev, 0, &priv->chan);
Is the mailbox not mentioned in the device tre?
> + if (ret < 0)
> + return ret;
> +
> + ctrl = readl(priv->asc + REG_CPU_CTRL);
> + writel(ctrl | REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
> +
> + ret = apple_rtkit_init(&priv->chan);
Should probe a driver here
> + if (ret < 0)
> + return ret;
> +
> + ret = readl_poll_timeout(priv->base + ANS_BOOT_STATUS, stat,
> + (stat == ANS_BOOT_STATUS_OK), 100, 500000);
> + if (ret < 0) {
> + printf("%s: NVMe firmware didn't boot\n", __func__);
> + return -ETIMEDOUT;
> + }
> +
> + writel(ANS_LINEAR_SQ_CTRL_EN, priv->base + ANS_LINEAR_SQ_CTRL);
> + writel(((ANS_MAX_QUEUE_DEPTH << 16) | ANS_MAX_QUEUE_DEPTH),
> + priv->base + ANS_MAX_PEND_CMDS_CTRL);
> +
> + writel(readl(priv->base + ANS_UNKNOWN_CTRL) & ~ANS_PRP_NULL_CHECK,
> + priv->base + ANS_UNKNOWN_CTRL);
> +
> + strcpy(priv->ndev.vendor, "Apple");
> +
> + writel((ANS_NVMMU_TCB_SIZE / ANS_NVMMU_TCB_PITCH) - 1,
> + priv->base + ANS_NVMMU_NUM);
> + writel(0, priv->base + ANS_MODESEL);
> +
> + priv->ndev.bar = priv->base;
> + return nvme_init(dev);
> +}
> +
> +static int apple_nvme_remove(struct udevice *dev)
> +{
> + struct apple_nvme_priv *priv = dev_get_priv(dev);
> + u32 ctrl;
> +
> + nvme_shutdown(dev);
> +
> + apple_rtkit_shutdown(&priv->chan, APPLE_RTKIT_PWR_STATE_SLEEP);
> +
> + ctrl = readl(priv->asc + REG_CPU_CTRL);
> + writel(ctrl & ~REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
> +
> + reset_assert_bulk(&priv->resets);
> + reset_deassert_bulk(&priv->resets);
> +
> + return 0;
> +}
> +
> +static const struct nvme_ops apple_nvme_ops = {
> + .alloc_queue = apple_nvme_alloc_queue,
> + .submit_cmd = apple_nvme_submit_cmd,
> + .complete_cmd = apple_nvme_complete_cmd,
> +};
> +
> +static const struct udevice_id apple_nvme_ids[] = {
> + { .compatible = "apple,nvme-ans2" },
> + { /* sentinel */ }
> +};
> +
> +U_BOOT_DRIVER(apple_nvme) = {
> + .name = "apple_nvme",
> + .id = UCLASS_NVME,
> + .of_match = apple_nvme_ids,
> + .priv_auto = sizeof(struct apple_nvme_priv),
> + .probe = apple_nvme_probe,
> + .remove = apple_nvme_remove,
> + .ops = &apple_nvme_ops,
> + .flags = DM_FLAG_OS_PREPARE,
> +};
> --
> 2.34.1
>
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 1/8] nvme: Split out PCI support
2022-01-22 1:40 ` Simon Glass
@ 2022-01-22 12:47 ` Mark Kettenis
2022-01-22 13:18 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 12:47 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 18:40:11 -0700
>
> Hi Mark,
>
> On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> >
> > Apple SoCs have an integrated NVMe controller that isn't connected
> > over a PCIe bus. In preparation for adding support for this NVMe
> > controller, split out the PCI support into its own file. This file
> > is selected through a new CONFIG_NVME_PCI Kconfig option, so do
> > a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.
> >
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > ---
> > configs/clearfog_gt_8k_defconfig | 2 +-
> > configs/firefly-rk3399_defconfig | 2 +-
> > configs/khadas-vim3_android_ab_defconfig | 2 +-
> > configs/khadas-vim3_android_defconfig | 2 +-
> > configs/khadas-vim3_defconfig | 2 +-
> > configs/khadas-vim3l_android_ab_defconfig | 2 +-
> > configs/khadas-vim3l_android_defconfig | 2 +-
> > configs/khadas-vim3l_defconfig | 2 +-
> > configs/kontron_sl28_defconfig | 2 +-
> > configs/ls1012afrdm_qspi_defconfig | 2 +-
> > configs/ls1012afrdm_tfa_defconfig | 2 +-
> > .../ls1012afrwy_qspi_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1012afrwy_qspi_defconfig | 2 +-
> > configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1012afrwy_tfa_defconfig | 2 +-
> > configs/ls1012aqds_qspi_defconfig | 2 +-
> > configs/ls1012aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1012aqds_tfa_defconfig | 2 +-
> > configs/ls1012ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1012ardb_qspi_defconfig | 2 +-
> > configs/ls1012ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1012ardb_tfa_defconfig | 2 +-
> > configs/ls1021aiot_qspi_defconfig | 2 +-
> > configs/ls1021aiot_sdcard_defconfig | 2 +-
> > configs/ls1021aqds_ddr4_nor_defconfig | 2 +-
> > configs/ls1021aqds_ddr4_nor_lpuart_defconfig | 2 +-
> > configs/ls1021aqds_nand_defconfig | 2 +-
> > configs/ls1021aqds_nor_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1021aqds_nor_defconfig | 2 +-
> > configs/ls1021aqds_nor_lpuart_defconfig | 2 +-
> > configs/ls1021aqds_qspi_defconfig | 2 +-
> > configs/ls1021aqds_sdcard_ifc_defconfig | 2 +-
> > configs/ls1021aqds_sdcard_qspi_defconfig | 2 +-
> > configs/ls1021atsn_qspi_defconfig | 2 +-
> > configs/ls1021atsn_sdcard_defconfig | 2 +-
> > configs/ls1021atwr_nor_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1021atwr_nor_defconfig | 2 +-
> > configs/ls1021atwr_nor_lpuart_defconfig | 2 +-
> > configs/ls1021atwr_qspi_defconfig | 2 +-
> > configs/ls1021atwr_sdcard_ifc_defconfig | 2 +-
> > configs/ls1021atwr_sdcard_qspi_defconfig | 2 +-
> > configs/ls1028aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1028aqds_tfa_defconfig | 2 +-
> > configs/ls1028aqds_tfa_lpuart_defconfig | 2 +-
> > configs/ls1028ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1028ardb_tfa_defconfig | 2 +-
> > configs/ls1043aqds_defconfig | 2 +-
> > configs/ls1043aqds_lpuart_defconfig | 2 +-
> > configs/ls1043aqds_nand_defconfig | 2 +-
> > configs/ls1043aqds_nor_ddr3_defconfig | 2 +-
> > configs/ls1043aqds_qspi_defconfig | 2 +-
> > configs/ls1043aqds_sdcard_ifc_defconfig | 2 +-
> > configs/ls1043aqds_sdcard_qspi_defconfig | 2 +-
> > configs/ls1043aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1043aqds_tfa_defconfig | 2 +-
> > configs/ls1043ardb_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1043ardb_defconfig | 2 +-
> > configs/ls1043ardb_nand_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1043ardb_nand_defconfig | 2 +-
> > configs/ls1043ardb_sdcard_defconfig | 2 +-
> > configs/ls1043ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1043ardb_tfa_defconfig | 2 +-
> > configs/ls1046afrwy_tfa_defconfig | 2 +-
> > configs/ls1046aqds_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1046aqds_defconfig | 2 +-
> > configs/ls1046aqds_lpuart_defconfig | 2 +-
> > configs/ls1046aqds_nand_defconfig | 2 +-
> > configs/ls1046aqds_qspi_defconfig | 2 +-
> > configs/ls1046aqds_sdcard_ifc_defconfig | 2 +-
> > configs/ls1046aqds_sdcard_qspi_defconfig | 2 +-
> > configs/ls1046aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1046aqds_tfa_defconfig | 2 +-
> > configs/ls1046ardb_emmc_defconfig | 2 +-
> > configs/ls1046ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1046ardb_qspi_defconfig | 2 +-
> > configs/ls1046ardb_qspi_spl_defconfig | 2 +-
> > configs/ls1046ardb_sdcard_defconfig | 2 +-
> > configs/ls1046ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1046ardb_tfa_defconfig | 2 +-
> > configs/ls1088aqds_defconfig | 2 +-
> > configs/ls1088aqds_qspi_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1088aqds_qspi_defconfig | 2 +-
> > configs/ls1088aqds_sdcard_ifc_defconfig | 2 +-
> > configs/ls1088aqds_sdcard_qspi_defconfig | 2 +-
> > configs/ls1088aqds_tfa_defconfig | 2 +-
> > configs/ls1088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1088ardb_qspi_defconfig | 2 +-
> > configs/ls1088ardb_sdcard_qspi_defconfig | 2 +-
> > configs/ls1088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls1088ardb_tfa_defconfig | 2 +-
> > configs/ls2080aqds_SECURE_BOOT_defconfig | 2 +-
> > configs/ls2080aqds_defconfig | 2 +-
> > configs/ls2080aqds_nand_defconfig | 2 +-
> > configs/ls2080aqds_qspi_defconfig | 2 +-
> > configs/ls2080aqds_sdcard_defconfig | 2 +-
> > configs/ls2080ardb_SECURE_BOOT_defconfig | 2 +-
> > configs/ls2080ardb_defconfig | 2 +-
> > configs/ls2080ardb_nand_defconfig | 2 +-
> > configs/ls2081ardb_defconfig | 2 +-
> > configs/ls2088aqds_tfa_defconfig | 2 +-
> > configs/ls2088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > configs/ls2088ardb_qspi_defconfig | 2 +-
> > configs/ls2088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/ls2088ardb_tfa_defconfig | 2 +-
> > configs/lx2160aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/lx2160aqds_tfa_defconfig | 2 +-
> > configs/lx2160ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > configs/lx2160ardb_tfa_defconfig | 2 +-
> > configs/lx2160ardb_tfa_stmm_defconfig | 2 +-
> > configs/mvebu_crb_cn9130_defconfig | 2 +-
> > configs/mvebu_db_armada8k_defconfig | 2 +-
> > configs/mvebu_db_cn9130_defconfig | 2 +-
> > configs/mvebu_espressobin-88f3720_defconfig | 2 +-
> > configs/mvebu_mcbin-88f8040_defconfig | 2 +-
> > configs/mvebu_puzzle-m801-88f8040_defconfig | 2 +-
> > configs/nanopc-t4-rk3399_defconfig | 2 +-
> > configs/octeontx2_96xx_defconfig | 2 +-
> > configs/octeontx_81xx_defconfig | 2 +-
> > configs/octeontx_83xx_defconfig | 2 +-
> > configs/p3450-0000_defconfig | 2 +-
> > configs/pinebook-pro-rk3399_defconfig | 2 +-
> > configs/qemu-x86_64_defconfig | 2 +-
> > configs/qemu-x86_defconfig | 2 +-
> > configs/qemu_arm64_defconfig | 2 +-
> > configs/qemu_arm_defconfig | 2 +-
> > configs/rcar3_salvator-x_defconfig | 2 +-
> > configs/roc-pc-mezzanine-rk3399_defconfig | 2 +-
> > configs/rock-pi-4-rk3399_defconfig | 2 +-
> > configs/rock-pi-4c-rk3399_defconfig | 2 +-
> > configs/rock-pi-n10-rk3399pro_defconfig | 2 +-
> > configs/rock960-rk3399_defconfig | 2 +-
> > configs/rockpro64-rk3399_defconfig | 2 +-
> > configs/sandbox64_defconfig | 2 +-
> > configs/sandbox_defconfig | 2 +-
> > configs/sandbox_flattree_defconfig | 2 +-
> > configs/sandbox_noinst_defconfig | 2 +-
> > configs/sandbox_spl_defconfig | 2 +-
> > configs/sifive_unmatched_defconfig | 2 +-
> > configs/synquacer_developerbox_defconfig | 2 +-
> > configs/turris_mox_defconfig | 2 +-
> > configs/turris_omnia_defconfig | 2 +-
> > doc/develop/driver-model/nvme.rst | 1 +
> > drivers/nvme/Kconfig | 10 +++-
> > drivers/nvme/Makefile | 1 +
> > drivers/nvme/nvme.c | 38 ++------------
> > drivers/nvme/nvme.h | 3 ++
> > drivers/nvme/nvme_pci.c | 49 +++++++++++++++++++
> > 147 files changed, 207 insertions(+), 177 deletions(-)
> > create mode 100644 drivers/nvme/nvme_pci.c
> >
>
> Shouldn't the vendor addition go in another patch?
I don't think that would make a ton of sense. It is intricately
linked to splitting out the PCI support, as the current code grovels
into pci_child_plat struct in a place that needs to be bus-agnostic.
> Otherwise:
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Tested on: Macbook Air M1
> Tested-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 1/8] nvme: Split out PCI support
2022-01-22 12:47 ` Mark Kettenis
@ 2022-01-22 13:18 ` Simon Glass
2022-01-22 14:57 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 13:18 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 05:48, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 18:40:11 -0700
> >
> > Hi Mark,
> >
> > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > >
> > > Apple SoCs have an integrated NVMe controller that isn't connected
> > > over a PCIe bus. In preparation for adding support for this NVMe
> > > controller, split out the PCI support into its own file. This file
> > > is selected through a new CONFIG_NVME_PCI Kconfig option, so do
> > > a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.
> > >
> > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > ---
> > > configs/clearfog_gt_8k_defconfig | 2 +-
> > > configs/firefly-rk3399_defconfig | 2 +-
> > > configs/khadas-vim3_android_ab_defconfig | 2 +-
> > > configs/khadas-vim3_android_defconfig | 2 +-
> > > configs/khadas-vim3_defconfig | 2 +-
> > > configs/khadas-vim3l_android_ab_defconfig | 2 +-
> > > configs/khadas-vim3l_android_defconfig | 2 +-
> > > configs/khadas-vim3l_defconfig | 2 +-
> > > configs/kontron_sl28_defconfig | 2 +-
> > > configs/ls1012afrdm_qspi_defconfig | 2 +-
> > > configs/ls1012afrdm_tfa_defconfig | 2 +-
> > > .../ls1012afrwy_qspi_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1012afrwy_qspi_defconfig | 2 +-
> > > configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1012afrwy_tfa_defconfig | 2 +-
> > > configs/ls1012aqds_qspi_defconfig | 2 +-
> > > configs/ls1012aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1012aqds_tfa_defconfig | 2 +-
> > > configs/ls1012ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1012ardb_qspi_defconfig | 2 +-
> > > configs/ls1012ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1012ardb_tfa_defconfig | 2 +-
> > > configs/ls1021aiot_qspi_defconfig | 2 +-
> > > configs/ls1021aiot_sdcard_defconfig | 2 +-
> > > configs/ls1021aqds_ddr4_nor_defconfig | 2 +-
> > > configs/ls1021aqds_ddr4_nor_lpuart_defconfig | 2 +-
> > > configs/ls1021aqds_nand_defconfig | 2 +-
> > > configs/ls1021aqds_nor_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1021aqds_nor_defconfig | 2 +-
> > > configs/ls1021aqds_nor_lpuart_defconfig | 2 +-
> > > configs/ls1021aqds_qspi_defconfig | 2 +-
> > > configs/ls1021aqds_sdcard_ifc_defconfig | 2 +-
> > > configs/ls1021aqds_sdcard_qspi_defconfig | 2 +-
> > > configs/ls1021atsn_qspi_defconfig | 2 +-
> > > configs/ls1021atsn_sdcard_defconfig | 2 +-
> > > configs/ls1021atwr_nor_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1021atwr_nor_defconfig | 2 +-
> > > configs/ls1021atwr_nor_lpuart_defconfig | 2 +-
> > > configs/ls1021atwr_qspi_defconfig | 2 +-
> > > configs/ls1021atwr_sdcard_ifc_defconfig | 2 +-
> > > configs/ls1021atwr_sdcard_qspi_defconfig | 2 +-
> > > configs/ls1028aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1028aqds_tfa_defconfig | 2 +-
> > > configs/ls1028aqds_tfa_lpuart_defconfig | 2 +-
> > > configs/ls1028ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1028ardb_tfa_defconfig | 2 +-
> > > configs/ls1043aqds_defconfig | 2 +-
> > > configs/ls1043aqds_lpuart_defconfig | 2 +-
> > > configs/ls1043aqds_nand_defconfig | 2 +-
> > > configs/ls1043aqds_nor_ddr3_defconfig | 2 +-
> > > configs/ls1043aqds_qspi_defconfig | 2 +-
> > > configs/ls1043aqds_sdcard_ifc_defconfig | 2 +-
> > > configs/ls1043aqds_sdcard_qspi_defconfig | 2 +-
> > > configs/ls1043aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1043aqds_tfa_defconfig | 2 +-
> > > configs/ls1043ardb_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1043ardb_defconfig | 2 +-
> > > configs/ls1043ardb_nand_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1043ardb_nand_defconfig | 2 +-
> > > configs/ls1043ardb_sdcard_defconfig | 2 +-
> > > configs/ls1043ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1043ardb_tfa_defconfig | 2 +-
> > > configs/ls1046afrwy_tfa_defconfig | 2 +-
> > > configs/ls1046aqds_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1046aqds_defconfig | 2 +-
> > > configs/ls1046aqds_lpuart_defconfig | 2 +-
> > > configs/ls1046aqds_nand_defconfig | 2 +-
> > > configs/ls1046aqds_qspi_defconfig | 2 +-
> > > configs/ls1046aqds_sdcard_ifc_defconfig | 2 +-
> > > configs/ls1046aqds_sdcard_qspi_defconfig | 2 +-
> > > configs/ls1046aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1046aqds_tfa_defconfig | 2 +-
> > > configs/ls1046ardb_emmc_defconfig | 2 +-
> > > configs/ls1046ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1046ardb_qspi_defconfig | 2 +-
> > > configs/ls1046ardb_qspi_spl_defconfig | 2 +-
> > > configs/ls1046ardb_sdcard_defconfig | 2 +-
> > > configs/ls1046ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1046ardb_tfa_defconfig | 2 +-
> > > configs/ls1088aqds_defconfig | 2 +-
> > > configs/ls1088aqds_qspi_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1088aqds_qspi_defconfig | 2 +-
> > > configs/ls1088aqds_sdcard_ifc_defconfig | 2 +-
> > > configs/ls1088aqds_sdcard_qspi_defconfig | 2 +-
> > > configs/ls1088aqds_tfa_defconfig | 2 +-
> > > configs/ls1088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1088ardb_qspi_defconfig | 2 +-
> > > configs/ls1088ardb_sdcard_qspi_defconfig | 2 +-
> > > configs/ls1088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls1088ardb_tfa_defconfig | 2 +-
> > > configs/ls2080aqds_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls2080aqds_defconfig | 2 +-
> > > configs/ls2080aqds_nand_defconfig | 2 +-
> > > configs/ls2080aqds_qspi_defconfig | 2 +-
> > > configs/ls2080aqds_sdcard_defconfig | 2 +-
> > > configs/ls2080ardb_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls2080ardb_defconfig | 2 +-
> > > configs/ls2080ardb_nand_defconfig | 2 +-
> > > configs/ls2081ardb_defconfig | 2 +-
> > > configs/ls2088aqds_tfa_defconfig | 2 +-
> > > configs/ls2088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls2088ardb_qspi_defconfig | 2 +-
> > > configs/ls2088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/ls2088ardb_tfa_defconfig | 2 +-
> > > configs/lx2160aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/lx2160aqds_tfa_defconfig | 2 +-
> > > configs/lx2160ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > configs/lx2160ardb_tfa_defconfig | 2 +-
> > > configs/lx2160ardb_tfa_stmm_defconfig | 2 +-
> > > configs/mvebu_crb_cn9130_defconfig | 2 +-
> > > configs/mvebu_db_armada8k_defconfig | 2 +-
> > > configs/mvebu_db_cn9130_defconfig | 2 +-
> > > configs/mvebu_espressobin-88f3720_defconfig | 2 +-
> > > configs/mvebu_mcbin-88f8040_defconfig | 2 +-
> > > configs/mvebu_puzzle-m801-88f8040_defconfig | 2 +-
> > > configs/nanopc-t4-rk3399_defconfig | 2 +-
> > > configs/octeontx2_96xx_defconfig | 2 +-
> > > configs/octeontx_81xx_defconfig | 2 +-
> > > configs/octeontx_83xx_defconfig | 2 +-
> > > configs/p3450-0000_defconfig | 2 +-
> > > configs/pinebook-pro-rk3399_defconfig | 2 +-
> > > configs/qemu-x86_64_defconfig | 2 +-
> > > configs/qemu-x86_defconfig | 2 +-
> > > configs/qemu_arm64_defconfig | 2 +-
> > > configs/qemu_arm_defconfig | 2 +-
> > > configs/rcar3_salvator-x_defconfig | 2 +-
> > > configs/roc-pc-mezzanine-rk3399_defconfig | 2 +-
> > > configs/rock-pi-4-rk3399_defconfig | 2 +-
> > > configs/rock-pi-4c-rk3399_defconfig | 2 +-
> > > configs/rock-pi-n10-rk3399pro_defconfig | 2 +-
> > > configs/rock960-rk3399_defconfig | 2 +-
> > > configs/rockpro64-rk3399_defconfig | 2 +-
> > > configs/sandbox64_defconfig | 2 +-
> > > configs/sandbox_defconfig | 2 +-
> > > configs/sandbox_flattree_defconfig | 2 +-
> > > configs/sandbox_noinst_defconfig | 2 +-
> > > configs/sandbox_spl_defconfig | 2 +-
> > > configs/sifive_unmatched_defconfig | 2 +-
> > > configs/synquacer_developerbox_defconfig | 2 +-
> > > configs/turris_mox_defconfig | 2 +-
> > > configs/turris_omnia_defconfig | 2 +-
> > > doc/develop/driver-model/nvme.rst | 1 +
> > > drivers/nvme/Kconfig | 10 +++-
> > > drivers/nvme/Makefile | 1 +
> > > drivers/nvme/nvme.c | 38 ++------------
> > > drivers/nvme/nvme.h | 3 ++
> > > drivers/nvme/nvme_pci.c | 49 +++++++++++++++++++
> > > 147 files changed, 207 insertions(+), 177 deletions(-)
> > > create mode 100644 drivers/nvme/nvme_pci.c
> > >
> >
> > Shouldn't the vendor addition go in another patch?
>
> I don't think that would make a ton of sense. It is intricately
> linked to splitting out the PCI support, as the current code grovels
> into pci_child_plat struct in a place that needs to be bus-agnostic.
OK.
BTW I noticed that the U-Boot logo has the wrong colours on the
Macbook Air, with these patches applied to mainline.
>
> > Otherwise:
> > Reviewed-by: Simon Glass <sjg@chromium.org>
> > Tested on: Macbook Air M1
> > Tested-by: Simon Glass <sjg@chromium.org>
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 4/8] nvme: Introduce driver ops
2022-01-22 1:40 ` Simon Glass
@ 2022-01-22 13:33 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 13:33 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 18:40:19 -0700
>
> Hi Mark,
>
> On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> >
> > The NVMe storage controller integrated on Apple SoCs deviates
> > from the NVMe standard in two aspects. It uses a "linear"
> > submission queue and it integrates an NVMMU that needs to be
> > programmed for each NVMe command. Introduce driver ops such
> > that we can set up the linear submission queue and program the
> > NVMMU in the driver for this strange beast.
> >
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > ---
> > drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
> > drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
> > 2 files changed, 51 insertions(+), 27 deletions(-)
>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Tested on: Macbook Air M1
> Tested-by: Simon Glass <sjg@chromium.org>
Thanks...
> But please see below
>
> >
> > diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
> > index be518ec20b..e2d0f9c668 100644
> > --- a/drivers/nvme/nvme.c
> > +++ b/drivers/nvme/nvme.c
> > @@ -27,33 +27,6 @@
> > #define IO_TIMEOUT 30
> > #define MAX_PRP_POOL 512
> >
> > -enum nvme_queue_id {
> > - NVME_ADMIN_Q,
> > - NVME_IO_Q,
> > - NVME_Q_NUM,
> > -};
> > -
> > -/*
> > - * An NVM Express queue. Each device has at least two (one for admin
> > - * commands and one for I/O commands).
> > - */
> > -struct nvme_queue {
> > - struct nvme_dev *dev;
> > - struct nvme_command *sq_cmds;
> > - struct nvme_completion *cqes;
> > - wait_queue_head_t sq_full;
> > - u32 __iomem *q_db;
> > - u16 q_depth;
> > - s16 cq_vector;
> > - u16 sq_head;
> > - u16 sq_tail;
> > - u16 cq_head;
> > - u16 qid;
> > - u8 cq_phase;
> > - u8 cqe_seen;
> > - unsigned long cmdid_data[];
> > -};
> > -
> > static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
> > {
> > u32 bit = enabled ? NVME_CSTS_RDY : 0;
> > @@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
> > */
> > static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
> > {
> > + struct nvme_ops *ops;
> > u16 tail = nvmeq->sq_tail;
> >
> > memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
> > flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
> > (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
> >
> > + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
> > + if (ops && ops->submit_cmd) {
> > + ops->submit_cmd(nvmeq, cmd);
> > + return;
> > + }
> > +
> > if (++tail == nvmeq->q_depth)
> > tail = 0;
> > writel(tail, nvmeq->q_db);
> > @@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
> > struct nvme_command *cmd,
> > u32 *result, unsigned timeout)
> > {
> > + struct nvme_ops *ops;
> > u16 head = nvmeq->cq_head;
> > u16 phase = nvmeq->cq_phase;
> > u16 status;
> > @@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
> > return -ETIMEDOUT;
> > }
> >
> > + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
> > + if (ops && ops->complete_cmd)
> > + ops->complete_cmd(nvmeq, cmd);
> > +
> > status >>= 1;
> > if (status) {
> > printf("ERROR: status = %x, phase = %d, head = %d\n",
> > @@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
> > static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
> > int qid, int depth)
> > {
> > + struct nvme_ops *ops;
> > struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
> > if (!nvmeq)
> > return NULL;
> > @@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
> > dev->queue_count++;
> > dev->queues[qid] = nvmeq;
> >
> > + ops = (struct nvme_ops *)dev->udev->driver->ops;
> > + if (ops && ops->alloc_queue)
> > + ops->alloc_queue(nvmeq);
> > +
> > return nvmeq;
> >
> > free_queue:
> > @@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
> > struct nvme_id_ns *id;
> > int ret;
> >
> > + ndev->udev = udev;
> > INIT_LIST_HEAD(&ndev->namespaces);
> > if (readl(&ndev->bar->csts) == -1) {
> > ret = -ENODEV;
> > diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
> > index 8e9ae3c7f6..57803b43fd 100644
> > --- a/drivers/nvme/nvme.h
> > +++ b/drivers/nvme/nvme.h
> > @@ -596,6 +596,7 @@ enum {
> >
> > /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
> > struct nvme_dev {
> > + struct udevice *udev;
> > struct list_head node;
> > struct nvme_queue **queues;
> > u32 __iomem *dbs;
> > @@ -622,6 +623,32 @@ struct nvme_dev {
> > u32 nn;
> > };
> >
> > +enum nvme_queue_id {
>
> comment
Hmm, I merely moved this definition. Feels a bit awkward to come up
with a comment when I didn't write the bit of code in question. I can
come up with something though if you insist.
>
> > + NVME_ADMIN_Q,
> > + NVME_IO_Q,
> > + NVME_Q_NUM,
> > +};
> > +
> > +/*
> > + * An NVM Express queue. Each device has at least two (one for admin
> > + * commands and one for I/O commands).
> > + */
> > +struct nvme_queue {
> > + struct nvme_dev *dev;
> > + struct nvme_command *sq_cmds;
> > + struct nvme_completion *cqes;
> > + u32 __iomem *q_db;
> > + u16 q_depth;
> > + s16 cq_vector;
> > + u16 sq_head;
> > + u16 sq_tail;
> > + u16 cq_head;
> > + u16 qid;
> > + u8 cq_phase;
> > + u8 cqe_seen;
> > + unsigned long cmdid_data[];
> > +};
> > +
> > /*
> > * An NVM Express namespace is equivalent to a SCSI LUN.
> > * Each namespace is operated as an independent "device".
> > @@ -636,6 +663,12 @@ struct nvme_ns {
> > u8 flbas;
> > };
> >
> > +struct nvme_ops {
> > + int (*alloc_queue)(struct nvme_queue *);
> > + void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
> > + void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);
>
> each of these needs a comment
Indeed. While writing the comment, I realized that setup_queue would
be a better name than alloc_queue, so I changed that. Is that a
small-enough change to retain your tags?
>
> > +};
> > +
> > int nvme_init(struct udevice *udev);
>
> (for later, this should be dev, not udev)
As in, this is fine for now but will need to be changed at some point
in the future?
>
> >
> > #endif /* __DRIVER_NVME_H__ */
> > --
> > 2.34.1
> >
>
> Regards,
> Simon
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox
2022-01-22 1:40 ` Simon Glass
@ 2022-01-22 13:54 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 13:54 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 18:40:12 -0700
>
> On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> >
> > This mailbox driver provides a communication channel with the
> > Apple IOP controllers found on Apple SoCs. These IOP controllers
> > are used to implement various functions such as the System
> > Manegement Controller (SMC) and NVMe storage. It allows sending
> > and receiving a 96-bit message over a single channel.
> >
> > The header file with the struct used for mailbox messages is taken
> > straight from Linux.
> >
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > Signed-off-by: Sven Peter <sven@svenpeter.dev>
> > ---
> > arch/arm/Kconfig | 1 +
> > drivers/mailbox/Kconfig | 11 +++++
> > drivers/mailbox/Makefile | 1 +
> > drivers/mailbox/apple-mbox.c | 92 +++++++++++++++++++++++++++++++++++
> > include/linux/apple-mailbox.h | 19 ++++++++
> > 5 files changed, 124 insertions(+)
> > create mode 100644 drivers/mailbox/apple-mbox.c
> > create mode 100644 include/linux/apple-mailbox.h
>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Tested on: Macbook Air M1
> Tested-by: Simon Glass <sjg@chromium.org>
>
> with nit below
>
> [..]
>
> > diff --git a/include/linux/apple-mailbox.h b/include/linux/apple-mailbox.h
> > new file mode 100644
> > index 0000000000..720fbb7029
> > --- /dev/null
> > +++ b/include/linux/apple-mailbox.h
> > @@ -0,0 +1,19 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
> > +/*
> > + * Apple mailbox message format
> > + *
> > + * Copyright (C) 2021 The Asahi Linux Contributors
> > + */
> > +
> > +#ifndef _LINUX_APPLE_MAILBOX_H_
> > +#define _LINUX_APPLE_MAILBOX_H_
>
> Drop the _LINUX
Hmm, this is a straight copy of the Linux header file. Isn't it
better to keep it unmodified? Most (all?) the other files in this
directory return the _LINUX bit...
> > +
> > +#include <linux/types.h>
> > +
> > +/* encodes a single 96bit message sent over the single channel */
> > +struct apple_mbox_msg {
> > + u64 msg0;
> > + u32 msg1;
> > +};
> > +
> > +#endif
> > --
> > 2.34.1
> >
>
> Regards,
> Simon
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 3/8] arm: apple: Add RTKit support
2022-01-22 1:40 ` Simon Glass
@ 2022-01-22 13:59 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 13:59 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 18:40:14 -0700
>
> Hi Mark,
>
> On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> >
> > Most Apple IOPs run a firmware that is based on what Apple calls
> > RTKit. RTKit implements a common mailbox protocol. This code
> > provides an implementation of the AP side of this protocol,
> > providing a function to initialize RTKit-based firmwares as well
> > as a function to do a clean shutdown of this firmware.
> >
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > ---
> > arch/arm/include/asm/arch-apple/rtkit.h | 11 ++
> > arch/arm/mach-apple/Makefile | 1 +
> > arch/arm/mach-apple/rtkit.c | 231 ++++++++++++++++++++++++
> > 3 files changed, 243 insertions(+)
> > create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
> > create mode 100644 arch/arm/mach-apple/rtkit.c
>
> This should be a driver.
This isn't a driver but just a bit of helper code that implements the
RTKit protocol. It will be used by the Apple NVMe driver and a future
SMC driver. The same approach is being taken in Linux. Since it is
intended to be shared, I didn't put it in the NVMe driver itself.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 6/8] power: domain: apple: Add reset support
2022-01-22 1:40 ` Simon Glass
@ 2022-01-22 14:11 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 14:11 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 18:40:23 -0700
>
> Hi Mark,
>
> On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> >
> > The power management controller found on Apple SoCs als provides
> > a way to reset all devices within a power domain. This is needed
> > to cleanly shutdown the NVMe controller before we hand over
> > control to the OS.
> >
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > ---
> > arch/arm/Kconfig | 1 +
> > drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
> > 2 files changed, 73 insertions(+), 1 deletion(-)
>
> This should use devicetree instead of device_bind() and be a reset
> driver in drivers/reset
Not sure what you mean with "this should use devicetree". The reset
and power domain functionality is integrated in the same hardware
register and there is a single node that decsribes the device.
The Linux driver implements the power domain and reset functionality
in a single driver as well. I suppose I could move the reset code
into a file of its own in drivers/reset, but I don't think it would
make the code easier to understand. And I'd still need to bind the
reset driver explicitly in apple_pmgr_probe() as it isn't possible to
automatically bind two drivers to a single device tree node as far as
I can tell.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller
2022-01-22 1:40 ` Simon Glass
@ 2022-01-22 14:45 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 14:45 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 18:40:24 -0700
>
> Hi Mark,
>
> On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> >
> > Add a driver for the NVMe storage controller integrated on
> > Apple SoCs. This NVMe controller isn't PCI based and deviates
> > from the NVMe standard in its implementation of the command
> > submission queue and the integration of an NVMMU that needs
> > to be managed. This commit tweaks the core NVMe code to
> > support the linear command submission queue implemented by
> > this controller. But setting up the submission queue and
> > managing the NVMMU controller is handled by implementing
> > the driver ops that were added in an earlier commit.
> >
> > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > Tested-on: firefly-rk3399
> > Tested-by: Mark Kettenis <kettenis@openbsd.org>
> > ---
> > configs/apple_m1_defconfig | 1 +
> > drivers/nvme/Kconfig | 11 ++
> > drivers/nvme/Makefile | 1 +
> > drivers/nvme/nvme_apple.c | 233 +++++++++++++++++++++++++++++++++++++
> > 4 files changed, 246 insertions(+)
> > create mode 100644 drivers/nvme/nvme_apple.c
>
> Tested on: Macbook Air M1
> Tested-by: Simon Glass <sjg@chromium.org>
>
> >
> > diff --git a/configs/apple_m1_defconfig b/configs/apple_m1_defconfig
> > index cb235e4e7d..1528217b17 100644
> > --- a/configs/apple_m1_defconfig
> > +++ b/configs/apple_m1_defconfig
> > @@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
> > # CONFIG_NET is not set
> > # CONFIG_MMC is not set
> > CONFIG_DEBUG_UART_ANNOUNCE=y
> > +CONFIG_NVME_APPLE=y
> > CONFIG_USB_XHCI_HCD=y
> > CONFIG_USB_XHCI_DWC3=y
> > CONFIG_USB_KEYBOARD=y
> > diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
> > index 78da444c8b..0cb465160b 100644
> > --- a/drivers/nvme/Kconfig
> > +++ b/drivers/nvme/Kconfig
> > @@ -10,6 +10,17 @@ config NVME
> > This option enables support for NVM Express devices.
> > It supports basic functions of NVMe (read/write).
> >
> > +config NVME_APPLE
> > + bool "Apple NVMe controller support"
> > + select NVME
> > + help
> > + This option enables support for the NVMe storage
> > + controller integrated on Apple SoCs. This controller
> > + isn't PCI-based based and deviates from the NVMe
> > + standard implementation in its implementation of
> > + the command submission queue and the integration
> > + of an NVMMU that needs to be managed.
> > +
> > config NVME_PCI
> > bool "NVM Express PCI device support"
> > depends on PCI
> > diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
> > index fad9724e17..fa7b619446 100644
> > --- a/drivers/nvme/Makefile
> > +++ b/drivers/nvme/Makefile
> > @@ -3,4 +3,5 @@
> > # Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
> >
> > obj-y += nvme-uclass.o nvme.o nvme_show.o
> > +obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
> > obj-$(CONFIG_NVME_PCI) += nvme_pci.o
> > diff --git a/drivers/nvme/nvme_apple.c b/drivers/nvme/nvme_apple.c
> > new file mode 100644
> > index 0000000000..b0dc8492f0
> > --- /dev/null
> > +++ b/drivers/nvme/nvme_apple.c
> > @@ -0,0 +1,233 @@
> > +// SPDX-License-Identifier: GPL-2.0+
> > +/*
> > + * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
> > + */
> > +
> > +#include <common.h>
> > +#include <dm.h>
> > +#include <mailbox.h>
> > +#include <mapmem.h>
> > +#include "nvme.h"
> > +#include <reset.h>
> > +
> > +#include <asm/io.h>
> > +#include <asm/arch-apple/rtkit.h>
>
> asm/arch/ should work
It doesn't. For some reason I end up with
arch/arm/include/asm/arch -> arch-m1
But this rtkit stuff is expected to be generic and apply to all Apple
SoCs, not just the M1.
>
> > +#include <linux/iopoll.h>
> > +
> > +#undef readl_poll_timeout
> > +#define readl_poll_timeout readl_poll_sleep_timeout
>
> Do we need this?
No we don't. Fixed.
> > +
> > +#define REG_CPU_CTRL 0x0044
> > +#define REG_CPU_CTRL_RUN BIT(4)
> > +
> > +#define ANS_MAX_PEND_CMDS_CTRL 0x01210
> > +#define ANS_MAX_QUEUE_DEPTH 64
> > +#define ANS_BOOT_STATUS 0x01300
> > +#define ANS_BOOT_STATUS_OK 0xde71ce55
> > +#define ANS_MODESEL 0x01304
> > +#define ANS_UNKNOWN_CTRL 0x24008
> > +#define ANS_PRP_NULL_CHECK (1 << 11)
> > +#define ANS_LINEAR_SQ_CTRL 0x24908
> > +#define ANS_LINEAR_SQ_CTRL_EN (1 << 0)
> > +#define ANS_ASQ_DB 0x2490c
> > +#define ANS_IOSQ_DB 0x24910
> > +#define ANS_NVMMU_NUM 0x28100
> > +#define ANS_NVMMU_BASE_ASQ 0x28108
> > +#define ANS_NVMMU_BASE_IOSQ 0x28110
> > +#define ANS_NVMMU_TCB_INVAL 0x28118
> > +#define ANS_NVMMU_TCB_STAT 0x28120
> > +
> > +#define ANS_NVMMU_TCB_SIZE 0x4000
> > +#define ANS_NVMMU_TCB_PITCH 0x80
> > +
> > +struct ans_nvmmu_tcb {
> > + u8 opcode;
> > + u8 flags;
> > + u8 command_id;
> > + u8 pad0;
> > + u32 prpl_len;
> > + u8 pad1[16];
> > + u64 prp1;
> > + u64 prp2;
>
> Needs comments
I put a comment at the top explaining my limited understanding of what
the TCBs are used for.
>
> > +};
> > +
> > +#define ANS_NVMMU_TCB_WRITE BIT(0)
> > +#define ANS_NVMMU_TCB_READ BIT(1)
> > +
> > +struct apple_nvme_priv {
> > + struct nvme_dev ndev;
> > + void *base;
> > + void *asc;
> > + struct reset_ctl_bulk resets;
> > + struct mbox_chan chan;
> > + struct ans_nvmmu_tcb *tcbs[NVME_Q_NUM];
> > + u32 __iomem *q_db[NVME_Q_NUM];
>
> Needs comments
Done, although they give me a captain obvious vibe...
> > +};
> > +
> > +static int apple_nvme_alloc_queue(struct nvme_queue *nvmeq)
> > +{
> > + struct apple_nvme_priv *priv =
> > + container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
> > + struct nvme_dev *dev = nvmeq->dev;
> > +
> > + switch (nvmeq->qid) {
> > + case NVME_ADMIN_Q:
> > + case NVME_IO_Q:
> > + break;
> > + default:
> > + return -EINVAL;
> > + }
> > +
> > + priv->tcbs[nvmeq->qid] = (void *)memalign(4096, ANS_NVMMU_TCB_SIZE);
> > + memset((void *)priv->tcbs[nvmeq->qid], 0, ANS_NVMMU_TCB_SIZE);
> > +
> > + switch (nvmeq->qid) {
> > + case NVME_ADMIN_Q:
> > + priv->q_db[nvmeq->qid] =
> > + ((void __iomem *)dev->bar) + ANS_ASQ_DB;
> > + nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
> > + ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_ASQ);
> > + break;
> > + case NVME_IO_Q:
> > + priv->q_db[nvmeq->qid] =
> > + ((void __iomem *)dev->bar) + ANS_IOSQ_DB;
> > + nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
> > + ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_IOSQ);
> > + break;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static void apple_nvme_submit_cmd(struct nvme_queue *nvmeq,
> > + struct nvme_command *cmd)
> > +{
> > + struct apple_nvme_priv *priv =
> > + container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
> > + struct ans_nvmmu_tcb *tcb;
> > + u16 tail = nvmeq->sq_tail;
> > +
> > + tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
> > + memset(tcb, 0, sizeof(*tcb));
> > + tcb->opcode = cmd->common.opcode;
> > + tcb->flags = ANS_NVMMU_TCB_WRITE | ANS_NVMMU_TCB_READ;
> > + tcb->command_id = tail;
> > + tcb->prpl_len = cmd->rw.length;
> > + tcb->prp1 = cmd->common.prp1;
> > + tcb->prp2 = cmd->common.prp2;
> > +
> > + writel(tail, priv->q_db[nvmeq->qid]);
> > +}
> > +
> > +static void apple_nvme_complete_cmd(struct nvme_queue *nvmeq,
> > + struct nvme_command *cmd)
> > +{
> > + struct apple_nvme_priv *priv =
> > + container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
> > + struct ans_nvmmu_tcb *tcb;
> > + u16 tail = nvmeq->sq_tail;
> > +
> > + tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
> > + memset(tcb, 0, sizeof(*tcb));
> > + writel(tail, ((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_INVAL);
> > + readl(((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_STAT);
> > +
> > + if (++tail == nvmeq->q_depth)
> > + tail = 0;
> > + nvmeq->sq_tail = tail;
> > +}
> > +
> > +static int apple_nvme_probe(struct udevice *dev)
> > +{
> > + struct apple_nvme_priv *priv = dev_get_priv(dev);
> > + fdt_addr_t addr;
> > + u32 ctrl, stat;
> > + int ret;
> > +
> > + priv->base = dev_read_addr_ptr(dev);
> > + if (!priv->base)
> > + return -EINVAL;
> > +
> > + addr = dev_read_addr_index(dev, 1);
> > + if (addr == FDT_ADDR_T_NONE)
> > + return -EINVAL;
> > + priv->asc = map_sysmem(addr, 0);
> > +
> > + ret = reset_get_bulk(dev, &priv->resets);
> > + if (ret < 0)
> > + return ret;
> > +
> > + ret = mbox_get_by_index(dev, 0, &priv->chan);
>
> Is the mailbox not mentioned in the device tre?
There is an "mboxes" property, which is what mbox_get_by_index() uses.
But there is only one mbox so it isn't named.
>
> > + if (ret < 0)
> > + return ret;
> > +
> > + ctrl = readl(priv->asc + REG_CPU_CTRL);
> > + writel(ctrl | REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
> > +
> > + ret = apple_rtkit_init(&priv->chan);
>
> Should probe a driver here
See my reply to PATCH 3/8 in this series.
> > + if (ret < 0)
> > + return ret;
> > +
> > + ret = readl_poll_timeout(priv->base + ANS_BOOT_STATUS, stat,
> > + (stat == ANS_BOOT_STATUS_OK), 100, 500000);
> > + if (ret < 0) {
> > + printf("%s: NVMe firmware didn't boot\n", __func__);
> > + return -ETIMEDOUT;
> > + }
> > +
> > + writel(ANS_LINEAR_SQ_CTRL_EN, priv->base + ANS_LINEAR_SQ_CTRL);
> > + writel(((ANS_MAX_QUEUE_DEPTH << 16) | ANS_MAX_QUEUE_DEPTH),
> > + priv->base + ANS_MAX_PEND_CMDS_CTRL);
> > +
> > + writel(readl(priv->base + ANS_UNKNOWN_CTRL) & ~ANS_PRP_NULL_CHECK,
> > + priv->base + ANS_UNKNOWN_CTRL);
> > +
> > + strcpy(priv->ndev.vendor, "Apple");
> > +
> > + writel((ANS_NVMMU_TCB_SIZE / ANS_NVMMU_TCB_PITCH) - 1,
> > + priv->base + ANS_NVMMU_NUM);
> > + writel(0, priv->base + ANS_MODESEL);
> > +
> > + priv->ndev.bar = priv->base;
> > + return nvme_init(dev);
> > +}
> > +
> > +static int apple_nvme_remove(struct udevice *dev)
> > +{
> > + struct apple_nvme_priv *priv = dev_get_priv(dev);
> > + u32 ctrl;
> > +
> > + nvme_shutdown(dev);
> > +
> > + apple_rtkit_shutdown(&priv->chan, APPLE_RTKIT_PWR_STATE_SLEEP);
> > +
> > + ctrl = readl(priv->asc + REG_CPU_CTRL);
> > + writel(ctrl & ~REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
> > +
> > + reset_assert_bulk(&priv->resets);
> > + reset_deassert_bulk(&priv->resets);
> > +
> > + return 0;
> > +}
> > +
> > +static const struct nvme_ops apple_nvme_ops = {
> > + .alloc_queue = apple_nvme_alloc_queue,
> > + .submit_cmd = apple_nvme_submit_cmd,
> > + .complete_cmd = apple_nvme_complete_cmd,
> > +};
> > +
> > +static const struct udevice_id apple_nvme_ids[] = {
> > + { .compatible = "apple,nvme-ans2" },
> > + { /* sentinel */ }
> > +};
> > +
> > +U_BOOT_DRIVER(apple_nvme) = {
> > + .name = "apple_nvme",
> > + .id = UCLASS_NVME,
> > + .of_match = apple_nvme_ids,
> > + .priv_auto = sizeof(struct apple_nvme_priv),
> > + .probe = apple_nvme_probe,
> > + .remove = apple_nvme_remove,
> > + .ops = &apple_nvme_ops,
> > + .flags = DM_FLAG_OS_PREPARE,
> > +};
> > --
> > 2.34.1
> >
>
> Regards,
> Simon
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 1/8] nvme: Split out PCI support
2022-01-22 13:18 ` Simon Glass
@ 2022-01-22 14:57 ` Mark Kettenis
0 siblings, 0 replies; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 14:57 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Sat, 22 Jan 2022 06:18:18 -0700
>
> Hi Mark,
>
> On Sat, 22 Jan 2022 at 05:48, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >
> > > From: Simon Glass <sjg@chromium.org>
> > > Date: Fri, 21 Jan 2022 18:40:11 -0700
> > >
> > > Hi Mark,
> > >
> > > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > > >
> > > > Apple SoCs have an integrated NVMe controller that isn't connected
> > > > over a PCIe bus. In preparation for adding support for this NVMe
> > > > controller, split out the PCI support into its own file. This file
> > > > is selected through a new CONFIG_NVME_PCI Kconfig option, so do
> > > > a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.
> > > >
> > > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > > ---
> > > > configs/clearfog_gt_8k_defconfig | 2 +-
> > > > configs/firefly-rk3399_defconfig | 2 +-
> > > > configs/khadas-vim3_android_ab_defconfig | 2 +-
> > > > configs/khadas-vim3_android_defconfig | 2 +-
> > > > configs/khadas-vim3_defconfig | 2 +-
> > > > configs/khadas-vim3l_android_ab_defconfig | 2 +-
> > > > configs/khadas-vim3l_android_defconfig | 2 +-
> > > > configs/khadas-vim3l_defconfig | 2 +-
> > > > configs/kontron_sl28_defconfig | 2 +-
> > > > configs/ls1012afrdm_qspi_defconfig | 2 +-
> > > > configs/ls1012afrdm_tfa_defconfig | 2 +-
> > > > .../ls1012afrwy_qspi_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1012afrwy_qspi_defconfig | 2 +-
> > > > configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1012afrwy_tfa_defconfig | 2 +-
> > > > configs/ls1012aqds_qspi_defconfig | 2 +-
> > > > configs/ls1012aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1012aqds_tfa_defconfig | 2 +-
> > > > configs/ls1012ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1012ardb_qspi_defconfig | 2 +-
> > > > configs/ls1012ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1012ardb_tfa_defconfig | 2 +-
> > > > configs/ls1021aiot_qspi_defconfig | 2 +-
> > > > configs/ls1021aiot_sdcard_defconfig | 2 +-
> > > > configs/ls1021aqds_ddr4_nor_defconfig | 2 +-
> > > > configs/ls1021aqds_ddr4_nor_lpuart_defconfig | 2 +-
> > > > configs/ls1021aqds_nand_defconfig | 2 +-
> > > > configs/ls1021aqds_nor_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1021aqds_nor_defconfig | 2 +-
> > > > configs/ls1021aqds_nor_lpuart_defconfig | 2 +-
> > > > configs/ls1021aqds_qspi_defconfig | 2 +-
> > > > configs/ls1021aqds_sdcard_ifc_defconfig | 2 +-
> > > > configs/ls1021aqds_sdcard_qspi_defconfig | 2 +-
> > > > configs/ls1021atsn_qspi_defconfig | 2 +-
> > > > configs/ls1021atsn_sdcard_defconfig | 2 +-
> > > > configs/ls1021atwr_nor_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1021atwr_nor_defconfig | 2 +-
> > > > configs/ls1021atwr_nor_lpuart_defconfig | 2 +-
> > > > configs/ls1021atwr_qspi_defconfig | 2 +-
> > > > configs/ls1021atwr_sdcard_ifc_defconfig | 2 +-
> > > > configs/ls1021atwr_sdcard_qspi_defconfig | 2 +-
> > > > configs/ls1028aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1028aqds_tfa_defconfig | 2 +-
> > > > configs/ls1028aqds_tfa_lpuart_defconfig | 2 +-
> > > > configs/ls1028ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1028ardb_tfa_defconfig | 2 +-
> > > > configs/ls1043aqds_defconfig | 2 +-
> > > > configs/ls1043aqds_lpuart_defconfig | 2 +-
> > > > configs/ls1043aqds_nand_defconfig | 2 +-
> > > > configs/ls1043aqds_nor_ddr3_defconfig | 2 +-
> > > > configs/ls1043aqds_qspi_defconfig | 2 +-
> > > > configs/ls1043aqds_sdcard_ifc_defconfig | 2 +-
> > > > configs/ls1043aqds_sdcard_qspi_defconfig | 2 +-
> > > > configs/ls1043aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1043aqds_tfa_defconfig | 2 +-
> > > > configs/ls1043ardb_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1043ardb_defconfig | 2 +-
> > > > configs/ls1043ardb_nand_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1043ardb_nand_defconfig | 2 +-
> > > > configs/ls1043ardb_sdcard_defconfig | 2 +-
> > > > configs/ls1043ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1043ardb_tfa_defconfig | 2 +-
> > > > configs/ls1046afrwy_tfa_defconfig | 2 +-
> > > > configs/ls1046aqds_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1046aqds_defconfig | 2 +-
> > > > configs/ls1046aqds_lpuart_defconfig | 2 +-
> > > > configs/ls1046aqds_nand_defconfig | 2 +-
> > > > configs/ls1046aqds_qspi_defconfig | 2 +-
> > > > configs/ls1046aqds_sdcard_ifc_defconfig | 2 +-
> > > > configs/ls1046aqds_sdcard_qspi_defconfig | 2 +-
> > > > configs/ls1046aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1046aqds_tfa_defconfig | 2 +-
> > > > configs/ls1046ardb_emmc_defconfig | 2 +-
> > > > configs/ls1046ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1046ardb_qspi_defconfig | 2 +-
> > > > configs/ls1046ardb_qspi_spl_defconfig | 2 +-
> > > > configs/ls1046ardb_sdcard_defconfig | 2 +-
> > > > configs/ls1046ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1046ardb_tfa_defconfig | 2 +-
> > > > configs/ls1088aqds_defconfig | 2 +-
> > > > configs/ls1088aqds_qspi_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1088aqds_qspi_defconfig | 2 +-
> > > > configs/ls1088aqds_sdcard_ifc_defconfig | 2 +-
> > > > configs/ls1088aqds_sdcard_qspi_defconfig | 2 +-
> > > > configs/ls1088aqds_tfa_defconfig | 2 +-
> > > > configs/ls1088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1088ardb_qspi_defconfig | 2 +-
> > > > configs/ls1088ardb_sdcard_qspi_defconfig | 2 +-
> > > > configs/ls1088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls1088ardb_tfa_defconfig | 2 +-
> > > > configs/ls2080aqds_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls2080aqds_defconfig | 2 +-
> > > > configs/ls2080aqds_nand_defconfig | 2 +-
> > > > configs/ls2080aqds_qspi_defconfig | 2 +-
> > > > configs/ls2080aqds_sdcard_defconfig | 2 +-
> > > > configs/ls2080ardb_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls2080ardb_defconfig | 2 +-
> > > > configs/ls2080ardb_nand_defconfig | 2 +-
> > > > configs/ls2081ardb_defconfig | 2 +-
> > > > configs/ls2088aqds_tfa_defconfig | 2 +-
> > > > configs/ls2088ardb_qspi_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls2088ardb_qspi_defconfig | 2 +-
> > > > configs/ls2088ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/ls2088ardb_tfa_defconfig | 2 +-
> > > > configs/lx2160aqds_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/lx2160aqds_tfa_defconfig | 2 +-
> > > > configs/lx2160ardb_tfa_SECURE_BOOT_defconfig | 2 +-
> > > > configs/lx2160ardb_tfa_defconfig | 2 +-
> > > > configs/lx2160ardb_tfa_stmm_defconfig | 2 +-
> > > > configs/mvebu_crb_cn9130_defconfig | 2 +-
> > > > configs/mvebu_db_armada8k_defconfig | 2 +-
> > > > configs/mvebu_db_cn9130_defconfig | 2 +-
> > > > configs/mvebu_espressobin-88f3720_defconfig | 2 +-
> > > > configs/mvebu_mcbin-88f8040_defconfig | 2 +-
> > > > configs/mvebu_puzzle-m801-88f8040_defconfig | 2 +-
> > > > configs/nanopc-t4-rk3399_defconfig | 2 +-
> > > > configs/octeontx2_96xx_defconfig | 2 +-
> > > > configs/octeontx_81xx_defconfig | 2 +-
> > > > configs/octeontx_83xx_defconfig | 2 +-
> > > > configs/p3450-0000_defconfig | 2 +-
> > > > configs/pinebook-pro-rk3399_defconfig | 2 +-
> > > > configs/qemu-x86_64_defconfig | 2 +-
> > > > configs/qemu-x86_defconfig | 2 +-
> > > > configs/qemu_arm64_defconfig | 2 +-
> > > > configs/qemu_arm_defconfig | 2 +-
> > > > configs/rcar3_salvator-x_defconfig | 2 +-
> > > > configs/roc-pc-mezzanine-rk3399_defconfig | 2 +-
> > > > configs/rock-pi-4-rk3399_defconfig | 2 +-
> > > > configs/rock-pi-4c-rk3399_defconfig | 2 +-
> > > > configs/rock-pi-n10-rk3399pro_defconfig | 2 +-
> > > > configs/rock960-rk3399_defconfig | 2 +-
> > > > configs/rockpro64-rk3399_defconfig | 2 +-
> > > > configs/sandbox64_defconfig | 2 +-
> > > > configs/sandbox_defconfig | 2 +-
> > > > configs/sandbox_flattree_defconfig | 2 +-
> > > > configs/sandbox_noinst_defconfig | 2 +-
> > > > configs/sandbox_spl_defconfig | 2 +-
> > > > configs/sifive_unmatched_defconfig | 2 +-
> > > > configs/synquacer_developerbox_defconfig | 2 +-
> > > > configs/turris_mox_defconfig | 2 +-
> > > > configs/turris_omnia_defconfig | 2 +-
> > > > doc/develop/driver-model/nvme.rst | 1 +
> > > > drivers/nvme/Kconfig | 10 +++-
> > > > drivers/nvme/Makefile | 1 +
> > > > drivers/nvme/nvme.c | 38 ++------------
> > > > drivers/nvme/nvme.h | 3 ++
> > > > drivers/nvme/nvme_pci.c | 49 +++++++++++++++++++
> > > > 147 files changed, 207 insertions(+), 177 deletions(-)
> > > > create mode 100644 drivers/nvme/nvme_pci.c
> > > >
> > >
> > > Shouldn't the vendor addition go in another patch?
> >
> > I don't think that would make a ton of sense. It is intricately
> > linked to splitting out the PCI support, as the current code grovels
> > into pci_child_plat struct in a place that needs to be bus-agnostic.
>
> OK.
>
> BTW I noticed that the U-Boot logo has the wrong colours on the
> Macbook Air, with these patches applied to mainline.
Yes. The code that turns on the logo was merged and I suspect that
the relevant code doesn't do the proper pixel format conversions.
I'll see if I can figure this out.
> > > Otherwise:
> > > Reviewed-by: Simon Glass <sjg@chromium.org>
> > > Tested on: Macbook Air M1
> > > Tested-by: Simon Glass <sjg@chromium.org>
>
> Regards,
> Simon
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox
2022-01-22 13:54 ` Mark Kettenis
@ 2022-01-22 17:17 ` Simon Glass
0 siblings, 0 replies; 33+ messages in thread
From: Simon Glass @ 2022-01-22 17:17 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 06:54, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 18:40:12 -0700
> >
> > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > >
> > > This mailbox driver provides a communication channel with the
> > > Apple IOP controllers found on Apple SoCs. These IOP controllers
> > > are used to implement various functions such as the System
> > > Manegement Controller (SMC) and NVMe storage. It allows sending
> > > and receiving a 96-bit message over a single channel.
> > >
> > > The header file with the struct used for mailbox messages is taken
> > > straight from Linux.
> > >
> > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > Signed-off-by: Sven Peter <sven@svenpeter.dev>
> > > ---
> > > arch/arm/Kconfig | 1 +
> > > drivers/mailbox/Kconfig | 11 +++++
> > > drivers/mailbox/Makefile | 1 +
> > > drivers/mailbox/apple-mbox.c | 92 +++++++++++++++++++++++++++++++++++
> > > include/linux/apple-mailbox.h | 19 ++++++++
> > > 5 files changed, 124 insertions(+)
> > > create mode 100644 drivers/mailbox/apple-mbox.c
> > > create mode 100644 include/linux/apple-mailbox.h
> >
> > Reviewed-by: Simon Glass <sjg@chromium.org>
> > Tested on: Macbook Air M1
> > Tested-by: Simon Glass <sjg@chromium.org>
> >
> > with nit below
> >
> > [..]
> >
> > > diff --git a/include/linux/apple-mailbox.h b/include/linux/apple-mailbox.h
> > > new file mode 100644
> > > index 0000000000..720fbb7029
> > > --- /dev/null
> > > +++ b/include/linux/apple-mailbox.h
> > > @@ -0,0 +1,19 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
> > > +/*
> > > + * Apple mailbox message format
> > > + *
> > > + * Copyright (C) 2021 The Asahi Linux Contributors
> > > + */
> > > +
> > > +#ifndef _LINUX_APPLE_MAILBOX_H_
> > > +#define _LINUX_APPLE_MAILBOX_H_
> >
> > Drop the _LINUX
>
> Hmm, this is a straight copy of the Linux header file. Isn't it
> better to keep it unmodified? Most (all?) the other files in this
> directory return the _LINUX bit...
Oh I missed that this was supposed to be in this dir. Yes seems OK.
>
> > > +
> > > +#include <linux/types.h>
> > > +
> > > +/* encodes a single 96bit message sent over the single channel */
> > > +struct apple_mbox_msg {
> > > + u64 msg0;
> > > + u32 msg1;
> > > +};
> > > +
> > > +#endif
> > > --
> > > 2.34.1
> > >
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller
2022-01-22 14:45 ` Mark Kettenis
@ 2022-01-22 17:17 ` Simon Glass
2022-01-22 17:41 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 17:17 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 07:45, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 18:40:24 -0700
> >
> > Hi Mark,
> >
> > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > >
> > > Add a driver for the NVMe storage controller integrated on
> > > Apple SoCs. This NVMe controller isn't PCI based and deviates
> > > from the NVMe standard in its implementation of the command
> > > submission queue and the integration of an NVMMU that needs
> > > to be managed. This commit tweaks the core NVMe code to
> > > support the linear command submission queue implemented by
> > > this controller. But setting up the submission queue and
> > > managing the NVMMU controller is handled by implementing
> > > the driver ops that were added in an earlier commit.
> > >
> > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > Tested-on: firefly-rk3399
> > > Tested-by: Mark Kettenis <kettenis@openbsd.org>
> > > ---
> > > configs/apple_m1_defconfig | 1 +
> > > drivers/nvme/Kconfig | 11 ++
> > > drivers/nvme/Makefile | 1 +
> > > drivers/nvme/nvme_apple.c | 233 +++++++++++++++++++++++++++++++++++++
> > > 4 files changed, 246 insertions(+)
> > > create mode 100644 drivers/nvme/nvme_apple.c
> >
> > Tested on: Macbook Air M1
> > Tested-by: Simon Glass <sjg@chromium.org>
> >
> > >
> > > diff --git a/configs/apple_m1_defconfig b/configs/apple_m1_defconfig
> > > index cb235e4e7d..1528217b17 100644
> > > --- a/configs/apple_m1_defconfig
> > > +++ b/configs/apple_m1_defconfig
> > > @@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
> > > # CONFIG_NET is not set
> > > # CONFIG_MMC is not set
> > > CONFIG_DEBUG_UART_ANNOUNCE=y
> > > +CONFIG_NVME_APPLE=y
> > > CONFIG_USB_XHCI_HCD=y
> > > CONFIG_USB_XHCI_DWC3=y
> > > CONFIG_USB_KEYBOARD=y
> > > diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
> > > index 78da444c8b..0cb465160b 100644
> > > --- a/drivers/nvme/Kconfig
> > > +++ b/drivers/nvme/Kconfig
> > > @@ -10,6 +10,17 @@ config NVME
> > > This option enables support for NVM Express devices.
> > > It supports basic functions of NVMe (read/write).
> > >
> > > +config NVME_APPLE
> > > + bool "Apple NVMe controller support"
> > > + select NVME
> > > + help
> > > + This option enables support for the NVMe storage
> > > + controller integrated on Apple SoCs. This controller
> > > + isn't PCI-based based and deviates from the NVMe
> > > + standard implementation in its implementation of
> > > + the command submission queue and the integration
> > > + of an NVMMU that needs to be managed.
> > > +
> > > config NVME_PCI
> > > bool "NVM Express PCI device support"
> > > depends on PCI
> > > diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
> > > index fad9724e17..fa7b619446 100644
> > > --- a/drivers/nvme/Makefile
> > > +++ b/drivers/nvme/Makefile
> > > @@ -3,4 +3,5 @@
> > > # Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
> > >
> > > obj-y += nvme-uclass.o nvme.o nvme_show.o
> > > +obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
> > > obj-$(CONFIG_NVME_PCI) += nvme_pci.o
> > > diff --git a/drivers/nvme/nvme_apple.c b/drivers/nvme/nvme_apple.c
> > > new file mode 100644
> > > index 0000000000..b0dc8492f0
> > > --- /dev/null
> > > +++ b/drivers/nvme/nvme_apple.c
> > > @@ -0,0 +1,233 @@
> > > +// SPDX-License-Identifier: GPL-2.0+
> > > +/*
> > > + * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
> > > + */
> > > +
> > > +#include <common.h>
> > > +#include <dm.h>
> > > +#include <mailbox.h>
> > > +#include <mapmem.h>
> > > +#include "nvme.h"
> > > +#include <reset.h>
> > > +
> > > +#include <asm/io.h>
> > > +#include <asm/arch-apple/rtkit.h>
> >
> > asm/arch/ should work
>
> It doesn't. For some reason I end up with
>
> arch/arm/include/asm/arch -> arch-m1
>
> But this rtkit stuff is expected to be generic and apply to all Apple
> SoCs, not just the M1.
Hmm so you plan to have an arch/m2 as well? Can we not put everything
in arch/apple then? We don't normally have arch directories for
different generations unless they are completely different.
[..]
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 6/8] power: domain: apple: Add reset support
2022-01-22 14:11 ` Mark Kettenis
@ 2022-01-22 17:17 ` Simon Glass
2022-01-22 19:35 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 17:17 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 07:12, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 18:40:23 -0700
> >
> > Hi Mark,
> >
> > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > >
> > > The power management controller found on Apple SoCs als provides
> > > a way to reset all devices within a power domain. This is needed
> > > to cleanly shutdown the NVMe controller before we hand over
> > > control to the OS.
> > >
> > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > ---
> > > arch/arm/Kconfig | 1 +
> > > drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
> > > 2 files changed, 73 insertions(+), 1 deletion(-)
> >
> > This should use devicetree instead of device_bind() and be a reset
> > driver in drivers/reset
>
> Not sure what you mean with "this should use devicetree". The reset
> and power domain functionality is integrated in the same hardware
> register and there is a single node that decsribes the device.
>
> The Linux driver implements the power domain and reset functionality
> in a single driver as well. I suppose I could move the reset code
> into a file of its own in drivers/reset, but I don't think it would
> make the code easier to understand. And I'd still need to bind the
> reset driver explicitly in apple_pmgr_probe() as it isn't possible to
> automatically bind two drivers to a single device tree node as far as
> I can tell.
It seems odd that Linux does this sort of thing. The normal U-Boot
approach would be to create a parent MFD driver with children for each
uclass. Does linux do that these days?
Driver model cannot create two drivers from one device tree node,
although you can create one manually later as you have done.
It seems OK to follow along with Linux, if it makes it easier to
maintain. Otherwise, drivers/reset would be better for the reset
driver.
Either way:
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 4/8] nvme: Introduce driver ops
2022-01-22 13:33 ` Mark Kettenis
@ 2022-01-22 17:17 ` Simon Glass
0 siblings, 0 replies; 33+ messages in thread
From: Simon Glass @ 2022-01-22 17:17 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 06:33, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 18:40:19 -0700
> >
> > Hi Mark,
> >
> > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > >
> > > The NVMe storage controller integrated on Apple SoCs deviates
> > > from the NVMe standard in two aspects. It uses a "linear"
> > > submission queue and it integrates an NVMMU that needs to be
> > > programmed for each NVMe command. Introduce driver ops such
> > > that we can set up the linear submission queue and program the
> > > NVMMU in the driver for this strange beast.
> > >
> > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > ---
> > > drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
> > > drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
> > > 2 files changed, 51 insertions(+), 27 deletions(-)
> >
> > Reviewed-by: Simon Glass <sjg@chromium.org>
> > Tested on: Macbook Air M1
> > Tested-by: Simon Glass <sjg@chromium.org>
>
> Thanks...
>
> > But please see below
> >
> > >
> > > diff --git a/drivers/nvme/nvme.c b/drivers/nvme/nvme.c
> > > index be518ec20b..e2d0f9c668 100644
> > > --- a/drivers/nvme/nvme.c
> > > +++ b/drivers/nvme/nvme.c
> > > @@ -27,33 +27,6 @@
> > > #define IO_TIMEOUT 30
> > > #define MAX_PRP_POOL 512
> > >
> > > -enum nvme_queue_id {
> > > - NVME_ADMIN_Q,
> > > - NVME_IO_Q,
> > > - NVME_Q_NUM,
> > > -};
> > > -
> > > -/*
> > > - * An NVM Express queue. Each device has at least two (one for admin
> > > - * commands and one for I/O commands).
> > > - */
> > > -struct nvme_queue {
> > > - struct nvme_dev *dev;
> > > - struct nvme_command *sq_cmds;
> > > - struct nvme_completion *cqes;
> > > - wait_queue_head_t sq_full;
> > > - u32 __iomem *q_db;
> > > - u16 q_depth;
> > > - s16 cq_vector;
> > > - u16 sq_head;
> > > - u16 sq_tail;
> > > - u16 cq_head;
> > > - u16 qid;
> > > - u8 cq_phase;
> > > - u8 cqe_seen;
> > > - unsigned long cmdid_data[];
> > > -};
> > > -
> > > static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
> > > {
> > > u32 bit = enabled ? NVME_CSTS_RDY : 0;
> > > @@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
> > > */
> > > static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
> > > {
> > > + struct nvme_ops *ops;
> > > u16 tail = nvmeq->sq_tail;
> > >
> > > memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
> > > flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
> > > (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
> > >
> > > + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
> > > + if (ops && ops->submit_cmd) {
> > > + ops->submit_cmd(nvmeq, cmd);
> > > + return;
> > > + }
> > > +
> > > if (++tail == nvmeq->q_depth)
> > > tail = 0;
> > > writel(tail, nvmeq->q_db);
> > > @@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
> > > struct nvme_command *cmd,
> > > u32 *result, unsigned timeout)
> > > {
> > > + struct nvme_ops *ops;
> > > u16 head = nvmeq->cq_head;
> > > u16 phase = nvmeq->cq_phase;
> > > u16 status;
> > > @@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
> > > return -ETIMEDOUT;
> > > }
> > >
> > > + ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
> > > + if (ops && ops->complete_cmd)
> > > + ops->complete_cmd(nvmeq, cmd);
> > > +
> > > status >>= 1;
> > > if (status) {
> > > printf("ERROR: status = %x, phase = %d, head = %d\n",
> > > @@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
> > > static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
> > > int qid, int depth)
> > > {
> > > + struct nvme_ops *ops;
> > > struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
> > > if (!nvmeq)
> > > return NULL;
> > > @@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
> > > dev->queue_count++;
> > > dev->queues[qid] = nvmeq;
> > >
> > > + ops = (struct nvme_ops *)dev->udev->driver->ops;
> > > + if (ops && ops->alloc_queue)
> > > + ops->alloc_queue(nvmeq);
> > > +
> > > return nvmeq;
> > >
> > > free_queue:
> > > @@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
> > > struct nvme_id_ns *id;
> > > int ret;
> > >
> > > + ndev->udev = udev;
> > > INIT_LIST_HEAD(&ndev->namespaces);
> > > if (readl(&ndev->bar->csts) == -1) {
> > > ret = -ENODEV;
> > > diff --git a/drivers/nvme/nvme.h b/drivers/nvme/nvme.h
> > > index 8e9ae3c7f6..57803b43fd 100644
> > > --- a/drivers/nvme/nvme.h
> > > +++ b/drivers/nvme/nvme.h
> > > @@ -596,6 +596,7 @@ enum {
> > >
> > > /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
> > > struct nvme_dev {
> > > + struct udevice *udev;
> > > struct list_head node;
> > > struct nvme_queue **queues;
> > > u32 __iomem *dbs;
> > > @@ -622,6 +623,32 @@ struct nvme_dev {
> > > u32 nn;
> > > };
> > >
> > > +enum nvme_queue_id {
> >
> > comment
>
> Hmm, I merely moved this definition. Feels a bit awkward to come up
> with a comment when I didn't write the bit of code in question. I can
> come up with something though if you insist.
Oh. Well, if you can...
>
> >
> > > + NVME_ADMIN_Q,
> > > + NVME_IO_Q,
> > > + NVME_Q_NUM,
> > > +};
> > > +
> > > +/*
> > > + * An NVM Express queue. Each device has at least two (one for admin
> > > + * commands and one for I/O commands).
> > > + */
> > > +struct nvme_queue {
> > > + struct nvme_dev *dev;
> > > + struct nvme_command *sq_cmds;
> > > + struct nvme_completion *cqes;
> > > + u32 __iomem *q_db;
> > > + u16 q_depth;
> > > + s16 cq_vector;
> > > + u16 sq_head;
> > > + u16 sq_tail;
> > > + u16 cq_head;
> > > + u16 qid;
> > > + u8 cq_phase;
> > > + u8 cqe_seen;
> > > + unsigned long cmdid_data[];
> > > +};
> > > +
> > > /*
> > > * An NVM Express namespace is equivalent to a SCSI LUN.
> > > * Each namespace is operated as an independent "device".
> > > @@ -636,6 +663,12 @@ struct nvme_ns {
> > > u8 flbas;
> > > };
> > >
> > > +struct nvme_ops {
> > > + int (*alloc_queue)(struct nvme_queue *);
> > > + void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
> > > + void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);
> >
> > each of these needs a comment
>
> Indeed. While writing the comment, I realized that setup_queue would
> be a better name than alloc_queue, so I changed that. Is that a
> small-enough change to retain your tags?
Yes
>
> >
> > > +};
> > > +
> > > int nvme_init(struct udevice *udev);
> >
> > (for later, this should be dev, not udev)
>
> As in, this is fine for now but will need to be changed at some point
> in the future?
Yes, it's just a style thing.
>
> >
> > >
> > > #endif /* __DRIVER_NVME_H__ */
> > > --
> > > 2.34.1
> > >
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 3/8] arm: apple: Add RTKit support
2022-01-22 13:59 ` Mark Kettenis
@ 2022-01-22 17:17 ` Simon Glass
2022-01-22 19:31 ` Mark Kettenis
0 siblings, 1 reply; 33+ messages in thread
From: Simon Glass @ 2022-01-22 17:17 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 06:59, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 18:40:14 -0700
> >
> > Hi Mark,
> >
> > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > >
> > > Most Apple IOPs run a firmware that is based on what Apple calls
> > > RTKit. RTKit implements a common mailbox protocol. This code
> > > provides an implementation of the AP side of this protocol,
> > > providing a function to initialize RTKit-based firmwares as well
> > > as a function to do a clean shutdown of this firmware.
> > >
> > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > ---
> > > arch/arm/include/asm/arch-apple/rtkit.h | 11 ++
> > > arch/arm/mach-apple/Makefile | 1 +
> > > arch/arm/mach-apple/rtkit.c | 231 ++++++++++++++++++++++++
> > > 3 files changed, 243 insertions(+)
> > > create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
> > > create mode 100644 arch/arm/mach-apple/rtkit.c
> >
> > This should be a driver.
>
> This isn't a driver but just a bit of helper code that implements the
> RTKit protocol. It will be used by the Apple NVMe driver and a future
> SMC driver. The same approach is being taken in Linux. Since it is
> intended to be shared, I didn't put it in the NVMe driver itself.
It just seems to set up and close down the peripheral, so this could
be a driver. What happens when someone needs to change it for a new
generation? Normally we would use the compatible string for that, but
here you have code that might get duplicated or parameterised in
another way.
Anyway if you really think this is the right way, it is OK. We do it
elsewhere, iwc
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller
2022-01-22 17:17 ` Simon Glass
@ 2022-01-22 17:41 ` Mark Kettenis
2022-01-22 18:28 ` Simon Glass
0 siblings, 1 reply; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 17:41 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Sat, 22 Jan 2022 10:17:08 -0700
>
> Hi Mark,
>
> On Sat, 22 Jan 2022 at 07:45, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >
> > > From: Simon Glass <sjg@chromium.org>
> > > Date: Fri, 21 Jan 2022 18:40:24 -0700
> > >
> > > Hi Mark,
> > >
> > > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > > >
> > > > Add a driver for the NVMe storage controller integrated on
> > > > Apple SoCs. This NVMe controller isn't PCI based and deviates
> > > > from the NVMe standard in its implementation of the command
> > > > submission queue and the integration of an NVMMU that needs
> > > > to be managed. This commit tweaks the core NVMe code to
> > > > support the linear command submission queue implemented by
> > > > this controller. But setting up the submission queue and
> > > > managing the NVMMU controller is handled by implementing
> > > > the driver ops that were added in an earlier commit.
> > > >
> > > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > > Tested-on: firefly-rk3399
> > > > Tested-by: Mark Kettenis <kettenis@openbsd.org>
> > > > ---
> > > > configs/apple_m1_defconfig | 1 +
> > > > drivers/nvme/Kconfig | 11 ++
> > > > drivers/nvme/Makefile | 1 +
> > > > drivers/nvme/nvme_apple.c | 233 +++++++++++++++++++++++++++++++++++++
> > > > 4 files changed, 246 insertions(+)
> > > > create mode 100644 drivers/nvme/nvme_apple.c
> > >
> > > Tested on: Macbook Air M1
> > > Tested-by: Simon Glass <sjg@chromium.org>
> > >
> > > >
> > > > diff --git a/configs/apple_m1_defconfig b/configs/apple_m1_defconfig
> > > > index cb235e4e7d..1528217b17 100644
> > > > --- a/configs/apple_m1_defconfig
> > > > +++ b/configs/apple_m1_defconfig
> > > > @@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
> > > > # CONFIG_NET is not set
> > > > # CONFIG_MMC is not set
> > > > CONFIG_DEBUG_UART_ANNOUNCE=y
> > > > +CONFIG_NVME_APPLE=y
> > > > CONFIG_USB_XHCI_HCD=y
> > > > CONFIG_USB_XHCI_DWC3=y
> > > > CONFIG_USB_KEYBOARD=y
> > > > diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
> > > > index 78da444c8b..0cb465160b 100644
> > > > --- a/drivers/nvme/Kconfig
> > > > +++ b/drivers/nvme/Kconfig
> > > > @@ -10,6 +10,17 @@ config NVME
> > > > This option enables support for NVM Express devices.
> > > > It supports basic functions of NVMe (read/write).
> > > >
> > > > +config NVME_APPLE
> > > > + bool "Apple NVMe controller support"
> > > > + select NVME
> > > > + help
> > > > + This option enables support for the NVMe storage
> > > > + controller integrated on Apple SoCs. This controller
> > > > + isn't PCI-based based and deviates from the NVMe
> > > > + standard implementation in its implementation of
> > > > + the command submission queue and the integration
> > > > + of an NVMMU that needs to be managed.
> > > > +
> > > > config NVME_PCI
> > > > bool "NVM Express PCI device support"
> > > > depends on PCI
> > > > diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
> > > > index fad9724e17..fa7b619446 100644
> > > > --- a/drivers/nvme/Makefile
> > > > +++ b/drivers/nvme/Makefile
> > > > @@ -3,4 +3,5 @@
> > > > # Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
> > > >
> > > > obj-y += nvme-uclass.o nvme.o nvme_show.o
> > > > +obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
> > > > obj-$(CONFIG_NVME_PCI) += nvme_pci.o
> > > > diff --git a/drivers/nvme/nvme_apple.c b/drivers/nvme/nvme_apple.c
> > > > new file mode 100644
> > > > index 0000000000..b0dc8492f0
> > > > --- /dev/null
> > > > +++ b/drivers/nvme/nvme_apple.c
> > > > @@ -0,0 +1,233 @@
> > > > +// SPDX-License-Identifier: GPL-2.0+
> > > > +/*
> > > > + * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
> > > > + */
> > > > +
> > > > +#include <common.h>
> > > > +#include <dm.h>
> > > > +#include <mailbox.h>
> > > > +#include <mapmem.h>
> > > > +#include "nvme.h"
> > > > +#include <reset.h>
> > > > +
> > > > +#include <asm/io.h>
> > > > +#include <asm/arch-apple/rtkit.h>
> > >
> > > asm/arch/ should work
> >
> > It doesn't. For some reason I end up with
> >
> > arch/arm/include/asm/arch -> arch-m1
> >
> > But this rtkit stuff is expected to be generic and apply to all Apple
> > SoCs, not just the M1.
>
> Hmm so you plan to have an arch/m2 as well? Can we not put everything
> in arch/apple then? We don't normally have arch directories for
> different generations unless they are completely different.
No, I think a single arch/arch-apple is better. That isolates us from
whatever silly marketing names Apple comes up with for their SoCs.
The goal is to have a single U-Boot binary that supports multiple
generations of the SoC.
That means I have to set SYS_SOC to "apple" instead of "m1" and move
the existing uart.h file into arch/arm/include/arch/arch-apple/ isn't
it?
I can add a diff that does this to the series.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller
2022-01-22 17:41 ` Mark Kettenis
@ 2022-01-22 18:28 ` Simon Glass
0 siblings, 0 replies; 33+ messages in thread
From: Simon Glass @ 2022-01-22 18:28 UTC (permalink / raw)
To: Mark Kettenis
Cc: Mark Kettenis, U-Boot Mailing List, Jaehoon Chung, Tom Rini,
sven, marcan, Bin Meng
Hi Mark,
On Sat, 22 Jan 2022 at 10:41, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Sat, 22 Jan 2022 10:17:08 -0700
> >
> > Hi Mark,
> >
> > On Sat, 22 Jan 2022 at 07:45, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> > >
> > > > From: Simon Glass <sjg@chromium.org>
> > > > Date: Fri, 21 Jan 2022 18:40:24 -0700
> > > >
> > > > Hi Mark,
> > > >
> > > > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > > > >
> > > > > Add a driver for the NVMe storage controller integrated on
> > > > > Apple SoCs. This NVMe controller isn't PCI based and deviates
> > > > > from the NVMe standard in its implementation of the command
> > > > > submission queue and the integration of an NVMMU that needs
> > > > > to be managed. This commit tweaks the core NVMe code to
> > > > > support the linear command submission queue implemented by
> > > > > this controller. But setting up the submission queue and
> > > > > managing the NVMMU controller is handled by implementing
> > > > > the driver ops that were added in an earlier commit.
> > > > >
> > > > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > > > Tested-on: firefly-rk3399
> > > > > Tested-by: Mark Kettenis <kettenis@openbsd.org>
> > > > > ---
> > > > > configs/apple_m1_defconfig | 1 +
> > > > > drivers/nvme/Kconfig | 11 ++
> > > > > drivers/nvme/Makefile | 1 +
> > > > > drivers/nvme/nvme_apple.c | 233 +++++++++++++++++++++++++++++++++++++
> > > > > 4 files changed, 246 insertions(+)
> > > > > create mode 100644 drivers/nvme/nvme_apple.c
> > > >
> > > > Tested on: Macbook Air M1
> > > > Tested-by: Simon Glass <sjg@chromium.org>
> > > >
> > > > >
> > > > > diff --git a/configs/apple_m1_defconfig b/configs/apple_m1_defconfig
> > > > > index cb235e4e7d..1528217b17 100644
> > > > > --- a/configs/apple_m1_defconfig
> > > > > +++ b/configs/apple_m1_defconfig
> > > > > @@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
> > > > > # CONFIG_NET is not set
> > > > > # CONFIG_MMC is not set
> > > > > CONFIG_DEBUG_UART_ANNOUNCE=y
> > > > > +CONFIG_NVME_APPLE=y
> > > > > CONFIG_USB_XHCI_HCD=y
> > > > > CONFIG_USB_XHCI_DWC3=y
> > > > > CONFIG_USB_KEYBOARD=y
> > > > > diff --git a/drivers/nvme/Kconfig b/drivers/nvme/Kconfig
> > > > > index 78da444c8b..0cb465160b 100644
> > > > > --- a/drivers/nvme/Kconfig
> > > > > +++ b/drivers/nvme/Kconfig
> > > > > @@ -10,6 +10,17 @@ config NVME
> > > > > This option enables support for NVM Express devices.
> > > > > It supports basic functions of NVMe (read/write).
> > > > >
> > > > > +config NVME_APPLE
> > > > > + bool "Apple NVMe controller support"
> > > > > + select NVME
> > > > > + help
> > > > > + This option enables support for the NVMe storage
> > > > > + controller integrated on Apple SoCs. This controller
> > > > > + isn't PCI-based based and deviates from the NVMe
> > > > > + standard implementation in its implementation of
> > > > > + the command submission queue and the integration
> > > > > + of an NVMMU that needs to be managed.
> > > > > +
> > > > > config NVME_PCI
> > > > > bool "NVM Express PCI device support"
> > > > > depends on PCI
> > > > > diff --git a/drivers/nvme/Makefile b/drivers/nvme/Makefile
> > > > > index fad9724e17..fa7b619446 100644
> > > > > --- a/drivers/nvme/Makefile
> > > > > +++ b/drivers/nvme/Makefile
> > > > > @@ -3,4 +3,5 @@
> > > > > # Copyright (C) 2017, Bin Meng <bmeng.cn@gmail.com>
> > > > >
> > > > > obj-y += nvme-uclass.o nvme.o nvme_show.o
> > > > > +obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
> > > > > obj-$(CONFIG_NVME_PCI) += nvme_pci.o
> > > > > diff --git a/drivers/nvme/nvme_apple.c b/drivers/nvme/nvme_apple.c
> > > > > new file mode 100644
> > > > > index 0000000000..b0dc8492f0
> > > > > --- /dev/null
> > > > > +++ b/drivers/nvme/nvme_apple.c
> > > > > @@ -0,0 +1,233 @@
> > > > > +// SPDX-License-Identifier: GPL-2.0+
> > > > > +/*
> > > > > + * (C) Copyright 2021 Mark Kettenis <kettenis@openbsd.org>
> > > > > + */
> > > > > +
> > > > > +#include <common.h>
> > > > > +#include <dm.h>
> > > > > +#include <mailbox.h>
> > > > > +#include <mapmem.h>
> > > > > +#include "nvme.h"
> > > > > +#include <reset.h>
> > > > > +
> > > > > +#include <asm/io.h>
> > > > > +#include <asm/arch-apple/rtkit.h>
> > > >
> > > > asm/arch/ should work
> > >
> > > It doesn't. For some reason I end up with
> > >
> > > arch/arm/include/asm/arch -> arch-m1
> > >
> > > But this rtkit stuff is expected to be generic and apply to all Apple
> > > SoCs, not just the M1.
> >
> > Hmm so you plan to have an arch/m2 as well? Can we not put everything
> > in arch/apple then? We don't normally have arch directories for
> > different generations unless they are completely different.
>
> No, I think a single arch/arch-apple is better. That isolates us from
> whatever silly marketing names Apple comes up with for their SoCs.
> The goal is to have a single U-Boot binary that supports multiple
> generations of the SoC.
Sounds good
>
> That means I have to set SYS_SOC to "apple" instead of "m1" and move
> the existing uart.h file into arch/arm/include/arch/arch-apple/ isn't
> it?
Yes that seems right.
>
> I can add a diff that does this to the series.
Regards,
Simon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 3/8] arm: apple: Add RTKit support
2022-01-22 17:17 ` Simon Glass
@ 2022-01-22 19:31 ` Mark Kettenis
0 siblings, 0 replies; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 19:31 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Sat, 22 Jan 2022 10:17:14 -0700
>
> Hi Mark,
>
> On Sat, 22 Jan 2022 at 06:59, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >
> > > From: Simon Glass <sjg@chromium.org>
> > > Date: Fri, 21 Jan 2022 18:40:14 -0700
> > >
> > > Hi Mark,
> > >
> > > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > > >
> > > > Most Apple IOPs run a firmware that is based on what Apple calls
> > > > RTKit. RTKit implements a common mailbox protocol. This code
> > > > provides an implementation of the AP side of this protocol,
> > > > providing a function to initialize RTKit-based firmwares as well
> > > > as a function to do a clean shutdown of this firmware.
> > > >
> > > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > > ---
> > > > arch/arm/include/asm/arch-apple/rtkit.h | 11 ++
> > > > arch/arm/mach-apple/Makefile | 1 +
> > > > arch/arm/mach-apple/rtkit.c | 231 ++++++++++++++++++++++++
> > > > 3 files changed, 243 insertions(+)
> > > > create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
> > > > create mode 100644 arch/arm/mach-apple/rtkit.c
> > >
> > > This should be a driver.
> >
> > This isn't a driver but just a bit of helper code that implements the
> > RTKit protocol. It will be used by the Apple NVMe driver and a future
> > SMC driver. The same approach is being taken in Linux. Since it is
> > intended to be shared, I didn't put it in the NVMe driver itself.
>
> It just seems to set up and close down the peripheral, so this could
> be a driver. What happens when someone needs to change it for a new
> generation? Normally we would use the compatible string for that, but
> here you have code that might get duplicated or parameterised in
> another way.
The protocol is versioned, and apple_rtkit_init() does the appropriate
version negotation. Currently we support protocol versions 11 and 12
(which seem to correspond to the firmware distributed with macOS 11
and macOS 12). But we haven't noticed any real differences between
those versions yet.
> Anyway if you really think this is the right way, it is OK. We do it
> elsewhere, iwc
>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Tested on: Macbook Air M1
> Tested-by: Simon Glass <sjg@chromium.org>
>
> Regards,
> Simon
>
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: [PATCH 6/8] power: domain: apple: Add reset support
2022-01-22 17:17 ` Simon Glass
@ 2022-01-22 19:35 ` Mark Kettenis
0 siblings, 0 replies; 33+ messages in thread
From: Mark Kettenis @ 2022-01-22 19:35 UTC (permalink / raw)
To: Simon Glass; +Cc: kettenis, u-boot, jh80.chung, trini, sven, marcan, bmeng.cn
> From: Simon Glass <sjg@chromium.org>
> Date: Sat, 22 Jan 2022 10:17:10 -0700
Hi Simon,
> Hi Mark,
>
> On Sat, 22 Jan 2022 at 07:12, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >
> > > From: Simon Glass <sjg@chromium.org>
> > > Date: Fri, 21 Jan 2022 18:40:23 -0700
> > >
> > > Hi Mark,
> > >
> > > On Fri, 14 Jan 2022 at 04:05, Mark Kettenis <kettenis@openbsd.org> wrote:
> > > >
> > > > The power management controller found on Apple SoCs als provides
> > > > a way to reset all devices within a power domain. This is needed
> > > > to cleanly shutdown the NVMe controller before we hand over
> > > > control to the OS.
> > > >
> > > > Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
> > > > ---
> > > > arch/arm/Kconfig | 1 +
> > > > drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
> > > > 2 files changed, 73 insertions(+), 1 deletion(-)
> > >
> > > This should use devicetree instead of device_bind() and be a reset
> > > driver in drivers/reset
> >
> > Not sure what you mean with "this should use devicetree". The reset
> > and power domain functionality is integrated in the same hardware
> > register and there is a single node that decsribes the device.
> >
> > The Linux driver implements the power domain and reset functionality
> > in a single driver as well. I suppose I could move the reset code
> > into a file of its own in drivers/reset, but I don't think it would
> > make the code easier to understand. And I'd still need to bind the
> > reset driver explicitly in apple_pmgr_probe() as it isn't possible to
> > automatically bind two drivers to a single device tree node as far as
> > I can tell.
>
> It seems odd that Linux does this sort of thing. The normal U-Boot
> approach would be to create a parent MFD driver with children for each
> uclass. Does linux do that these days?
It seems to be common for pinctrl/gpio and clk/reset drivers to be
combined in Linux. Not sure if there was a pd/reset combo before the
M1 support was accepted, but it is there.
MFD is typically used for higher-level functions, i.e. PMIC IC's that
include both voltage regulators and an RTC for example.
> Driver model cannot create two drivers from one device tree node,
> although you can create one manually later as you have done.
>
> It seems OK to follow along with Linux, if it makes it easier to
> maintain. Otherwise, drivers/reset would be better for the reset
> driver.
>
> Either way:
>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> Tested on: Macbook Air M1
> Tested-by: Simon Glass <sjg@chromium.org>
I'll keep it as-is then.
Thanks,
Mark
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2022-01-22 19:35 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-14 11:04 [PATCH 0/8] Apple M1 NVMe storage support Mark Kettenis
2022-01-14 11:04 ` [PATCH 1/8] nvme: Split out PCI support Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-22 12:47 ` Mark Kettenis
2022-01-22 13:18 ` Simon Glass
2022-01-22 14:57 ` Mark Kettenis
2022-01-14 11:04 ` [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-22 13:54 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
2022-01-14 11:04 ` [PATCH 3/8] arm: apple: Add RTKit support Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-22 13:59 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
2022-01-22 19:31 ` Mark Kettenis
2022-01-14 11:04 ` [PATCH 4/8] nvme: Introduce driver ops Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-22 13:33 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
2022-01-14 11:04 ` [PATCH 5/8] nvme: Add shutdown function Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-14 11:04 ` [PATCH 6/8] power: domain: apple: Add reset support Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-22 14:11 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
2022-01-22 19:35 ` Mark Kettenis
2022-01-14 11:04 ` [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller Mark Kettenis
2022-01-22 1:40 ` Simon Glass
2022-01-22 14:45 ` Mark Kettenis
2022-01-22 17:17 ` Simon Glass
2022-01-22 17:41 ` Mark Kettenis
2022-01-22 18:28 ` Simon Glass
2022-01-14 11:04 ` [PATCH 8/8] configs: apple: Add NVMe boot target Mark Kettenis
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.