* [RFC PATCH v2 00/10] firmware: arm_scmi: Add virtio transport @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, igor.skalkin, mikhail.golubev, anton.yakovlev This series implements an SCMI virtio driver according to the virtio SCMI device spec patch v5 [1], after simple preparatory changes to the existing arm-scmi driver. The virtio transport differs in some respects from the existing shared-memory based SCMI transports. Message timeouts can be a problem if the virtio device (SCMI platform) does not have real-time properties. I set a high message rx timeout value which should work for non real-time virtio devices as well. There are other timeouts for delayed response and polling which were not addressed yet. Delayed responses are not really needed since, with the virtio transport, message responses may be transmitted out of order. Polling doesn't make sense at least for virtio devices without real-time behavior, in my understanding. There are some known issues which will be resolved before removing the RFC tag: - Further work is needed on the scmi_xfer management. Unlike shmem based transports, the virtio transport is actually exchanging messages with the SCMI agent through the scmi_xfer tx and rx buffers. In case of a message rx timeout, the arm-scmi driver could try to re-use the scmi_xfer, while that might still be used by the virtio device. I think part of the scmi_xfers_info bookkeeping could be optionally outsourced to the transport to remediate this. - After arm-scmi driver probe failure, or after remove, the scmi-virtio driver may still try to process and forward message responses from the virtio device. - We must be sure that the virtio transport option (such as virtio over MMIO) is available when the virtio SCMI device is probed. The series is based on for-next/scmi [2], on top of commit b9ceca6be432 ("firmware: arm_scmi: Fix duplicate workqueue name") The series was actually tested with a v5.4 based kernel, with the Base protocol and Sensor management protocol. The virtio SCMI device used was a proprietary implementation by OpenSynergy. Delayed responses were not tested. Changes in RFC v2: - Remove the DT virtio_transport phandle, since the SCMI virtio device may not be known in advance. Instead, use the first suitable probed device. Change due to Rob Herring's comment. Any comments are very welcome. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html [2] https://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git Igor Skalkin (7): firmware: arm_scmi, smccc, mailbox: Make shmem based transports optional firmware: arm_scmi: Document that max_msg is a per channel type limit firmware: arm_scmi: Add op to override max message # firmware: arm_scmi: Add per message transport data firmware: arm_scmi: Add xfer_init_buffers transport op dt-bindings: arm: Add virtio transport for SCMI firmware: arm_scmi: Add virtio transport Peter Hilber (3): firmware: arm_scmi: Add optional link_supplier() transport op firmware: arm_scmi: Add per-device transport private info firmware: arm_scmi: Add is_scmi_protocol_device() .../devicetree/bindings/arm/arm,scmi.txt | 35 +- MAINTAINERS | 1 + drivers/firmware/Kconfig | 19 +- drivers/firmware/arm_scmi/Makefile | 3 +- drivers/firmware/arm_scmi/bus.c | 5 + drivers/firmware/arm_scmi/common.h | 37 +- drivers/firmware/arm_scmi/driver.c | 124 ++++- drivers/firmware/arm_scmi/virtio.c | 493 ++++++++++++++++++ drivers/firmware/smccc/Kconfig | 1 + drivers/mailbox/Kconfig | 1 + include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_scmi.h | 41 ++ 12 files changed, 736 insertions(+), 25 deletions(-) create mode 100644 drivers/firmware/arm_scmi/virtio.c create mode 100644 include/uapi/linux/virtio_scmi.h base-commit: b9ceca6be43233845be70792be9b5ab315d2e010 -- 2.25.1 ^ permalink raw reply [flat|nested] 34+ messages in thread
* [virtio-dev] [RFC PATCH v2 00/10] firmware: arm_scmi: Add virtio transport @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, igor.skalkin, mikhail.golubev, anton.yakovlev This series implements an SCMI virtio driver according to the virtio SCMI device spec patch v5 [1], after simple preparatory changes to the existing arm-scmi driver. The virtio transport differs in some respects from the existing shared-memory based SCMI transports. Message timeouts can be a problem if the virtio device (SCMI platform) does not have real-time properties. I set a high message rx timeout value which should work for non real-time virtio devices as well. There are other timeouts for delayed response and polling which were not addressed yet. Delayed responses are not really needed since, with the virtio transport, message responses may be transmitted out of order. Polling doesn't make sense at least for virtio devices without real-time behavior, in my understanding. There are some known issues which will be resolved before removing the RFC tag: - Further work is needed on the scmi_xfer management. Unlike shmem based transports, the virtio transport is actually exchanging messages with the SCMI agent through the scmi_xfer tx and rx buffers. In case of a message rx timeout, the arm-scmi driver could try to re-use the scmi_xfer, while that might still be used by the virtio device. I think part of the scmi_xfers_info bookkeeping could be optionally outsourced to the transport to remediate this. - After arm-scmi driver probe failure, or after remove, the scmi-virtio driver may still try to process and forward message responses from the virtio device. - We must be sure that the virtio transport option (such as virtio over MMIO) is available when the virtio SCMI device is probed. The series is based on for-next/scmi [2], on top of commit b9ceca6be432 ("firmware: arm_scmi: Fix duplicate workqueue name") The series was actually tested with a v5.4 based kernel, with the Base protocol and Sensor management protocol. The virtio SCMI device used was a proprietary implementation by OpenSynergy. Delayed responses were not tested. Changes in RFC v2: - Remove the DT virtio_transport phandle, since the SCMI virtio device may not be known in advance. Instead, use the first suitable probed device. Change due to Rob Herring's comment. Any comments are very welcome. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html [2] https://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git Igor Skalkin (7): firmware: arm_scmi, smccc, mailbox: Make shmem based transports optional firmware: arm_scmi: Document that max_msg is a per channel type limit firmware: arm_scmi: Add op to override max message # firmware: arm_scmi: Add per message transport data firmware: arm_scmi: Add xfer_init_buffers transport op dt-bindings: arm: Add virtio transport for SCMI firmware: arm_scmi: Add virtio transport Peter Hilber (3): firmware: arm_scmi: Add optional link_supplier() transport op firmware: arm_scmi: Add per-device transport private info firmware: arm_scmi: Add is_scmi_protocol_device() .../devicetree/bindings/arm/arm,scmi.txt | 35 +- MAINTAINERS | 1 + drivers/firmware/Kconfig | 19 +- drivers/firmware/arm_scmi/Makefile | 3 +- drivers/firmware/arm_scmi/bus.c | 5 + drivers/firmware/arm_scmi/common.h | 37 +- drivers/firmware/arm_scmi/driver.c | 124 ++++- drivers/firmware/arm_scmi/virtio.c | 493 ++++++++++++++++++ drivers/firmware/smccc/Kconfig | 1 + drivers/mailbox/Kconfig | 1 + include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_scmi.h | 41 ++ 12 files changed, 736 insertions(+), 25 deletions(-) create mode 100644 drivers/firmware/arm_scmi/virtio.c create mode 100644 include/uapi/linux/virtio_scmi.h base-commit: b9ceca6be43233845be70792be9b5ab315d2e010 -- 2.25.1 --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org ^ permalink raw reply [flat|nested] 34+ messages in thread
* [RFC PATCH v2 00/10] firmware: arm_scmi: Add virtio transport @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, igor.skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee This series implements an SCMI virtio driver according to the virtio SCMI device spec patch v5 [1], after simple preparatory changes to the existing arm-scmi driver. The virtio transport differs in some respects from the existing shared-memory based SCMI transports. Message timeouts can be a problem if the virtio device (SCMI platform) does not have real-time properties. I set a high message rx timeout value which should work for non real-time virtio devices as well. There are other timeouts for delayed response and polling which were not addressed yet. Delayed responses are not really needed since, with the virtio transport, message responses may be transmitted out of order. Polling doesn't make sense at least for virtio devices without real-time behavior, in my understanding. There are some known issues which will be resolved before removing the RFC tag: - Further work is needed on the scmi_xfer management. Unlike shmem based transports, the virtio transport is actually exchanging messages with the SCMI agent through the scmi_xfer tx and rx buffers. In case of a message rx timeout, the arm-scmi driver could try to re-use the scmi_xfer, while that might still be used by the virtio device. I think part of the scmi_xfers_info bookkeeping could be optionally outsourced to the transport to remediate this. - After arm-scmi driver probe failure, or after remove, the scmi-virtio driver may still try to process and forward message responses from the virtio device. - We must be sure that the virtio transport option (such as virtio over MMIO) is available when the virtio SCMI device is probed. The series is based on for-next/scmi [2], on top of commit b9ceca6be432 ("firmware: arm_scmi: Fix duplicate workqueue name") The series was actually tested with a v5.4 based kernel, with the Base protocol and Sensor management protocol. The virtio SCMI device used was a proprietary implementation by OpenSynergy. Delayed responses were not tested. Changes in RFC v2: - Remove the DT virtio_transport phandle, since the SCMI virtio device may not be known in advance. Instead, use the first suitable probed device. Change due to Rob Herring's comment. Any comments are very welcome. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html [2] https://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git Igor Skalkin (7): firmware: arm_scmi, smccc, mailbox: Make shmem based transports optional firmware: arm_scmi: Document that max_msg is a per channel type limit firmware: arm_scmi: Add op to override max message # firmware: arm_scmi: Add per message transport data firmware: arm_scmi: Add xfer_init_buffers transport op dt-bindings: arm: Add virtio transport for SCMI firmware: arm_scmi: Add virtio transport Peter Hilber (3): firmware: arm_scmi: Add optional link_supplier() transport op firmware: arm_scmi: Add per-device transport private info firmware: arm_scmi: Add is_scmi_protocol_device() .../devicetree/bindings/arm/arm,scmi.txt | 35 +- MAINTAINERS | 1 + drivers/firmware/Kconfig | 19 +- drivers/firmware/arm_scmi/Makefile | 3 +- drivers/firmware/arm_scmi/bus.c | 5 + drivers/firmware/arm_scmi/common.h | 37 +- drivers/firmware/arm_scmi/driver.c | 124 ++++- drivers/firmware/arm_scmi/virtio.c | 493 ++++++++++++++++++ drivers/firmware/smccc/Kconfig | 1 + drivers/mailbox/Kconfig | 1 + include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_scmi.h | 41 ++ 12 files changed, 736 insertions(+), 25 deletions(-) create mode 100644 drivers/firmware/arm_scmi/virtio.c create mode 100644 include/uapi/linux/virtio_scmi.h base-commit: b9ceca6be43233845be70792be9b5ab315d2e010 -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 34+ messages in thread
* [RFC PATCH v2 01/10] firmware: arm_scmi, smccc, mailbox: Make shmem based transports optional 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Mark Rutland, Lorenzo Pieralisi, Jassi Brar From: Igor Skalkin <igor.skalkin@opensynergy.com> Upon adding the virtio transport in this patch series, SCMI will also work without shared memory based transports. Also, the mailbox transport may not be needed if the smc transport is used. - Compile shmem.c only if a shmem based transport is available. - Remove hard dependency of SCMI on mailbox. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/Kconfig | 9 ++++++++- drivers/firmware/arm_scmi/Makefile | 2 +- drivers/firmware/arm_scmi/common.h | 2 ++ drivers/firmware/arm_scmi/driver.c | 2 ++ drivers/firmware/smccc/Kconfig | 1 + drivers/mailbox/Kconfig | 1 + 6 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index afdbebba628a..bdde51adb267 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -9,7 +9,7 @@ menu "Firmware Drivers" config ARM_SCMI_PROTOCOL tristate "ARM System Control and Management Interface (SCMI) Message Protocol" depends on ARM || ARM64 || COMPILE_TEST - depends on MAILBOX + depends on ARM_SCMI_HAVE_SHMEM help ARM System Control and Management Interface (SCMI) protocol is a set of operating system-independent software interfaces that are @@ -27,6 +27,13 @@ config ARM_SCMI_PROTOCOL This protocol library provides interface for all the client drivers making use of the features offered by the SCMI. +config ARM_SCMI_HAVE_SHMEM + bool + default n + help + This declares whether a shared memory based transport for SCMI is + available. + config ARM_SCMI_POWER_DOMAIN tristate "SCMI power domain driver" depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile index bc0d54f8e861..3cc7fa40a464 100644 --- a/drivers/firmware/arm_scmi/Makefile +++ b/drivers/firmware/arm_scmi/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only scmi-bus-y = bus.o scmi-driver-y = driver.o notify.o -scmi-transport-y = shmem.o +scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o scmi-transport-$(CONFIG_MAILBOX) += mailbox.o scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 65063fa948d4..aed192238177 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -242,7 +242,9 @@ struct scmi_desc { int max_msg_size; }; +#ifdef CONFIG_MAILBOX extern const struct scmi_desc scmi_mailbox_desc; +#endif #ifdef CONFIG_HAVE_ARM_SMCCC extern const struct scmi_desc scmi_smc_desc; #endif diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 3dfd8b6a0ebf..9e2f36127793 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -918,7 +918,9 @@ ATTRIBUTE_GROUPS(versions); /* Each compatible listed below must have descriptor associated with it */ static const struct of_device_id scmi_of_match[] = { +#ifdef CONFIG_MAILBOX { .compatible = "arm,scmi", .data = &scmi_mailbox_desc }, +#endif #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, #endif diff --git a/drivers/firmware/smccc/Kconfig b/drivers/firmware/smccc/Kconfig index 15e7466179a6..69c4d6cabf62 100644 --- a/drivers/firmware/smccc/Kconfig +++ b/drivers/firmware/smccc/Kconfig @@ -9,6 +9,7 @@ config HAVE_ARM_SMCCC_DISCOVERY bool depends on ARM_PSCI_FW default y + select ARM_SCMI_HAVE_SHMEM help SMCCC v1.0 lacked discoverability and hence PSCI v1.0 was updated to add SMCCC discovery mechanism though the PSCI firmware diff --git a/drivers/mailbox/Kconfig b/drivers/mailbox/Kconfig index 05b1009e2820..5ffe1ab0c869 100644 --- a/drivers/mailbox/Kconfig +++ b/drivers/mailbox/Kconfig @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only menuconfig MAILBOX bool "Mailbox Hardware Support" + select ARM_SCMI_HAVE_SHMEM help Mailbox is a framework to control hardware communication between on-chip processors through queued messages and interrupt driven -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 01/10] firmware: arm_scmi, smccc, mailbox: Make shmem based transports optional @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Mark Rutland, jean-philippe, mikhail.golubev, Lorenzo Pieralisi, souvik.chakravarty, Igor Skalkin, Jassi Brar, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> Upon adding the virtio transport in this patch series, SCMI will also work without shared memory based transports. Also, the mailbox transport may not be needed if the smc transport is used. - Compile shmem.c only if a shmem based transport is available. - Remove hard dependency of SCMI on mailbox. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/Kconfig | 9 ++++++++- drivers/firmware/arm_scmi/Makefile | 2 +- drivers/firmware/arm_scmi/common.h | 2 ++ drivers/firmware/arm_scmi/driver.c | 2 ++ drivers/firmware/smccc/Kconfig | 1 + drivers/mailbox/Kconfig | 1 + 6 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index afdbebba628a..bdde51adb267 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -9,7 +9,7 @@ menu "Firmware Drivers" config ARM_SCMI_PROTOCOL tristate "ARM System Control and Management Interface (SCMI) Message Protocol" depends on ARM || ARM64 || COMPILE_TEST - depends on MAILBOX + depends on ARM_SCMI_HAVE_SHMEM help ARM System Control and Management Interface (SCMI) protocol is a set of operating system-independent software interfaces that are @@ -27,6 +27,13 @@ config ARM_SCMI_PROTOCOL This protocol library provides interface for all the client drivers making use of the features offered by the SCMI. +config ARM_SCMI_HAVE_SHMEM + bool + default n + help + This declares whether a shared memory based transport for SCMI is + available. + config ARM_SCMI_POWER_DOMAIN tristate "SCMI power domain driver" depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile index bc0d54f8e861..3cc7fa40a464 100644 --- a/drivers/firmware/arm_scmi/Makefile +++ b/drivers/firmware/arm_scmi/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only scmi-bus-y = bus.o scmi-driver-y = driver.o notify.o -scmi-transport-y = shmem.o +scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o scmi-transport-$(CONFIG_MAILBOX) += mailbox.o scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 65063fa948d4..aed192238177 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -242,7 +242,9 @@ struct scmi_desc { int max_msg_size; }; +#ifdef CONFIG_MAILBOX extern const struct scmi_desc scmi_mailbox_desc; +#endif #ifdef CONFIG_HAVE_ARM_SMCCC extern const struct scmi_desc scmi_smc_desc; #endif diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 3dfd8b6a0ebf..9e2f36127793 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -918,7 +918,9 @@ ATTRIBUTE_GROUPS(versions); /* Each compatible listed below must have descriptor associated with it */ static const struct of_device_id scmi_of_match[] = { +#ifdef CONFIG_MAILBOX { .compatible = "arm,scmi", .data = &scmi_mailbox_desc }, +#endif #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, #endif diff --git a/drivers/firmware/smccc/Kconfig b/drivers/firmware/smccc/Kconfig index 15e7466179a6..69c4d6cabf62 100644 --- a/drivers/firmware/smccc/Kconfig +++ b/drivers/firmware/smccc/Kconfig @@ -9,6 +9,7 @@ config HAVE_ARM_SMCCC_DISCOVERY bool depends on ARM_PSCI_FW default y + select ARM_SCMI_HAVE_SHMEM help SMCCC v1.0 lacked discoverability and hence PSCI v1.0 was updated to add SMCCC discovery mechanism though the PSCI firmware diff --git a/drivers/mailbox/Kconfig b/drivers/mailbox/Kconfig index 05b1009e2820..5ffe1ab0c869 100644 --- a/drivers/mailbox/Kconfig +++ b/drivers/mailbox/Kconfig @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only menuconfig MAILBOX bool "Mailbox Hardware Support" + select ARM_SCMI_HAVE_SHMEM help Mailbox is a framework to control hardware communication between on-chip processors through queued messages and interrupt driven -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 02/10] firmware: arm_scmi: Document that max_msg is a per channel type limit 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev From: Igor Skalkin <igor.skalkin@opensynergy.com> struct scmi_desc.max_msg specifies a limit for the pending messages. This limit is a per SCMI channel type (tx, rx) limit. State that explicitly in the inline documentation. The following patch will add an op to override the limit per channel type. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index aed192238177..38e6aabbe3dd 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -231,8 +231,8 @@ struct scmi_transport_ops { * * @ops: Pointer to the transport specific ops structure * @max_rx_timeout_ms: Timeout for communication with SoC (in Milliseconds) - * @max_msg: Maximum number of messages that can be pending - * simultaneously in the system + * @max_msg: Maximum number of messages for a channel type (tx or rx) that can + * be pending simultaneously in the system * @max_msg_size: Maximum size of data per message that can be handled. */ struct scmi_desc { -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 02/10] firmware: arm_scmi: Document that max_msg is a per channel type limit @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> struct scmi_desc.max_msg specifies a limit for the pending messages. This limit is a per SCMI channel type (tx, rx) limit. State that explicitly in the inline documentation. The following patch will add an op to override the limit per channel type. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index aed192238177..38e6aabbe3dd 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -231,8 +231,8 @@ struct scmi_transport_ops { * * @ops: Pointer to the transport specific ops structure * @max_rx_timeout_ms: Timeout for communication with SoC (in Milliseconds) - * @max_msg: Maximum number of messages that can be pending - * simultaneously in the system + * @max_msg: Maximum number of messages for a channel type (tx or rx) that can + * be pending simultaneously in the system * @max_msg_size: Maximum size of data per message that can be handled. */ struct scmi_desc { -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 03/10] firmware: arm_scmi: Add op to override max message # 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev From: Igor Skalkin <igor.skalkin@opensynergy.com> The number of messages that the upcoming scmi-virtio transport can support depends on the virtio device (SCMI platform) and can differ for each channel. (The scmi-virtio transport does only have one tx and at most 1 rx channel.) Add an optional transport op so that scmi-virtio can report the actual max message # for each channel type. Respect these new limits. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 8 ++++- drivers/firmware/arm_scmi/driver.c | 49 ++++++++++++++++++++++-------- 2 files changed, 43 insertions(+), 14 deletions(-) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 38e6aabbe3dd..9a8359ecd220 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -203,6 +203,9 @@ struct scmi_chan_info { * @chan_available: Callback to check if channel is available or not * @chan_setup: Callback to allocate and setup a channel * @chan_free: Callback to free a channel + * @get_max_msg: Optional callback to provide max_msg dynamically + * @max_msg: Maximum number of messages for the channel type (tx or rx) + * that can be pending simultaneously in the system * @send_message: Callback to send a message * @mark_txdone: Callback to mark tx as done * @fetch_response: Callback to fetch response @@ -215,6 +218,8 @@ struct scmi_transport_ops { int (*chan_setup)(struct scmi_chan_info *cinfo, struct device *dev, bool tx); int (*chan_free)(int id, void *p, void *data); + int (*get_max_msg)(bool tx, struct scmi_chan_info *base_cinfo, + int *max_msg); int (*send_message)(struct scmi_chan_info *cinfo, struct scmi_xfer *xfer); void (*mark_txdone)(struct scmi_chan_info *cinfo, int ret); @@ -232,7 +237,8 @@ struct scmi_transport_ops { * @ops: Pointer to the transport specific ops structure * @max_rx_timeout_ms: Timeout for communication with SoC (in Milliseconds) * @max_msg: Maximum number of messages for a channel type (tx or rx) that can - * be pending simultaneously in the system + * be pending simultaneously in the system. May be overridden by the + * get_max_msg op. * @max_msg_size: Maximum size of data per message that can be handled. */ struct scmi_desc { diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 9e2f36127793..e2faa526dfce 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -61,11 +61,13 @@ static atomic_t transfer_last_id; * Index of this bitmap table is also used for message * sequence identifier. * @xfer_lock: Protection for message allocation + * @max_msg: Maximum number of messages that can be pending */ struct scmi_xfers_info { struct scmi_xfer *xfer_block; unsigned long *xfer_alloc_table; spinlock_t xfer_lock; + int max_msg; }; /** @@ -157,13 +159,11 @@ static struct scmi_xfer *scmi_xfer_get(const struct scmi_handle *handle, u16 xfer_id; struct scmi_xfer *xfer; unsigned long flags, bit_pos; - struct scmi_info *info = handle_to_scmi_info(handle); /* Keep the locked section as small as possible */ spin_lock_irqsave(&minfo->xfer_lock, flags); - bit_pos = find_first_zero_bit(minfo->xfer_alloc_table, - info->desc->max_msg); - if (bit_pos == info->desc->max_msg) { + bit_pos = find_first_zero_bit(minfo->xfer_alloc_table, minfo->max_msg); + if (bit_pos == minfo->max_msg) { spin_unlock_irqrestore(&minfo->xfer_lock, flags); return ERR_PTR(-ENOMEM); } @@ -602,32 +602,44 @@ int scmi_handle_put(const struct scmi_handle *handle) } static int __scmi_xfer_info_init(struct scmi_info *sinfo, - struct scmi_xfers_info *info) + struct scmi_xfers_info *info, + bool tx, + struct scmi_chan_info *base_cinfo) { int i; struct scmi_xfer *xfer; struct device *dev = sinfo->dev; const struct scmi_desc *desc = sinfo->desc; + info->max_msg = desc->max_msg; + + if (desc->ops->get_max_msg) { + int ret = + desc->ops->get_max_msg(tx, base_cinfo, &info->max_msg); + + if (ret) + return ret; + } + /* Pre-allocated messages, no more than what hdr.seq can support */ - if (WARN_ON(desc->max_msg >= MSG_TOKEN_MAX)) { + if (WARN_ON(info->max_msg >= MSG_TOKEN_MAX)) { dev_err(dev, "Maximum message of %d exceeds supported %ld\n", - desc->max_msg, MSG_TOKEN_MAX); + info->max_msg, MSG_TOKEN_MAX); return -EINVAL; } - info->xfer_block = devm_kcalloc(dev, desc->max_msg, + info->xfer_block = devm_kcalloc(dev, info->max_msg, sizeof(*info->xfer_block), GFP_KERNEL); if (!info->xfer_block) return -ENOMEM; - info->xfer_alloc_table = devm_kcalloc(dev, BITS_TO_LONGS(desc->max_msg), + info->xfer_alloc_table = devm_kcalloc(dev, BITS_TO_LONGS(info->max_msg), sizeof(long), GFP_KERNEL); if (!info->xfer_alloc_table) return -ENOMEM; /* Pre-initialize the buffer pointer to pre-allocated buffers */ - for (i = 0, xfer = info->xfer_block; i < desc->max_msg; i++, xfer++) { + for (i = 0, xfer = info->xfer_block; i < info->max_msg; i++, xfer++) { xfer->rx.buf = devm_kcalloc(dev, sizeof(u8), desc->max_msg_size, GFP_KERNEL); if (!xfer->rx.buf) @@ -644,10 +656,21 @@ static int __scmi_xfer_info_init(struct scmi_info *sinfo, static int scmi_xfer_info_init(struct scmi_info *sinfo) { - int ret = __scmi_xfer_info_init(sinfo, &sinfo->tx_minfo); + int ret; + struct scmi_chan_info *base_tx_cinfo; + struct scmi_chan_info *base_rx_cinfo; + + base_tx_cinfo = idr_find(&sinfo->tx_idr, SCMI_PROTOCOL_BASE); + if (unlikely(!base_tx_cinfo)) + return -EINVAL; + + ret = __scmi_xfer_info_init(sinfo, &sinfo->tx_minfo, true, + base_tx_cinfo); - if (!ret && idr_find(&sinfo->rx_idr, SCMI_PROTOCOL_BASE)) - ret = __scmi_xfer_info_init(sinfo, &sinfo->rx_minfo); + base_rx_cinfo = idr_find(&sinfo->rx_idr, SCMI_PROTOCOL_BASE); + if (!ret && base_rx_cinfo) + ret = __scmi_xfer_info_init(sinfo, &sinfo->rx_minfo, false, + base_rx_cinfo); return ret; } -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 03/10] firmware: arm_scmi: Add op to override max message # @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> The number of messages that the upcoming scmi-virtio transport can support depends on the virtio device (SCMI platform) and can differ for each channel. (The scmi-virtio transport does only have one tx and at most 1 rx channel.) Add an optional transport op so that scmi-virtio can report the actual max message # for each channel type. Respect these new limits. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 8 ++++- drivers/firmware/arm_scmi/driver.c | 49 ++++++++++++++++++++++-------- 2 files changed, 43 insertions(+), 14 deletions(-) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 38e6aabbe3dd..9a8359ecd220 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -203,6 +203,9 @@ struct scmi_chan_info { * @chan_available: Callback to check if channel is available or not * @chan_setup: Callback to allocate and setup a channel * @chan_free: Callback to free a channel + * @get_max_msg: Optional callback to provide max_msg dynamically + * @max_msg: Maximum number of messages for the channel type (tx or rx) + * that can be pending simultaneously in the system * @send_message: Callback to send a message * @mark_txdone: Callback to mark tx as done * @fetch_response: Callback to fetch response @@ -215,6 +218,8 @@ struct scmi_transport_ops { int (*chan_setup)(struct scmi_chan_info *cinfo, struct device *dev, bool tx); int (*chan_free)(int id, void *p, void *data); + int (*get_max_msg)(bool tx, struct scmi_chan_info *base_cinfo, + int *max_msg); int (*send_message)(struct scmi_chan_info *cinfo, struct scmi_xfer *xfer); void (*mark_txdone)(struct scmi_chan_info *cinfo, int ret); @@ -232,7 +237,8 @@ struct scmi_transport_ops { * @ops: Pointer to the transport specific ops structure * @max_rx_timeout_ms: Timeout for communication with SoC (in Milliseconds) * @max_msg: Maximum number of messages for a channel type (tx or rx) that can - * be pending simultaneously in the system + * be pending simultaneously in the system. May be overridden by the + * get_max_msg op. * @max_msg_size: Maximum size of data per message that can be handled. */ struct scmi_desc { diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 9e2f36127793..e2faa526dfce 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -61,11 +61,13 @@ static atomic_t transfer_last_id; * Index of this bitmap table is also used for message * sequence identifier. * @xfer_lock: Protection for message allocation + * @max_msg: Maximum number of messages that can be pending */ struct scmi_xfers_info { struct scmi_xfer *xfer_block; unsigned long *xfer_alloc_table; spinlock_t xfer_lock; + int max_msg; }; /** @@ -157,13 +159,11 @@ static struct scmi_xfer *scmi_xfer_get(const struct scmi_handle *handle, u16 xfer_id; struct scmi_xfer *xfer; unsigned long flags, bit_pos; - struct scmi_info *info = handle_to_scmi_info(handle); /* Keep the locked section as small as possible */ spin_lock_irqsave(&minfo->xfer_lock, flags); - bit_pos = find_first_zero_bit(minfo->xfer_alloc_table, - info->desc->max_msg); - if (bit_pos == info->desc->max_msg) { + bit_pos = find_first_zero_bit(minfo->xfer_alloc_table, minfo->max_msg); + if (bit_pos == minfo->max_msg) { spin_unlock_irqrestore(&minfo->xfer_lock, flags); return ERR_PTR(-ENOMEM); } @@ -602,32 +602,44 @@ int scmi_handle_put(const struct scmi_handle *handle) } static int __scmi_xfer_info_init(struct scmi_info *sinfo, - struct scmi_xfers_info *info) + struct scmi_xfers_info *info, + bool tx, + struct scmi_chan_info *base_cinfo) { int i; struct scmi_xfer *xfer; struct device *dev = sinfo->dev; const struct scmi_desc *desc = sinfo->desc; + info->max_msg = desc->max_msg; + + if (desc->ops->get_max_msg) { + int ret = + desc->ops->get_max_msg(tx, base_cinfo, &info->max_msg); + + if (ret) + return ret; + } + /* Pre-allocated messages, no more than what hdr.seq can support */ - if (WARN_ON(desc->max_msg >= MSG_TOKEN_MAX)) { + if (WARN_ON(info->max_msg >= MSG_TOKEN_MAX)) { dev_err(dev, "Maximum message of %d exceeds supported %ld\n", - desc->max_msg, MSG_TOKEN_MAX); + info->max_msg, MSG_TOKEN_MAX); return -EINVAL; } - info->xfer_block = devm_kcalloc(dev, desc->max_msg, + info->xfer_block = devm_kcalloc(dev, info->max_msg, sizeof(*info->xfer_block), GFP_KERNEL); if (!info->xfer_block) return -ENOMEM; - info->xfer_alloc_table = devm_kcalloc(dev, BITS_TO_LONGS(desc->max_msg), + info->xfer_alloc_table = devm_kcalloc(dev, BITS_TO_LONGS(info->max_msg), sizeof(long), GFP_KERNEL); if (!info->xfer_alloc_table) return -ENOMEM; /* Pre-initialize the buffer pointer to pre-allocated buffers */ - for (i = 0, xfer = info->xfer_block; i < desc->max_msg; i++, xfer++) { + for (i = 0, xfer = info->xfer_block; i < info->max_msg; i++, xfer++) { xfer->rx.buf = devm_kcalloc(dev, sizeof(u8), desc->max_msg_size, GFP_KERNEL); if (!xfer->rx.buf) @@ -644,10 +656,21 @@ static int __scmi_xfer_info_init(struct scmi_info *sinfo, static int scmi_xfer_info_init(struct scmi_info *sinfo) { - int ret = __scmi_xfer_info_init(sinfo, &sinfo->tx_minfo); + int ret; + struct scmi_chan_info *base_tx_cinfo; + struct scmi_chan_info *base_rx_cinfo; + + base_tx_cinfo = idr_find(&sinfo->tx_idr, SCMI_PROTOCOL_BASE); + if (unlikely(!base_tx_cinfo)) + return -EINVAL; + + ret = __scmi_xfer_info_init(sinfo, &sinfo->tx_minfo, true, + base_tx_cinfo); - if (!ret && idr_find(&sinfo->rx_idr, SCMI_PROTOCOL_BASE)) - ret = __scmi_xfer_info_init(sinfo, &sinfo->rx_minfo); + base_rx_cinfo = idr_find(&sinfo->rx_idr, SCMI_PROTOCOL_BASE); + if (!ret && base_rx_cinfo) + ret = __scmi_xfer_info_init(sinfo, &sinfo->rx_minfo, false, + base_rx_cinfo); return ret; } -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 04/10] firmware: arm_scmi: Add per message transport data 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev From: Igor Skalkin <igor.skalkin@opensynergy.com> The virtio transport in this patch series can be simplified by using the scmi_xfer tx/rx buffers for data exchange with the virtio device, and for saving the message state. But the virtio transport requires prepending a transport-specific header. Also, for data exchange using virtqueues, the tx and rx buffers should not overlap. The first step to solve the aforementioned issues is to add a transport-specific data pointer to scmi_xfer. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 9a8359ecd220..c998ec29018e 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -131,6 +131,7 @@ struct scmi_msg { * buffer for the rx path as we use for the tx path. * @done: command message transmit completion event * @async_done: pointer to delayed response message received event completion + * @extra_data: Transport-specific private data pointer */ struct scmi_xfer { int transfer_id; @@ -139,6 +140,7 @@ struct scmi_xfer { struct scmi_msg rx; struct completion done; struct completion *async_done; + void *extra_data; }; void scmi_xfer_put(const struct scmi_handle *h, struct scmi_xfer *xfer); -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 04/10] firmware: arm_scmi: Add per message transport data @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> The virtio transport in this patch series can be simplified by using the scmi_xfer tx/rx buffers for data exchange with the virtio device, and for saving the message state. But the virtio transport requires prepending a transport-specific header. Also, for data exchange using virtqueues, the tx and rx buffers should not overlap. The first step to solve the aforementioned issues is to add a transport-specific data pointer to scmi_xfer. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 9a8359ecd220..c998ec29018e 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -131,6 +131,7 @@ struct scmi_msg { * buffer for the rx path as we use for the tx path. * @done: command message transmit completion event * @async_done: pointer to delayed response message received event completion + * @extra_data: Transport-specific private data pointer */ struct scmi_xfer { int transfer_id; @@ -139,6 +140,7 @@ struct scmi_xfer { struct scmi_msg rx; struct completion done; struct completion *async_done; + void *extra_data; }; void scmi_xfer_put(const struct scmi_handle *h, struct scmi_xfer *xfer); -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev From: Igor Skalkin <igor.skalkin@opensynergy.com> The virtio transport in this patch series can be simplified by using the scmi_xfer tx/rx buffers for data exchange with the virtio device, and for saving the message state. But the virtio transport requires prepending a transport-specific header. Also, for data exchange using virtqueues, the tx and rx buffers should not overlap. After the previous patch, this is the second and final step to enable the virtio transport to use the scmi_xfer buffers for data exchange. Add an optional op through which the transport can allocate the tx/rx buffers along with room for the prepended transport-specific headers. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 3 +++ drivers/firmware/arm_scmi/driver.c | 21 +++++++++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index c998ec29018e..ae5db602e45d 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -208,6 +208,7 @@ struct scmi_chan_info { * @get_max_msg: Optional callback to provide max_msg dynamically * @max_msg: Maximum number of messages for the channel type (tx or rx) * that can be pending simultaneously in the system + * @xfer_init_buffers: Callback to initialize buffers for scmi_xfer * @send_message: Callback to send a message * @mark_txdone: Callback to mark tx as done * @fetch_response: Callback to fetch response @@ -222,6 +223,8 @@ struct scmi_transport_ops { int (*chan_free)(int id, void *p, void *data); int (*get_max_msg)(bool tx, struct scmi_chan_info *base_cinfo, int *max_msg); + int (*xfer_init_buffers)(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer, int max_msg_size); int (*send_message)(struct scmi_chan_info *cinfo, struct scmi_xfer *xfer); void (*mark_txdone)(struct scmi_chan_info *cinfo, int ret); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index e2faa526dfce..d7ad5a77b9dc 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -640,12 +640,21 @@ static int __scmi_xfer_info_init(struct scmi_info *sinfo, /* Pre-initialize the buffer pointer to pre-allocated buffers */ for (i = 0, xfer = info->xfer_block; i < info->max_msg; i++, xfer++) { - xfer->rx.buf = devm_kcalloc(dev, sizeof(u8), desc->max_msg_size, - GFP_KERNEL); - if (!xfer->rx.buf) - return -ENOMEM; - - xfer->tx.buf = xfer->rx.buf; + if (desc->ops->xfer_init_buffers) { + int ret = desc->ops->xfer_init_buffers( + base_cinfo, xfer, desc->max_msg_size); + + if (ret) + return ret; + } else { + xfer->rx.buf = devm_kcalloc(dev, sizeof(u8), + desc->max_msg_size, + GFP_KERNEL); + if (!xfer->rx.buf) + return -ENOMEM; + + xfer->tx.buf = xfer->rx.buf; + } init_completion(&xfer->done); } -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> The virtio transport in this patch series can be simplified by using the scmi_xfer tx/rx buffers for data exchange with the virtio device, and for saving the message state. But the virtio transport requires prepending a transport-specific header. Also, for data exchange using virtqueues, the tx and rx buffers should not overlap. After the previous patch, this is the second and final step to enable the virtio transport to use the scmi_xfer buffers for data exchange. Add an optional op through which the transport can allocate the tx/rx buffers along with room for the prepended transport-specific headers. Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 3 +++ drivers/firmware/arm_scmi/driver.c | 21 +++++++++++++++------ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index c998ec29018e..ae5db602e45d 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -208,6 +208,7 @@ struct scmi_chan_info { * @get_max_msg: Optional callback to provide max_msg dynamically * @max_msg: Maximum number of messages for the channel type (tx or rx) * that can be pending simultaneously in the system + * @xfer_init_buffers: Callback to initialize buffers for scmi_xfer * @send_message: Callback to send a message * @mark_txdone: Callback to mark tx as done * @fetch_response: Callback to fetch response @@ -222,6 +223,8 @@ struct scmi_transport_ops { int (*chan_free)(int id, void *p, void *data); int (*get_max_msg)(bool tx, struct scmi_chan_info *base_cinfo, int *max_msg); + int (*xfer_init_buffers)(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer, int max_msg_size); int (*send_message)(struct scmi_chan_info *cinfo, struct scmi_xfer *xfer); void (*mark_txdone)(struct scmi_chan_info *cinfo, int ret); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index e2faa526dfce..d7ad5a77b9dc 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -640,12 +640,21 @@ static int __scmi_xfer_info_init(struct scmi_info *sinfo, /* Pre-initialize the buffer pointer to pre-allocated buffers */ for (i = 0, xfer = info->xfer_block; i < info->max_msg; i++, xfer++) { - xfer->rx.buf = devm_kcalloc(dev, sizeof(u8), desc->max_msg_size, - GFP_KERNEL); - if (!xfer->rx.buf) - return -ENOMEM; - - xfer->tx.buf = xfer->rx.buf; + if (desc->ops->xfer_init_buffers) { + int ret = desc->ops->xfer_init_buffers( + base_cinfo, xfer, desc->max_msg_size); + + if (ret) + return ret; + } else { + xfer->rx.buf = devm_kcalloc(dev, sizeof(u8), + desc->max_msg_size, + GFP_KERNEL); + if (!xfer->rx.buf) + return -ENOMEM; + + xfer->tx.buf = xfer->rx.buf; + } init_completion(&xfer->done); } -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 06/10] firmware: arm_scmi: Add optional link_supplier() transport op 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, igor.skalkin, mikhail.golubev, anton.yakovlev For the scmi-virtio transport, it might not be possible to refer to the proper virtio device at device tree build time. Therefore, add an op which will allow scmi-virtio to dynamically link to the proper virtio device during probe. Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 2 ++ drivers/firmware/arm_scmi/driver.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index ae5db602e45d..2f55ac71555a 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -202,6 +202,7 @@ struct scmi_chan_info { /** * struct scmi_transport_ops - Structure representing a SCMI transport ops * + * @link_supplier: Optional callback to add link to a supplier device * @chan_available: Callback to check if channel is available or not * @chan_setup: Callback to allocate and setup a channel * @chan_free: Callback to free a channel @@ -217,6 +218,7 @@ struct scmi_chan_info { * @poll_done: Callback to poll transfer status */ struct scmi_transport_ops { + int (*link_supplier)(struct device *dev); bool (*chan_available)(struct device *dev, int idx); int (*chan_setup)(struct scmi_chan_info *cinfo, struct device *dev, bool tx); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index d7ad5a77b9dc..dedc9b3f3e97 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -826,6 +826,12 @@ static int scmi_probe(struct platform_device *pdev) handle->dev = info->dev; handle->version = &info->version; + if (desc->ops->link_supplier) { + ret = desc->ops->link_supplier(dev); + if (ret) + return ret; + } + ret = scmi_txrx_setup(info, dev, SCMI_PROTOCOL_BASE); if (ret) return ret; -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 06/10] firmware: arm_scmi: Add optional link_supplier() transport op @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, igor.skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee For the scmi-virtio transport, it might not be possible to refer to the proper virtio device at device tree build time. Therefore, add an op which will allow scmi-virtio to dynamically link to the proper virtio device during probe. Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 2 ++ drivers/firmware/arm_scmi/driver.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index ae5db602e45d..2f55ac71555a 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -202,6 +202,7 @@ struct scmi_chan_info { /** * struct scmi_transport_ops - Structure representing a SCMI transport ops * + * @link_supplier: Optional callback to add link to a supplier device * @chan_available: Callback to check if channel is available or not * @chan_setup: Callback to allocate and setup a channel * @chan_free: Callback to free a channel @@ -217,6 +218,7 @@ struct scmi_chan_info { * @poll_done: Callback to poll transfer status */ struct scmi_transport_ops { + int (*link_supplier)(struct device *dev); bool (*chan_available)(struct device *dev, int idx); int (*chan_setup)(struct scmi_chan_info *cinfo, struct device *dev, bool tx); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index d7ad5a77b9dc..dedc9b3f3e97 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -826,6 +826,12 @@ static int scmi_probe(struct platform_device *pdev) handle->dev = info->dev; handle->version = &info->version; + if (desc->ops->link_supplier) { + ret = desc->ops->link_supplier(dev); + if (ret) + return ret; + } + ret = scmi_txrx_setup(info, dev, SCMI_PROTOCOL_BASE); if (ret) return ret; -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 07/10] firmware: arm_scmi: Add per-device transport private info 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, igor.skalkin, mikhail.golubev, anton.yakovlev The scmi-virtio transport will link a supplier device to the arm-scmi device in the link_supplier() op. The transport should then save a pointer to the linked device. To enable this, add a transport private info to the scmi_info. (The scmi_info is already reachable through the arm-scmi device driver_data.) Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 2 ++ drivers/firmware/arm_scmi/driver.c | 35 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 2f55ac71555a..ec9fd7fce3c7 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -262,6 +262,8 @@ extern const struct scmi_desc scmi_mailbox_desc; extern const struct scmi_desc scmi_smc_desc; #endif +int scmi_set_transport_info(struct device *dev, void *transport_info); +void *scmi_get_transport_info(struct device *dev); void scmi_rx_callback(struct scmi_chan_info *cinfo, u32 msg_hdr); void scmi_free_channel(struct scmi_chan_info *cinfo, struct idr *idr, int id); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index dedc9b3f3e97..244141e45e88 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -84,6 +84,7 @@ struct scmi_xfers_info { * @rx_idr: IDR object to map protocol id to Rx channel info pointer * @protocols_imp: List of protocols implemented, currently maximum of * MAX_PROTOCOLS_IMP elements allocated by the base protocol + * @transport_info: Transport private info * @node: List head * @users: Number of users of this instance */ @@ -97,6 +98,7 @@ struct scmi_info { struct idr tx_idr; struct idr rx_idr; u8 *protocols_imp; + void *transport_info; struct list_head node; int users; }; @@ -315,6 +317,39 @@ void scmi_rx_callback(struct scmi_chan_info *cinfo, u32 msg_hdr) } } +/** + * scmi_set_transport_info() - Set transport private info + * + * @dev: SCMI instance device + * @transport_info: transport private info + * + * Return: 0 on success, otherwise error. + */ +int scmi_set_transport_info(struct device *dev, void *transport_info) +{ + struct scmi_info *info = dev_get_drvdata(dev); + + if (!info) + return -EBADR; + + info->transport_info = transport_info; + return 0; +} + +/** + * scmi_get_transport_info() - Get transport private info + * + * @dev: SCMI instance device + * + * Return: transport private info on success, otherwise NULL. + */ +void *scmi_get_transport_info(struct device *dev) +{ + struct scmi_info *info = dev_get_drvdata(dev); + + return info ? info->transport_info : NULL; +} + /** * scmi_xfer_put() - Release a transmit message * -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 07/10] firmware: arm_scmi: Add per-device transport private info @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, igor.skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee The scmi-virtio transport will link a supplier device to the arm-scmi device in the link_supplier() op. The transport should then save a pointer to the linked device. To enable this, add a transport private info to the scmi_info. (The scmi_info is already reachable through the arm-scmi device driver_data.) Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/firmware/arm_scmi/common.h | 2 ++ drivers/firmware/arm_scmi/driver.c | 35 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 2f55ac71555a..ec9fd7fce3c7 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -262,6 +262,8 @@ extern const struct scmi_desc scmi_mailbox_desc; extern const struct scmi_desc scmi_smc_desc; #endif +int scmi_set_transport_info(struct device *dev, void *transport_info); +void *scmi_get_transport_info(struct device *dev); void scmi_rx_callback(struct scmi_chan_info *cinfo, u32 msg_hdr); void scmi_free_channel(struct scmi_chan_info *cinfo, struct idr *idr, int id); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index dedc9b3f3e97..244141e45e88 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -84,6 +84,7 @@ struct scmi_xfers_info { * @rx_idr: IDR object to map protocol id to Rx channel info pointer * @protocols_imp: List of protocols implemented, currently maximum of * MAX_PROTOCOLS_IMP elements allocated by the base protocol + * @transport_info: Transport private info * @node: List head * @users: Number of users of this instance */ @@ -97,6 +98,7 @@ struct scmi_info { struct idr tx_idr; struct idr rx_idr; u8 *protocols_imp; + void *transport_info; struct list_head node; int users; }; @@ -315,6 +317,39 @@ void scmi_rx_callback(struct scmi_chan_info *cinfo, u32 msg_hdr) } } +/** + * scmi_set_transport_info() - Set transport private info + * + * @dev: SCMI instance device + * @transport_info: transport private info + * + * Return: 0 on success, otherwise error. + */ +int scmi_set_transport_info(struct device *dev, void *transport_info) +{ + struct scmi_info *info = dev_get_drvdata(dev); + + if (!info) + return -EBADR; + + info->transport_info = transport_info; + return 0; +} + +/** + * scmi_get_transport_info() - Get transport private info + * + * @dev: SCMI instance device + * + * Return: transport private info on success, otherwise NULL. + */ +void *scmi_get_transport_info(struct device *dev) +{ + struct scmi_info *info = dev_get_drvdata(dev); + + return info ? info->transport_info : NULL; +} + /** * scmi_xfer_put() - Release a transmit message * -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 08/10] firmware: arm_scmi: Add is_scmi_protocol_device() 2020-11-05 21:21 ` Peter Hilber @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, igor.skalkin, mikhail.golubev, anton.yakovlev The scmi-virtio transport driver will need to distinguish SCMI protocol devices from the SCMI instance device in the chan_setup() and chan_free() ops. Add this internal helper to be able to distinguish the two. Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/firmware/arm_scmi/bus.c | 5 +++++ drivers/firmware/arm_scmi/common.h | 2 ++ 2 files changed, 7 insertions(+) diff --git a/drivers/firmware/arm_scmi/bus.c b/drivers/firmware/arm_scmi/bus.c index 1377ec76a45d..4f19faafb2c5 100644 --- a/drivers/firmware/arm_scmi/bus.c +++ b/drivers/firmware/arm_scmi/bus.c @@ -108,6 +108,11 @@ static struct bus_type scmi_bus_type = { .remove = scmi_dev_remove, }; +bool is_scmi_protocol_device(struct device *dev) +{ + return dev->bus == &scmi_bus_type; +} + int scmi_driver_register(struct scmi_driver *driver, struct module *owner, const char *mod_name) { diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index ec9fd7fce3c7..13c9ac176b23 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -158,6 +158,8 @@ int scmi_version_get(const struct scmi_handle *h, u8 protocol, u32 *version); void scmi_setup_protocol_implemented(const struct scmi_handle *handle, u8 *prot_imp); +bool is_scmi_protocol_device(struct device *dev); + int scmi_base_protocol_init(struct scmi_handle *h); int __init scmi_bus_init(void); -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 08/10] firmware: arm_scmi: Add is_scmi_protocol_device() @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, igor.skalkin, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee The scmi-virtio transport driver will need to distinguish SCMI protocol devices from the SCMI instance device in the chan_setup() and chan_free() ops. Add this internal helper to be able to distinguish the two. Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/firmware/arm_scmi/bus.c | 5 +++++ drivers/firmware/arm_scmi/common.h | 2 ++ 2 files changed, 7 insertions(+) diff --git a/drivers/firmware/arm_scmi/bus.c b/drivers/firmware/arm_scmi/bus.c index 1377ec76a45d..4f19faafb2c5 100644 --- a/drivers/firmware/arm_scmi/bus.c +++ b/drivers/firmware/arm_scmi/bus.c @@ -108,6 +108,11 @@ static struct bus_type scmi_bus_type = { .remove = scmi_dev_remove, }; +bool is_scmi_protocol_device(struct device *dev) +{ + return dev->bus == &scmi_bus_type; +} + int scmi_driver_register(struct scmi_driver *driver, struct module *owner, const char *mod_name) { diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index ec9fd7fce3c7..13c9ac176b23 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -158,6 +158,8 @@ int scmi_version_get(const struct scmi_handle *h, u8 protocol, u32 *version); void scmi_setup_protocol_implemented(const struct scmi_handle *handle, u8 *prot_imp); +bool is_scmi_protocol_device(struct device *dev); + int scmi_base_protocol_init(struct scmi_handle *h); int __init scmi_bus_init(void); -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 09/10] dt-bindings: arm: Add virtio transport for SCMI 2020-11-05 21:21 ` Peter Hilber (?) @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Michael S. Tsirkin, Jason Wang From: Igor Skalkin <igor.skalkin@opensynergy.com> Document the properties for arm,scmi-virtio compatible nodes. The backing virtio SCMI device is described in patch [1]. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- .../devicetree/bindings/arm/arm,scmi.txt | 35 +++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/Documentation/devicetree/bindings/arm/arm,scmi.txt b/Documentation/devicetree/bindings/arm/arm,scmi.txt index 55deb68230eb..6ded49d82773 100644 --- a/Documentation/devicetree/bindings/arm/arm,scmi.txt +++ b/Documentation/devicetree/bindings/arm/arm,scmi.txt @@ -13,6 +13,9 @@ the device tree. Required properties: The scmi node with the following properties shall be under the /firmware/ node. +Some properties are specific to a transport type. + +shmem-based transports (mailbox, smc/hvc): - compatible : shall be "arm,scmi" or "arm,scmi-smc" for smc/hvc transports - mboxes: List of phandle and mailbox channel specifiers. It should contain @@ -21,6 +24,15 @@ The scmi node with the following properties shall be under the /firmware/ node. supported. - shmem : List of phandle pointing to the shared memory(SHM) area as per generic mailbox client binding. + +Virtio transport: + +- compatible : shall be "arm,scmi-virtio". + +The virtio transport only supports a single device. + +Additional required properties: + - #address-cells : should be '1' if the device has sub-nodes, maps to protocol identifier for a given sub-node. - #size-cells : should be '0' as 'reg' property doesn't have any size @@ -42,7 +54,8 @@ Each protocol supported shall have a sub-node with corresponding compatible as described in the following sections. If the platform supports dedicated communication channel for a particular protocol, the 3 properties namely: mboxes, mbox-names and shmem shall be present in the sub-node corresponding -to that protocol. +to that protocol. The virtio transport does not support dedicated communication +channels. Clock/Performance bindings for the clocks/OPPs based on SCMI Message Protocol ------------------------------------------------------------ @@ -106,7 +119,8 @@ Required sub-node properties: [4] Documentation/devicetree/bindings/sram/sram.yaml [5] Documentation/devicetree/bindings/reset/reset.txt -Example: +Example (mailbox transport): +---------------------------- sram@50000000 { compatible = "mmio-sram"; @@ -195,3 +209,20 @@ thermal-zones { ... }; }; + +Example (virtio transport): +--------------------------- + +virtio_mmio@4b001000 { + compatible = "virtio,mmio"; + ... +}; + +firmware { + ... + scmi { + compatible = "arm,scmi-virtio"; + ... + +The rest is similar to the mailbox transport example, when omitting the +mailbox/shmem-specific properties. -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [virtio-dev] [RFC PATCH v2 09/10] dt-bindings: arm: Add virtio transport for SCMI @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Michael S. Tsirkin, Jason Wang From: Igor Skalkin <igor.skalkin@opensynergy.com> Document the properties for arm,scmi-virtio compatible nodes. The backing virtio SCMI device is described in patch [1]. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- .../devicetree/bindings/arm/arm,scmi.txt | 35 +++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/Documentation/devicetree/bindings/arm/arm,scmi.txt b/Documentation/devicetree/bindings/arm/arm,scmi.txt index 55deb68230eb..6ded49d82773 100644 --- a/Documentation/devicetree/bindings/arm/arm,scmi.txt +++ b/Documentation/devicetree/bindings/arm/arm,scmi.txt @@ -13,6 +13,9 @@ the device tree. Required properties: The scmi node with the following properties shall be under the /firmware/ node. +Some properties are specific to a transport type. + +shmem-based transports (mailbox, smc/hvc): - compatible : shall be "arm,scmi" or "arm,scmi-smc" for smc/hvc transports - mboxes: List of phandle and mailbox channel specifiers. It should contain @@ -21,6 +24,15 @@ The scmi node with the following properties shall be under the /firmware/ node. supported. - shmem : List of phandle pointing to the shared memory(SHM) area as per generic mailbox client binding. + +Virtio transport: + +- compatible : shall be "arm,scmi-virtio". + +The virtio transport only supports a single device. + +Additional required properties: + - #address-cells : should be '1' if the device has sub-nodes, maps to protocol identifier for a given sub-node. - #size-cells : should be '0' as 'reg' property doesn't have any size @@ -42,7 +54,8 @@ Each protocol supported shall have a sub-node with corresponding compatible as described in the following sections. If the platform supports dedicated communication channel for a particular protocol, the 3 properties namely: mboxes, mbox-names and shmem shall be present in the sub-node corresponding -to that protocol. +to that protocol. The virtio transport does not support dedicated communication +channels. Clock/Performance bindings for the clocks/OPPs based on SCMI Message Protocol ------------------------------------------------------------ @@ -106,7 +119,8 @@ Required sub-node properties: [4] Documentation/devicetree/bindings/sram/sram.yaml [5] Documentation/devicetree/bindings/reset/reset.txt -Example: +Example (mailbox transport): +---------------------------- sram@50000000 { compatible = "mmio-sram"; @@ -195,3 +209,20 @@ thermal-zones { ... }; }; + +Example (virtio transport): +--------------------------- + +virtio_mmio@4b001000 { + compatible = "virtio,mmio"; + ... +}; + +firmware { + ... + scmi { + compatible = "arm,scmi-virtio"; + ... + +The rest is similar to the mailbox transport example, when omitting the +mailbox/shmem-specific properties. -- 2.25.1 --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 09/10] dt-bindings: arm: Add virtio transport for SCMI @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, Michael S. Tsirkin, Jason Wang, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> Document the properties for arm,scmi-virtio compatible nodes. The backing virtio SCMI device is described in patch [1]. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- .../devicetree/bindings/arm/arm,scmi.txt | 35 +++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/Documentation/devicetree/bindings/arm/arm,scmi.txt b/Documentation/devicetree/bindings/arm/arm,scmi.txt index 55deb68230eb..6ded49d82773 100644 --- a/Documentation/devicetree/bindings/arm/arm,scmi.txt +++ b/Documentation/devicetree/bindings/arm/arm,scmi.txt @@ -13,6 +13,9 @@ the device tree. Required properties: The scmi node with the following properties shall be under the /firmware/ node. +Some properties are specific to a transport type. + +shmem-based transports (mailbox, smc/hvc): - compatible : shall be "arm,scmi" or "arm,scmi-smc" for smc/hvc transports - mboxes: List of phandle and mailbox channel specifiers. It should contain @@ -21,6 +24,15 @@ The scmi node with the following properties shall be under the /firmware/ node. supported. - shmem : List of phandle pointing to the shared memory(SHM) area as per generic mailbox client binding. + +Virtio transport: + +- compatible : shall be "arm,scmi-virtio". + +The virtio transport only supports a single device. + +Additional required properties: + - #address-cells : should be '1' if the device has sub-nodes, maps to protocol identifier for a given sub-node. - #size-cells : should be '0' as 'reg' property doesn't have any size @@ -42,7 +54,8 @@ Each protocol supported shall have a sub-node with corresponding compatible as described in the following sections. If the platform supports dedicated communication channel for a particular protocol, the 3 properties namely: mboxes, mbox-names and shmem shall be present in the sub-node corresponding -to that protocol. +to that protocol. The virtio transport does not support dedicated communication +channels. Clock/Performance bindings for the clocks/OPPs based on SCMI Message Protocol ------------------------------------------------------------ @@ -106,7 +119,8 @@ Required sub-node properties: [4] Documentation/devicetree/bindings/sram/sram.yaml [5] Documentation/devicetree/bindings/reset/reset.txt -Example: +Example (mailbox transport): +---------------------------- sram@50000000 { compatible = "mmio-sram"; @@ -195,3 +209,20 @@ thermal-zones { ... }; }; + +Example (virtio transport): +--------------------------- + +virtio_mmio@4b001000 { + compatible = "virtio,mmio"; + ... +}; + +firmware { + ... + scmi { + compatible = "arm,scmi-virtio"; + ... + +The rest is similar to the mailbox transport example, when omitting the +mailbox/shmem-specific properties. -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport 2020-11-05 21:21 ` Peter Hilber (?) @ 2020-11-05 21:21 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Michael S. Tsirkin, Jason Wang From: Igor Skalkin <igor.skalkin@opensynergy.com> This transport enables accessing an SCMI platform as a virtio device. Implement an SCMI virtio driver according to the virtio SCMI device spec patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device [2]. The virtio transport has one tx channel (virtio cmdq, A2P channel) and at most one rx channel (virtio eventq, P2A channel). The following feature bit defined in [1] is not implemented: VIRTIO_SCMI_F_SHARED_MEMORY. After the preparatory patches, implement the virtio transport as paraphrased: Only support a single arm-scmi device (which is consistent with the SCMI spec). Call scmi-virtio init from arm-scmi module init. During the arm-scmi probing, link to the first probed scmi-virtio device. Defer arm-scmi probing if no scmi-virtio device is bound yet. Use the scmi_xfer tx/rx buffers for data exchange with the virtio device in order to avoid redundant maintenance of additional buffers. Allocate the buffers in the SCMI transport, and prepend room for a small header used by the virtio transport to the tx/rx buffers. For simplicity, restrict the number of messages which can be pending simultaneously according to the virtqueue capacity. (The virtqueue sizes are negotiated with the virtio device.) As soon as rx channel message buffers are allocated or have been read out by the arm-scmi driver, feed them to the virtio device. Since some virtio devices may not have the short response time exhibited by SCMI platforms using other transports, set a generous response timeout. Limitations: Do not adjust the other SCMI timeouts for delayed response and polling for now, since these timeouts are only relevant in special cases which are not yet deemed relevant for this transport. To do (as discussed in the cover letter): - Avoid re-use of buffers still being used by the virtio device on timeouts. - Avoid race conditions on receiving messages during/after channel free on driver probe failure or remove. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html [2] https://www.oasis-open.org/committees/ballot.php?id=3496 Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- MAINTAINERS | 1 + drivers/firmware/Kconfig | 12 +- drivers/firmware/arm_scmi/Makefile | 1 + drivers/firmware/arm_scmi/common.h | 14 + drivers/firmware/arm_scmi/driver.c | 11 + drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_scmi.h | 41 +++ 8 files changed, 573 insertions(+), 1 deletion(-) create mode 100644 drivers/firmware/arm_scmi/virtio.c create mode 100644 include/uapi/linux/virtio_scmi.h diff --git a/MAINTAINERS b/MAINTAINERS index deaafb617361..8df73d6ddfc1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c F: drivers/reset/reset-scmi.c F: include/linux/sc[mp]i_protocol.h F: include/trace/events/scmi.h +F: include/uapi/linux/virtio_scmi.h SYSTEM RESET/SHUTDOWN DRIVERS M: Sebastian Reichel <sre@kernel.org> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index bdde51adb267..c4bdd84f7405 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -9,7 +9,7 @@ menu "Firmware Drivers" config ARM_SCMI_PROTOCOL tristate "ARM System Control and Management Interface (SCMI) Message Protocol" depends on ARM || ARM64 || COMPILE_TEST - depends on ARM_SCMI_HAVE_SHMEM + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI help ARM System Control and Management Interface (SCMI) protocol is a set of operating system-independent software interfaces that are @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM This declares whether a shared memory based transport for SCMI is available. +config VIRTIO_SCMI + bool "Virtio transport for SCMI" + default n + depends on VIRTIO + help + This enables the virtio based transport for SCMI. + + If you want to use the ARM SCMI protocol between the virtio guest and + a host providing a virtio SCMI device, answer Y. + config ARM_SCMI_POWER_DOMAIN tristate "SCMI power domain driver" depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile index 3cc7fa40a464..25caea5e1969 100644 --- a/drivers/firmware/arm_scmi/Makefile +++ b/drivers/firmware/arm_scmi/Makefile @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o scmi-transport-$(CONFIG_MAILBOX) += mailbox.o scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ $(scmi-transport-y) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 13c9ac176b23..b46dfe84e78b 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); int __init scmi_bus_init(void); void __exit scmi_bus_exit(void); +#ifdef CONFIG_VIRTIO_SCMI +int __init virtio_scmi_init(void); +void __exit virtio_scmi_exit(void); +#else +static inline int __init virtio_scmi_init(void) +{ + return 0; +} +#define virtio_scmi_exit() do { } while (0) +#endif + #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ int __init scmi_##func##_register(void); \ void __exit scmi_##func##_unregister(void) @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; #ifdef CONFIG_HAVE_ARM_SMCCC extern const struct scmi_desc scmi_smc_desc; #endif +#ifdef CONFIG_VIRTIO_SCMI +extern const struct scmi_desc scmi_virtio_desc; +#endif int scmi_set_transport_info(struct device *dev, void *transport_info); void *scmi_get_transport_info(struct device *dev); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 244141e45e88..923ba526e829 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { #endif #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, +#endif +#ifdef CONFIG_VIRTIO_SCMI + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, #endif { /* Sentinel */ }, }; @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { static int __init scmi_driver_init(void) { + int ret; + scmi_bus_init(); + ret = virtio_scmi_init(); + if (ret) + return ret; + scmi_clock_register(); scmi_perf_register(); scmi_power_register(); @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) scmi_sensors_unregister(); scmi_system_unregister(); + virtio_scmi_exit(); + platform_driver_unregister(&scmi_driver); } module_exit(scmi_driver_exit); diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c new file mode 100644 index 000000000000..f70aa72f34f1 --- /dev/null +++ b/drivers/firmware/arm_scmi/virtio.c @@ -0,0 +1,493 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Virtio Transport driver for Arm System Control and Management Interface + * (SCMI). + * + * Copyright (C) 2020 OpenSynergy. + */ + +/** + * DOC: Theory of Operation + * + * The scmi-virtio transport implements a driver for the virtio SCMI device + * proposed in virtio spec patch v5[1]. + * + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx + * channel (virtio eventq, P2A channel). Each channel is implemented through a + * virtqueue. Access to each virtqueue is protected by a spinlock. + * + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with + * the virtio device to avoid maintenance of additional buffers. + * + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html + */ + +#include <linux/errno.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/virtio.h> +#include <linux/virtio_config.h> +#include <uapi/linux/virtio_ids.h> +#include <uapi/linux/virtio_scmi.h> + +#include "common.h" + +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ +#define DESCR_PER_TX_MSG 2 + +struct scmi_vio_channel { + spinlock_t lock; + struct virtqueue *vqueue; + struct scmi_chan_info *cinfo; + u8 is_rx; +}; + +union virtio_scmi_input { + __virtio32 hdr; + struct virtio_scmi_response response; + struct virtio_scmi_notification notification; +}; + +struct scmi_vio_msg { + struct virtio_scmi_request *request; + union virtio_scmi_input *input; + u8 completed; +}; + +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, + struct scmi_xfer *xfer) +{ + struct scatterlist sg_in; + struct scmi_vio_msg *msg = xfer->extra_data; + int rc; + + msg->completed = false; + + sg_init_one(&sg_in, msg->input, + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); + + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); + if (rc) + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); + else + virtqueue_kick(vioch->vqueue); + + return rc; +} + +static void scmi_vio_complete_cb(struct virtqueue *vqueue) +{ + struct scmi_vio_channel *vioch = vqueue->priv; + unsigned long iflags; + unsigned int length; + + spin_lock_irqsave(&vioch->lock, iflags); + + do { + struct scmi_xfer *xfer; + + virtqueue_disable_cb(vqueue); + + while ((xfer = virtqueue_get_buf(vqueue, &length))) { + struct scmi_vio_msg *msg = xfer->extra_data; + u32 msg_hdr = + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); + + if (!vioch->is_rx) { /* tx queue response */ + msg->completed = true; + xfer->rx.len = + length - sizeof(msg->input->response); + if (!xfer->hdr.poll_completion) + scmi_rx_callback(vioch->cinfo, msg_hdr); + continue; + } + + /* rx queue - notification or delayed response */ + switch (msg_type) { + case MSG_TYPE_NOTIFICATION: + xfer->rx.len = length - + sizeof(msg->input->notification); + xfer->rx.buf = msg->input->notification.data; + break; + case MSG_TYPE_DELAYED_RESP: + xfer->rx.len = + length - sizeof(msg->input->response); + xfer->rx.buf = msg->input->response.data; + break; + default: + dev_warn_once(vioch->cinfo->dev, + "rx: unknown message_type %d\n", + msg_type); + scmi_vio_populate_vq_rx(vioch, xfer); + continue; + } + + scmi_rx_callback(vioch->cinfo, msg_hdr); + scmi_vio_populate_vq_rx(vioch, xfer); + } + + if (unlikely(virtqueue_is_broken(vqueue))) + break; + } while (!virtqueue_enable_cb(vqueue)); + + spin_unlock_irqrestore(&vioch->lock, iflags); +} + +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; + +static vq_callback_t *scmi_vio_complete_callbacks[] = { + scmi_vio_complete_cb, + scmi_vio_complete_cb +}; + +static int scmi_vio_match_any_dev(struct device *dev, const void *data) +{ + (void)dev; + (void)data; + + return 1; +} + +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ + +static int virtio_link_supplier(struct device *dev) +{ + struct device *vdev = driver_find_device( + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); + + if (!vdev) { + dev_notice_once( + dev, + "Deferring probe after not finding a bound scmi-virtio device\n"); + return -EPROBE_DEFER; + } + + /* + * Add plain device link for completeness. It might have no effect + * beyond sysfs. + */ + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { + put_device(vdev); + dev_err(dev, "Adding link to supplier virtio device failed\n"); + return -ECANCELED; + } + + put_device(vdev); + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); +} + +static bool virtio_chan_available(struct device *dev, int idx) +{ + struct virtio_device *vdev; + struct scmi_vio_channel **vioch; + + /* scmi-virtio doesn't support per-protocol channels */ + if (is_scmi_protocol_device(dev)) + return false; + + vdev = scmi_get_transport_info(dev); + if (!vdev) + return false; + + vioch = vdev->priv; + if (!vioch) + return false; + + return vioch[idx] && vioch[idx]->vqueue; +} + +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, + bool tx) +{ + struct virtio_device *vdev; + struct scmi_vio_channel **vioch; + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; + + /* scmi-virtio doesn't support per-protocol channels */ + if (is_scmi_protocol_device(dev)) + return -1; + + vdev = scmi_get_transport_info(dev); + if (!vdev) + return -1; + + vioch = vdev->priv; + if (!vioch) { + dev_err(dev, "Data from scmi-virtio probe not found\n"); + return -1; + } + cinfo->transport_info = vioch[vioch_index]; + vioch[vioch_index]->cinfo = cinfo; + + return 0; +} + +static int virtio_chan_free(int id, void *p, void *data) +{ + struct scmi_chan_info *cinfo = p; + struct scmi_vio_channel *vioch = cinfo->transport_info; + + if (vioch) { + cinfo->transport_info = NULL; + kfree(vioch); + } + + scmi_free_channel(cinfo, data, id); + return 0; +} + +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, + int *max_msg) +{ + struct scmi_vio_channel *vioch = base_cinfo->transport_info; + + *max_msg = virtqueue_get_vring_size(vioch->vqueue); + + /* Tx messages need multiple descriptors. */ + if (tx) + *max_msg /= DESCR_PER_TX_MSG; + + if (*max_msg > MSG_TOKEN_MAX) { + dev_notice( + base_cinfo->dev, + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", + MSG_TOKEN_MAX, *max_msg); + *max_msg = MSG_TOKEN_MAX; + } + + return 0; +} + +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer, int max_msg_size) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg; + + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); + if (!msg) + return -ENOMEM; + + xfer->extra_data = msg; + + if (vioch->is_rx) { + int rc; + unsigned long iflags; + + msg->input = devm_kzalloc(cinfo->dev, + sizeof(*msg->input) + max_msg_size, + GFP_KERNEL); + if (!msg->input) + return -ENOMEM; + + /* + * xfer->rx.buf will be set to notification or delayed response + * specific values in the receive callback, according to the + * type of the received message. + */ + + spin_lock_irqsave(&vioch->lock, iflags); + rc = scmi_vio_populate_vq_rx(vioch, xfer); + spin_unlock_irqrestore(&vioch->lock, iflags); + if (rc) + return rc; + } else { + msg->request = + devm_kzalloc(cinfo->dev, + sizeof(*msg->request) + max_msg_size, + GFP_KERNEL); + if (!msg->request) + return -ENOMEM; + + xfer->tx.buf = msg->request->data; + + msg->input = devm_kzalloc( + cinfo->dev, sizeof(msg->input->response) + max_msg_size, + GFP_KERNEL); + if (!msg->input) + return -ENOMEM; + + xfer->rx.buf = msg->input->response.data; + } + + return 0; +} + +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) +{ + struct scatterlist sg_out; + struct scatterlist sg_in; + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; + struct scmi_vio_msg *msg = xfer->extra_data; + unsigned long iflags; + int rc; + + msg->completed = false; + + sg_init_one(&sg_out, msg->request, + sizeof(*msg->request) + xfer->tx.len); + sg_init_one(&sg_in, &msg->input->response, + sizeof(msg->input->response) + xfer->rx.len); + + spin_lock_irqsave(&vioch->lock, iflags); + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); + if (rc) + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); + else + virtqueue_kick(vioch->vqueue); + spin_unlock_irqrestore(&vioch->lock, iflags); + + return rc; +} + +static int virtio_send_message(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + uint32_t hdr; + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct virtio_device *vdev = vioch->vqueue->vdev; + struct scmi_vio_msg *msg = xfer->extra_data; + + hdr = pack_scmi_header(&xfer->hdr); + + msg->request->hdr = cpu_to_virtio32(vdev, hdr); + + return scmi_vio_send(vioch, xfer); +} + +static void virtio_fetch_response(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg = xfer->extra_data; + + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, + msg->input->response.status); +} + +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, + size_t max_len, struct scmi_xfer *xfer) +{ + (void)cinfo; + (void)max_len; + (void)xfer; +} + +static void dummy_clear_channel(struct scmi_chan_info *cinfo) +{ + (void)cinfo; +} + +static bool virtio_poll_done(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg = xfer->extra_data; + unsigned long iflags; + bool completed; + + spin_lock_irqsave(&vioch->lock, iflags); + completed = msg->completed; + spin_unlock_irqrestore(&vioch->lock, iflags); + + return completed; +} + +static const struct scmi_transport_ops scmi_virtio_ops = { + .link_supplier = virtio_link_supplier, + .chan_available = virtio_chan_available, + .chan_setup = virtio_chan_setup, + .chan_free = virtio_chan_free, + .get_max_msg = virtio_get_max_msg, + .send_message = virtio_send_message, + .fetch_response = virtio_fetch_response, + .fetch_notification = dummy_fetch_notification, + .clear_channel = dummy_clear_channel, + .poll_done = virtio_poll_done, + .xfer_init_buffers = virtio_xfer_init_buffers, +}; + +const struct scmi_desc scmi_virtio_desc = { + .ops = &scmi_virtio_ops, + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ + .max_msg = 0, /* overridden by virtio_get_max_msg() */ + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, +}; + +static int scmi_vio_probe(struct virtio_device *vdev) +{ + struct device *dev = &vdev->dev; + struct scmi_vio_channel **vioch; + bool have_vq_rx; + int vq_cnt; + int i; + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; + + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), + GFP_KERNEL); + if (!vioch) + return -ENOMEM; + + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; + + for (i = 0; i < vq_cnt; i++) { + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); + if (!vioch[i]) + return -ENOMEM; + } + + if (have_vq_rx) + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; + + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, + scmi_vio_vqueue_names, NULL)) { + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); + return -1; + } + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); + + for (i = 0; i < vq_cnt; i++) { + spin_lock_init(&vioch[i]->lock); + vioch[i]->vqueue = vqs[i]; + vioch[i]->vqueue->priv = vioch[i]; + } + + vdev->priv = vioch; + + virtio_device_ready(vdev); + + return 0; +} + +static unsigned int features[] = { + VIRTIO_SCMI_F_P2A_CHANNELS, +}; + +static const struct virtio_device_id id_table[] = { + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, + { 0 } +}; + +static struct virtio_driver virtio_scmi_driver = { + .driver.name = "scmi-virtio", + .driver.owner = THIS_MODULE, + .feature_table = features, + .feature_table_size = ARRAY_SIZE(features), + .id_table = id_table, + .probe = scmi_vio_probe, +}; + +int __init virtio_scmi_init(void) +{ + return register_virtio_driver(&virtio_scmi_driver); +} + +void __exit virtio_scmi_exit(void) +{ + unregister_virtio_driver(&virtio_scmi_driver); +} diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h index b052355ac7a3..57d233c02720 100644 --- a/include/uapi/linux/virtio_ids.h +++ b/include/uapi/linux/virtio_ids.h @@ -48,5 +48,6 @@ #define VIRTIO_ID_FS 26 /* virtio filesystem */ #define VIRTIO_ID_PMEM 27 /* virtio pmem */ #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ #endif /* _LINUX_VIRTIO_IDS_H */ diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h new file mode 100644 index 000000000000..9f21b3dbbfe2 --- /dev/null +++ b/include/uapi/linux/virtio_scmi.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* + * Copyright (C) 2020 OpenSynergy GmbH + */ + +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H +#define _UAPI_LINUX_VIRTIO_SCMI_H + +#include <linux/virtio_types.h> + +/* Feature bits */ + +/* Device implements some SCMI notifications, or delayed responses. */ +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 + +/* Device implements any SCMI statistics shared memory region */ +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 + +/* Virtqueues */ + +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ +#define VIRTIO_SCMI_VQ_MAX_CNT 2 + +struct virtio_scmi_request { + __virtio32 hdr; + __u8 data[]; +}; + +struct virtio_scmi_response { + __virtio32 hdr; + __virtio32 status; + __u8 data[]; +}; + +struct virtio_scmi_notification { + __virtio32 hdr; + __u8 data[]; +}; + +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ -- 2.25.1 ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [virtio-dev] [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: Igor Skalkin, Peter Hilber, Rob Herring, linux-kernel, sudeep.holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Michael S. Tsirkin, Jason Wang From: Igor Skalkin <igor.skalkin@opensynergy.com> This transport enables accessing an SCMI platform as a virtio device. Implement an SCMI virtio driver according to the virtio SCMI device spec patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device [2]. The virtio transport has one tx channel (virtio cmdq, A2P channel) and at most one rx channel (virtio eventq, P2A channel). The following feature bit defined in [1] is not implemented: VIRTIO_SCMI_F_SHARED_MEMORY. After the preparatory patches, implement the virtio transport as paraphrased: Only support a single arm-scmi device (which is consistent with the SCMI spec). Call scmi-virtio init from arm-scmi module init. During the arm-scmi probing, link to the first probed scmi-virtio device. Defer arm-scmi probing if no scmi-virtio device is bound yet. Use the scmi_xfer tx/rx buffers for data exchange with the virtio device in order to avoid redundant maintenance of additional buffers. Allocate the buffers in the SCMI transport, and prepend room for a small header used by the virtio transport to the tx/rx buffers. For simplicity, restrict the number of messages which can be pending simultaneously according to the virtqueue capacity. (The virtqueue sizes are negotiated with the virtio device.) As soon as rx channel message buffers are allocated or have been read out by the arm-scmi driver, feed them to the virtio device. Since some virtio devices may not have the short response time exhibited by SCMI platforms using other transports, set a generous response timeout. Limitations: Do not adjust the other SCMI timeouts for delayed response and polling for now, since these timeouts are only relevant in special cases which are not yet deemed relevant for this transport. To do (as discussed in the cover letter): - Avoid re-use of buffers still being used by the virtio device on timeouts. - Avoid race conditions on receiving messages during/after channel free on driver probe failure or remove. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html [2] https://www.oasis-open.org/committees/ballot.php?id=3496 Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- MAINTAINERS | 1 + drivers/firmware/Kconfig | 12 +- drivers/firmware/arm_scmi/Makefile | 1 + drivers/firmware/arm_scmi/common.h | 14 + drivers/firmware/arm_scmi/driver.c | 11 + drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_scmi.h | 41 +++ 8 files changed, 573 insertions(+), 1 deletion(-) create mode 100644 drivers/firmware/arm_scmi/virtio.c create mode 100644 include/uapi/linux/virtio_scmi.h diff --git a/MAINTAINERS b/MAINTAINERS index deaafb617361..8df73d6ddfc1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c F: drivers/reset/reset-scmi.c F: include/linux/sc[mp]i_protocol.h F: include/trace/events/scmi.h +F: include/uapi/linux/virtio_scmi.h SYSTEM RESET/SHUTDOWN DRIVERS M: Sebastian Reichel <sre@kernel.org> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index bdde51adb267..c4bdd84f7405 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -9,7 +9,7 @@ menu "Firmware Drivers" config ARM_SCMI_PROTOCOL tristate "ARM System Control and Management Interface (SCMI) Message Protocol" depends on ARM || ARM64 || COMPILE_TEST - depends on ARM_SCMI_HAVE_SHMEM + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI help ARM System Control and Management Interface (SCMI) protocol is a set of operating system-independent software interfaces that are @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM This declares whether a shared memory based transport for SCMI is available. +config VIRTIO_SCMI + bool "Virtio transport for SCMI" + default n + depends on VIRTIO + help + This enables the virtio based transport for SCMI. + + If you want to use the ARM SCMI protocol between the virtio guest and + a host providing a virtio SCMI device, answer Y. + config ARM_SCMI_POWER_DOMAIN tristate "SCMI power domain driver" depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile index 3cc7fa40a464..25caea5e1969 100644 --- a/drivers/firmware/arm_scmi/Makefile +++ b/drivers/firmware/arm_scmi/Makefile @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o scmi-transport-$(CONFIG_MAILBOX) += mailbox.o scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ $(scmi-transport-y) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 13c9ac176b23..b46dfe84e78b 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); int __init scmi_bus_init(void); void __exit scmi_bus_exit(void); +#ifdef CONFIG_VIRTIO_SCMI +int __init virtio_scmi_init(void); +void __exit virtio_scmi_exit(void); +#else +static inline int __init virtio_scmi_init(void) +{ + return 0; +} +#define virtio_scmi_exit() do { } while (0) +#endif + #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ int __init scmi_##func##_register(void); \ void __exit scmi_##func##_unregister(void) @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; #ifdef CONFIG_HAVE_ARM_SMCCC extern const struct scmi_desc scmi_smc_desc; #endif +#ifdef CONFIG_VIRTIO_SCMI +extern const struct scmi_desc scmi_virtio_desc; +#endif int scmi_set_transport_info(struct device *dev, void *transport_info); void *scmi_get_transport_info(struct device *dev); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 244141e45e88..923ba526e829 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { #endif #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, +#endif +#ifdef CONFIG_VIRTIO_SCMI + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, #endif { /* Sentinel */ }, }; @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { static int __init scmi_driver_init(void) { + int ret; + scmi_bus_init(); + ret = virtio_scmi_init(); + if (ret) + return ret; + scmi_clock_register(); scmi_perf_register(); scmi_power_register(); @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) scmi_sensors_unregister(); scmi_system_unregister(); + virtio_scmi_exit(); + platform_driver_unregister(&scmi_driver); } module_exit(scmi_driver_exit); diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c new file mode 100644 index 000000000000..f70aa72f34f1 --- /dev/null +++ b/drivers/firmware/arm_scmi/virtio.c @@ -0,0 +1,493 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Virtio Transport driver for Arm System Control and Management Interface + * (SCMI). + * + * Copyright (C) 2020 OpenSynergy. + */ + +/** + * DOC: Theory of Operation + * + * The scmi-virtio transport implements a driver for the virtio SCMI device + * proposed in virtio spec patch v5[1]. + * + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx + * channel (virtio eventq, P2A channel). Each channel is implemented through a + * virtqueue. Access to each virtqueue is protected by a spinlock. + * + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with + * the virtio device to avoid maintenance of additional buffers. + * + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html + */ + +#include <linux/errno.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/virtio.h> +#include <linux/virtio_config.h> +#include <uapi/linux/virtio_ids.h> +#include <uapi/linux/virtio_scmi.h> + +#include "common.h" + +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ +#define DESCR_PER_TX_MSG 2 + +struct scmi_vio_channel { + spinlock_t lock; + struct virtqueue *vqueue; + struct scmi_chan_info *cinfo; + u8 is_rx; +}; + +union virtio_scmi_input { + __virtio32 hdr; + struct virtio_scmi_response response; + struct virtio_scmi_notification notification; +}; + +struct scmi_vio_msg { + struct virtio_scmi_request *request; + union virtio_scmi_input *input; + u8 completed; +}; + +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, + struct scmi_xfer *xfer) +{ + struct scatterlist sg_in; + struct scmi_vio_msg *msg = xfer->extra_data; + int rc; + + msg->completed = false; + + sg_init_one(&sg_in, msg->input, + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); + + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); + if (rc) + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); + else + virtqueue_kick(vioch->vqueue); + + return rc; +} + +static void scmi_vio_complete_cb(struct virtqueue *vqueue) +{ + struct scmi_vio_channel *vioch = vqueue->priv; + unsigned long iflags; + unsigned int length; + + spin_lock_irqsave(&vioch->lock, iflags); + + do { + struct scmi_xfer *xfer; + + virtqueue_disable_cb(vqueue); + + while ((xfer = virtqueue_get_buf(vqueue, &length))) { + struct scmi_vio_msg *msg = xfer->extra_data; + u32 msg_hdr = + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); + + if (!vioch->is_rx) { /* tx queue response */ + msg->completed = true; + xfer->rx.len = + length - sizeof(msg->input->response); + if (!xfer->hdr.poll_completion) + scmi_rx_callback(vioch->cinfo, msg_hdr); + continue; + } + + /* rx queue - notification or delayed response */ + switch (msg_type) { + case MSG_TYPE_NOTIFICATION: + xfer->rx.len = length - + sizeof(msg->input->notification); + xfer->rx.buf = msg->input->notification.data; + break; + case MSG_TYPE_DELAYED_RESP: + xfer->rx.len = + length - sizeof(msg->input->response); + xfer->rx.buf = msg->input->response.data; + break; + default: + dev_warn_once(vioch->cinfo->dev, + "rx: unknown message_type %d\n", + msg_type); + scmi_vio_populate_vq_rx(vioch, xfer); + continue; + } + + scmi_rx_callback(vioch->cinfo, msg_hdr); + scmi_vio_populate_vq_rx(vioch, xfer); + } + + if (unlikely(virtqueue_is_broken(vqueue))) + break; + } while (!virtqueue_enable_cb(vqueue)); + + spin_unlock_irqrestore(&vioch->lock, iflags); +} + +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; + +static vq_callback_t *scmi_vio_complete_callbacks[] = { + scmi_vio_complete_cb, + scmi_vio_complete_cb +}; + +static int scmi_vio_match_any_dev(struct device *dev, const void *data) +{ + (void)dev; + (void)data; + + return 1; +} + +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ + +static int virtio_link_supplier(struct device *dev) +{ + struct device *vdev = driver_find_device( + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); + + if (!vdev) { + dev_notice_once( + dev, + "Deferring probe after not finding a bound scmi-virtio device\n"); + return -EPROBE_DEFER; + } + + /* + * Add plain device link for completeness. It might have no effect + * beyond sysfs. + */ + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { + put_device(vdev); + dev_err(dev, "Adding link to supplier virtio device failed\n"); + return -ECANCELED; + } + + put_device(vdev); + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); +} + +static bool virtio_chan_available(struct device *dev, int idx) +{ + struct virtio_device *vdev; + struct scmi_vio_channel **vioch; + + /* scmi-virtio doesn't support per-protocol channels */ + if (is_scmi_protocol_device(dev)) + return false; + + vdev = scmi_get_transport_info(dev); + if (!vdev) + return false; + + vioch = vdev->priv; + if (!vioch) + return false; + + return vioch[idx] && vioch[idx]->vqueue; +} + +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, + bool tx) +{ + struct virtio_device *vdev; + struct scmi_vio_channel **vioch; + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; + + /* scmi-virtio doesn't support per-protocol channels */ + if (is_scmi_protocol_device(dev)) + return -1; + + vdev = scmi_get_transport_info(dev); + if (!vdev) + return -1; + + vioch = vdev->priv; + if (!vioch) { + dev_err(dev, "Data from scmi-virtio probe not found\n"); + return -1; + } + cinfo->transport_info = vioch[vioch_index]; + vioch[vioch_index]->cinfo = cinfo; + + return 0; +} + +static int virtio_chan_free(int id, void *p, void *data) +{ + struct scmi_chan_info *cinfo = p; + struct scmi_vio_channel *vioch = cinfo->transport_info; + + if (vioch) { + cinfo->transport_info = NULL; + kfree(vioch); + } + + scmi_free_channel(cinfo, data, id); + return 0; +} + +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, + int *max_msg) +{ + struct scmi_vio_channel *vioch = base_cinfo->transport_info; + + *max_msg = virtqueue_get_vring_size(vioch->vqueue); + + /* Tx messages need multiple descriptors. */ + if (tx) + *max_msg /= DESCR_PER_TX_MSG; + + if (*max_msg > MSG_TOKEN_MAX) { + dev_notice( + base_cinfo->dev, + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", + MSG_TOKEN_MAX, *max_msg); + *max_msg = MSG_TOKEN_MAX; + } + + return 0; +} + +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer, int max_msg_size) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg; + + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); + if (!msg) + return -ENOMEM; + + xfer->extra_data = msg; + + if (vioch->is_rx) { + int rc; + unsigned long iflags; + + msg->input = devm_kzalloc(cinfo->dev, + sizeof(*msg->input) + max_msg_size, + GFP_KERNEL); + if (!msg->input) + return -ENOMEM; + + /* + * xfer->rx.buf will be set to notification or delayed response + * specific values in the receive callback, according to the + * type of the received message. + */ + + spin_lock_irqsave(&vioch->lock, iflags); + rc = scmi_vio_populate_vq_rx(vioch, xfer); + spin_unlock_irqrestore(&vioch->lock, iflags); + if (rc) + return rc; + } else { + msg->request = + devm_kzalloc(cinfo->dev, + sizeof(*msg->request) + max_msg_size, + GFP_KERNEL); + if (!msg->request) + return -ENOMEM; + + xfer->tx.buf = msg->request->data; + + msg->input = devm_kzalloc( + cinfo->dev, sizeof(msg->input->response) + max_msg_size, + GFP_KERNEL); + if (!msg->input) + return -ENOMEM; + + xfer->rx.buf = msg->input->response.data; + } + + return 0; +} + +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) +{ + struct scatterlist sg_out; + struct scatterlist sg_in; + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; + struct scmi_vio_msg *msg = xfer->extra_data; + unsigned long iflags; + int rc; + + msg->completed = false; + + sg_init_one(&sg_out, msg->request, + sizeof(*msg->request) + xfer->tx.len); + sg_init_one(&sg_in, &msg->input->response, + sizeof(msg->input->response) + xfer->rx.len); + + spin_lock_irqsave(&vioch->lock, iflags); + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); + if (rc) + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); + else + virtqueue_kick(vioch->vqueue); + spin_unlock_irqrestore(&vioch->lock, iflags); + + return rc; +} + +static int virtio_send_message(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + uint32_t hdr; + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct virtio_device *vdev = vioch->vqueue->vdev; + struct scmi_vio_msg *msg = xfer->extra_data; + + hdr = pack_scmi_header(&xfer->hdr); + + msg->request->hdr = cpu_to_virtio32(vdev, hdr); + + return scmi_vio_send(vioch, xfer); +} + +static void virtio_fetch_response(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg = xfer->extra_data; + + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, + msg->input->response.status); +} + +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, + size_t max_len, struct scmi_xfer *xfer) +{ + (void)cinfo; + (void)max_len; + (void)xfer; +} + +static void dummy_clear_channel(struct scmi_chan_info *cinfo) +{ + (void)cinfo; +} + +static bool virtio_poll_done(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg = xfer->extra_data; + unsigned long iflags; + bool completed; + + spin_lock_irqsave(&vioch->lock, iflags); + completed = msg->completed; + spin_unlock_irqrestore(&vioch->lock, iflags); + + return completed; +} + +static const struct scmi_transport_ops scmi_virtio_ops = { + .link_supplier = virtio_link_supplier, + .chan_available = virtio_chan_available, + .chan_setup = virtio_chan_setup, + .chan_free = virtio_chan_free, + .get_max_msg = virtio_get_max_msg, + .send_message = virtio_send_message, + .fetch_response = virtio_fetch_response, + .fetch_notification = dummy_fetch_notification, + .clear_channel = dummy_clear_channel, + .poll_done = virtio_poll_done, + .xfer_init_buffers = virtio_xfer_init_buffers, +}; + +const struct scmi_desc scmi_virtio_desc = { + .ops = &scmi_virtio_ops, + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ + .max_msg = 0, /* overridden by virtio_get_max_msg() */ + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, +}; + +static int scmi_vio_probe(struct virtio_device *vdev) +{ + struct device *dev = &vdev->dev; + struct scmi_vio_channel **vioch; + bool have_vq_rx; + int vq_cnt; + int i; + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; + + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), + GFP_KERNEL); + if (!vioch) + return -ENOMEM; + + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; + + for (i = 0; i < vq_cnt; i++) { + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); + if (!vioch[i]) + return -ENOMEM; + } + + if (have_vq_rx) + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; + + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, + scmi_vio_vqueue_names, NULL)) { + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); + return -1; + } + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); + + for (i = 0; i < vq_cnt; i++) { + spin_lock_init(&vioch[i]->lock); + vioch[i]->vqueue = vqs[i]; + vioch[i]->vqueue->priv = vioch[i]; + } + + vdev->priv = vioch; + + virtio_device_ready(vdev); + + return 0; +} + +static unsigned int features[] = { + VIRTIO_SCMI_F_P2A_CHANNELS, +}; + +static const struct virtio_device_id id_table[] = { + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, + { 0 } +}; + +static struct virtio_driver virtio_scmi_driver = { + .driver.name = "scmi-virtio", + .driver.owner = THIS_MODULE, + .feature_table = features, + .feature_table_size = ARRAY_SIZE(features), + .id_table = id_table, + .probe = scmi_vio_probe, +}; + +int __init virtio_scmi_init(void) +{ + return register_virtio_driver(&virtio_scmi_driver); +} + +void __exit virtio_scmi_exit(void) +{ + unregister_virtio_driver(&virtio_scmi_driver); +} diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h index b052355ac7a3..57d233c02720 100644 --- a/include/uapi/linux/virtio_ids.h +++ b/include/uapi/linux/virtio_ids.h @@ -48,5 +48,6 @@ #define VIRTIO_ID_FS 26 /* virtio filesystem */ #define VIRTIO_ID_PMEM 27 /* virtio pmem */ #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ #endif /* _LINUX_VIRTIO_IDS_H */ diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h new file mode 100644 index 000000000000..9f21b3dbbfe2 --- /dev/null +++ b/include/uapi/linux/virtio_scmi.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* + * Copyright (C) 2020 OpenSynergy GmbH + */ + +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H +#define _UAPI_LINUX_VIRTIO_SCMI_H + +#include <linux/virtio_types.h> + +/* Feature bits */ + +/* Device implements some SCMI notifications, or delayed responses. */ +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 + +/* Device implements any SCMI statistics shared memory region */ +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 + +/* Virtqueues */ + +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ +#define VIRTIO_SCMI_VQ_MAX_CNT 2 + +struct virtio_scmi_request { + __virtio32 hdr; + __u8 data[]; +}; + +struct virtio_scmi_response { + __virtio32 hdr; + __virtio32 status; + __u8 data[]; +}; + +struct virtio_scmi_notification { + __virtio32 hdr; + __u8 data[]; +}; + +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ -- 2.25.1 --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org ^ permalink raw reply related [flat|nested] 34+ messages in thread
* [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2020-11-05 21:21 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-05 21:21 UTC (permalink / raw) To: linux-arm-kernel, devicetree, virtualization, virtio-dev Cc: jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, Michael S. Tsirkin, Jason Wang, linux-kernel, Rob Herring, Peter Hilber, anton.yakovlev, sudeep.holla, alex.bennee From: Igor Skalkin <igor.skalkin@opensynergy.com> This transport enables accessing an SCMI platform as a virtio device. Implement an SCMI virtio driver according to the virtio SCMI device spec patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device [2]. The virtio transport has one tx channel (virtio cmdq, A2P channel) and at most one rx channel (virtio eventq, P2A channel). The following feature bit defined in [1] is not implemented: VIRTIO_SCMI_F_SHARED_MEMORY. After the preparatory patches, implement the virtio transport as paraphrased: Only support a single arm-scmi device (which is consistent with the SCMI spec). Call scmi-virtio init from arm-scmi module init. During the arm-scmi probing, link to the first probed scmi-virtio device. Defer arm-scmi probing if no scmi-virtio device is bound yet. Use the scmi_xfer tx/rx buffers for data exchange with the virtio device in order to avoid redundant maintenance of additional buffers. Allocate the buffers in the SCMI transport, and prepend room for a small header used by the virtio transport to the tx/rx buffers. For simplicity, restrict the number of messages which can be pending simultaneously according to the virtqueue capacity. (The virtqueue sizes are negotiated with the virtio device.) As soon as rx channel message buffers are allocated or have been read out by the arm-scmi driver, feed them to the virtio device. Since some virtio devices may not have the short response time exhibited by SCMI platforms using other transports, set a generous response timeout. Limitations: Do not adjust the other SCMI timeouts for delayed response and polling for now, since these timeouts are only relevant in special cases which are not yet deemed relevant for this transport. To do (as discussed in the cover letter): - Avoid re-use of buffers still being used by the virtio device on timeouts. - Avoid race conditions on receiving messages during/after channel free on driver probe failure or remove. [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html [2] https://www.oasis-open.org/committees/ballot.php?id=3496 Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> --- MAINTAINERS | 1 + drivers/firmware/Kconfig | 12 +- drivers/firmware/arm_scmi/Makefile | 1 + drivers/firmware/arm_scmi/common.h | 14 + drivers/firmware/arm_scmi/driver.c | 11 + drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ include/uapi/linux/virtio_ids.h | 1 + include/uapi/linux/virtio_scmi.h | 41 +++ 8 files changed, 573 insertions(+), 1 deletion(-) create mode 100644 drivers/firmware/arm_scmi/virtio.c create mode 100644 include/uapi/linux/virtio_scmi.h diff --git a/MAINTAINERS b/MAINTAINERS index deaafb617361..8df73d6ddfc1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c F: drivers/reset/reset-scmi.c F: include/linux/sc[mp]i_protocol.h F: include/trace/events/scmi.h +F: include/uapi/linux/virtio_scmi.h SYSTEM RESET/SHUTDOWN DRIVERS M: Sebastian Reichel <sre@kernel.org> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index bdde51adb267..c4bdd84f7405 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -9,7 +9,7 @@ menu "Firmware Drivers" config ARM_SCMI_PROTOCOL tristate "ARM System Control and Management Interface (SCMI) Message Protocol" depends on ARM || ARM64 || COMPILE_TEST - depends on ARM_SCMI_HAVE_SHMEM + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI help ARM System Control and Management Interface (SCMI) protocol is a set of operating system-independent software interfaces that are @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM This declares whether a shared memory based transport for SCMI is available. +config VIRTIO_SCMI + bool "Virtio transport for SCMI" + default n + depends on VIRTIO + help + This enables the virtio based transport for SCMI. + + If you want to use the ARM SCMI protocol between the virtio guest and + a host providing a virtio SCMI device, answer Y. + config ARM_SCMI_POWER_DOMAIN tristate "SCMI power domain driver" depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile index 3cc7fa40a464..25caea5e1969 100644 --- a/drivers/firmware/arm_scmi/Makefile +++ b/drivers/firmware/arm_scmi/Makefile @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o scmi-transport-$(CONFIG_MAILBOX) += mailbox.o scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ $(scmi-transport-y) diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h index 13c9ac176b23..b46dfe84e78b 100644 --- a/drivers/firmware/arm_scmi/common.h +++ b/drivers/firmware/arm_scmi/common.h @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); int __init scmi_bus_init(void); void __exit scmi_bus_exit(void); +#ifdef CONFIG_VIRTIO_SCMI +int __init virtio_scmi_init(void); +void __exit virtio_scmi_exit(void); +#else +static inline int __init virtio_scmi_init(void) +{ + return 0; +} +#define virtio_scmi_exit() do { } while (0) +#endif + #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ int __init scmi_##func##_register(void); \ void __exit scmi_##func##_unregister(void) @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; #ifdef CONFIG_HAVE_ARM_SMCCC extern const struct scmi_desc scmi_smc_desc; #endif +#ifdef CONFIG_VIRTIO_SCMI +extern const struct scmi_desc scmi_virtio_desc; +#endif int scmi_set_transport_info(struct device *dev, void *transport_info); void *scmi_get_transport_info(struct device *dev); diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c index 244141e45e88..923ba526e829 100644 --- a/drivers/firmware/arm_scmi/driver.c +++ b/drivers/firmware/arm_scmi/driver.c @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { #endif #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, +#endif +#ifdef CONFIG_VIRTIO_SCMI + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, #endif { /* Sentinel */ }, }; @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { static int __init scmi_driver_init(void) { + int ret; + scmi_bus_init(); + ret = virtio_scmi_init(); + if (ret) + return ret; + scmi_clock_register(); scmi_perf_register(); scmi_power_register(); @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) scmi_sensors_unregister(); scmi_system_unregister(); + virtio_scmi_exit(); + platform_driver_unregister(&scmi_driver); } module_exit(scmi_driver_exit); diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c new file mode 100644 index 000000000000..f70aa72f34f1 --- /dev/null +++ b/drivers/firmware/arm_scmi/virtio.c @@ -0,0 +1,493 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Virtio Transport driver for Arm System Control and Management Interface + * (SCMI). + * + * Copyright (C) 2020 OpenSynergy. + */ + +/** + * DOC: Theory of Operation + * + * The scmi-virtio transport implements a driver for the virtio SCMI device + * proposed in virtio spec patch v5[1]. + * + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx + * channel (virtio eventq, P2A channel). Each channel is implemented through a + * virtqueue. Access to each virtqueue is protected by a spinlock. + * + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with + * the virtio device to avoid maintenance of additional buffers. + * + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html + */ + +#include <linux/errno.h> +#include <linux/of.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/virtio.h> +#include <linux/virtio_config.h> +#include <uapi/linux/virtio_ids.h> +#include <uapi/linux/virtio_scmi.h> + +#include "common.h" + +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ +#define DESCR_PER_TX_MSG 2 + +struct scmi_vio_channel { + spinlock_t lock; + struct virtqueue *vqueue; + struct scmi_chan_info *cinfo; + u8 is_rx; +}; + +union virtio_scmi_input { + __virtio32 hdr; + struct virtio_scmi_response response; + struct virtio_scmi_notification notification; +}; + +struct scmi_vio_msg { + struct virtio_scmi_request *request; + union virtio_scmi_input *input; + u8 completed; +}; + +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, + struct scmi_xfer *xfer) +{ + struct scatterlist sg_in; + struct scmi_vio_msg *msg = xfer->extra_data; + int rc; + + msg->completed = false; + + sg_init_one(&sg_in, msg->input, + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); + + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); + if (rc) + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); + else + virtqueue_kick(vioch->vqueue); + + return rc; +} + +static void scmi_vio_complete_cb(struct virtqueue *vqueue) +{ + struct scmi_vio_channel *vioch = vqueue->priv; + unsigned long iflags; + unsigned int length; + + spin_lock_irqsave(&vioch->lock, iflags); + + do { + struct scmi_xfer *xfer; + + virtqueue_disable_cb(vqueue); + + while ((xfer = virtqueue_get_buf(vqueue, &length))) { + struct scmi_vio_msg *msg = xfer->extra_data; + u32 msg_hdr = + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); + + if (!vioch->is_rx) { /* tx queue response */ + msg->completed = true; + xfer->rx.len = + length - sizeof(msg->input->response); + if (!xfer->hdr.poll_completion) + scmi_rx_callback(vioch->cinfo, msg_hdr); + continue; + } + + /* rx queue - notification or delayed response */ + switch (msg_type) { + case MSG_TYPE_NOTIFICATION: + xfer->rx.len = length - + sizeof(msg->input->notification); + xfer->rx.buf = msg->input->notification.data; + break; + case MSG_TYPE_DELAYED_RESP: + xfer->rx.len = + length - sizeof(msg->input->response); + xfer->rx.buf = msg->input->response.data; + break; + default: + dev_warn_once(vioch->cinfo->dev, + "rx: unknown message_type %d\n", + msg_type); + scmi_vio_populate_vq_rx(vioch, xfer); + continue; + } + + scmi_rx_callback(vioch->cinfo, msg_hdr); + scmi_vio_populate_vq_rx(vioch, xfer); + } + + if (unlikely(virtqueue_is_broken(vqueue))) + break; + } while (!virtqueue_enable_cb(vqueue)); + + spin_unlock_irqrestore(&vioch->lock, iflags); +} + +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; + +static vq_callback_t *scmi_vio_complete_callbacks[] = { + scmi_vio_complete_cb, + scmi_vio_complete_cb +}; + +static int scmi_vio_match_any_dev(struct device *dev, const void *data) +{ + (void)dev; + (void)data; + + return 1; +} + +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ + +static int virtio_link_supplier(struct device *dev) +{ + struct device *vdev = driver_find_device( + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); + + if (!vdev) { + dev_notice_once( + dev, + "Deferring probe after not finding a bound scmi-virtio device\n"); + return -EPROBE_DEFER; + } + + /* + * Add plain device link for completeness. It might have no effect + * beyond sysfs. + */ + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { + put_device(vdev); + dev_err(dev, "Adding link to supplier virtio device failed\n"); + return -ECANCELED; + } + + put_device(vdev); + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); +} + +static bool virtio_chan_available(struct device *dev, int idx) +{ + struct virtio_device *vdev; + struct scmi_vio_channel **vioch; + + /* scmi-virtio doesn't support per-protocol channels */ + if (is_scmi_protocol_device(dev)) + return false; + + vdev = scmi_get_transport_info(dev); + if (!vdev) + return false; + + vioch = vdev->priv; + if (!vioch) + return false; + + return vioch[idx] && vioch[idx]->vqueue; +} + +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, + bool tx) +{ + struct virtio_device *vdev; + struct scmi_vio_channel **vioch; + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; + + /* scmi-virtio doesn't support per-protocol channels */ + if (is_scmi_protocol_device(dev)) + return -1; + + vdev = scmi_get_transport_info(dev); + if (!vdev) + return -1; + + vioch = vdev->priv; + if (!vioch) { + dev_err(dev, "Data from scmi-virtio probe not found\n"); + return -1; + } + cinfo->transport_info = vioch[vioch_index]; + vioch[vioch_index]->cinfo = cinfo; + + return 0; +} + +static int virtio_chan_free(int id, void *p, void *data) +{ + struct scmi_chan_info *cinfo = p; + struct scmi_vio_channel *vioch = cinfo->transport_info; + + if (vioch) { + cinfo->transport_info = NULL; + kfree(vioch); + } + + scmi_free_channel(cinfo, data, id); + return 0; +} + +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, + int *max_msg) +{ + struct scmi_vio_channel *vioch = base_cinfo->transport_info; + + *max_msg = virtqueue_get_vring_size(vioch->vqueue); + + /* Tx messages need multiple descriptors. */ + if (tx) + *max_msg /= DESCR_PER_TX_MSG; + + if (*max_msg > MSG_TOKEN_MAX) { + dev_notice( + base_cinfo->dev, + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", + MSG_TOKEN_MAX, *max_msg); + *max_msg = MSG_TOKEN_MAX; + } + + return 0; +} + +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer, int max_msg_size) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg; + + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); + if (!msg) + return -ENOMEM; + + xfer->extra_data = msg; + + if (vioch->is_rx) { + int rc; + unsigned long iflags; + + msg->input = devm_kzalloc(cinfo->dev, + sizeof(*msg->input) + max_msg_size, + GFP_KERNEL); + if (!msg->input) + return -ENOMEM; + + /* + * xfer->rx.buf will be set to notification or delayed response + * specific values in the receive callback, according to the + * type of the received message. + */ + + spin_lock_irqsave(&vioch->lock, iflags); + rc = scmi_vio_populate_vq_rx(vioch, xfer); + spin_unlock_irqrestore(&vioch->lock, iflags); + if (rc) + return rc; + } else { + msg->request = + devm_kzalloc(cinfo->dev, + sizeof(*msg->request) + max_msg_size, + GFP_KERNEL); + if (!msg->request) + return -ENOMEM; + + xfer->tx.buf = msg->request->data; + + msg->input = devm_kzalloc( + cinfo->dev, sizeof(msg->input->response) + max_msg_size, + GFP_KERNEL); + if (!msg->input) + return -ENOMEM; + + xfer->rx.buf = msg->input->response.data; + } + + return 0; +} + +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) +{ + struct scatterlist sg_out; + struct scatterlist sg_in; + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; + struct scmi_vio_msg *msg = xfer->extra_data; + unsigned long iflags; + int rc; + + msg->completed = false; + + sg_init_one(&sg_out, msg->request, + sizeof(*msg->request) + xfer->tx.len); + sg_init_one(&sg_in, &msg->input->response, + sizeof(msg->input->response) + xfer->rx.len); + + spin_lock_irqsave(&vioch->lock, iflags); + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); + if (rc) + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); + else + virtqueue_kick(vioch->vqueue); + spin_unlock_irqrestore(&vioch->lock, iflags); + + return rc; +} + +static int virtio_send_message(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + uint32_t hdr; + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct virtio_device *vdev = vioch->vqueue->vdev; + struct scmi_vio_msg *msg = xfer->extra_data; + + hdr = pack_scmi_header(&xfer->hdr); + + msg->request->hdr = cpu_to_virtio32(vdev, hdr); + + return scmi_vio_send(vioch, xfer); +} + +static void virtio_fetch_response(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg = xfer->extra_data; + + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, + msg->input->response.status); +} + +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, + size_t max_len, struct scmi_xfer *xfer) +{ + (void)cinfo; + (void)max_len; + (void)xfer; +} + +static void dummy_clear_channel(struct scmi_chan_info *cinfo) +{ + (void)cinfo; +} + +static bool virtio_poll_done(struct scmi_chan_info *cinfo, + struct scmi_xfer *xfer) +{ + struct scmi_vio_channel *vioch = cinfo->transport_info; + struct scmi_vio_msg *msg = xfer->extra_data; + unsigned long iflags; + bool completed; + + spin_lock_irqsave(&vioch->lock, iflags); + completed = msg->completed; + spin_unlock_irqrestore(&vioch->lock, iflags); + + return completed; +} + +static const struct scmi_transport_ops scmi_virtio_ops = { + .link_supplier = virtio_link_supplier, + .chan_available = virtio_chan_available, + .chan_setup = virtio_chan_setup, + .chan_free = virtio_chan_free, + .get_max_msg = virtio_get_max_msg, + .send_message = virtio_send_message, + .fetch_response = virtio_fetch_response, + .fetch_notification = dummy_fetch_notification, + .clear_channel = dummy_clear_channel, + .poll_done = virtio_poll_done, + .xfer_init_buffers = virtio_xfer_init_buffers, +}; + +const struct scmi_desc scmi_virtio_desc = { + .ops = &scmi_virtio_ops, + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ + .max_msg = 0, /* overridden by virtio_get_max_msg() */ + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, +}; + +static int scmi_vio_probe(struct virtio_device *vdev) +{ + struct device *dev = &vdev->dev; + struct scmi_vio_channel **vioch; + bool have_vq_rx; + int vq_cnt; + int i; + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; + + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), + GFP_KERNEL); + if (!vioch) + return -ENOMEM; + + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; + + for (i = 0; i < vq_cnt; i++) { + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); + if (!vioch[i]) + return -ENOMEM; + } + + if (have_vq_rx) + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; + + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, + scmi_vio_vqueue_names, NULL)) { + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); + return -1; + } + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); + + for (i = 0; i < vq_cnt; i++) { + spin_lock_init(&vioch[i]->lock); + vioch[i]->vqueue = vqs[i]; + vioch[i]->vqueue->priv = vioch[i]; + } + + vdev->priv = vioch; + + virtio_device_ready(vdev); + + return 0; +} + +static unsigned int features[] = { + VIRTIO_SCMI_F_P2A_CHANNELS, +}; + +static const struct virtio_device_id id_table[] = { + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, + { 0 } +}; + +static struct virtio_driver virtio_scmi_driver = { + .driver.name = "scmi-virtio", + .driver.owner = THIS_MODULE, + .feature_table = features, + .feature_table_size = ARRAY_SIZE(features), + .id_table = id_table, + .probe = scmi_vio_probe, +}; + +int __init virtio_scmi_init(void) +{ + return register_virtio_driver(&virtio_scmi_driver); +} + +void __exit virtio_scmi_exit(void) +{ + unregister_virtio_driver(&virtio_scmi_driver); +} diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h index b052355ac7a3..57d233c02720 100644 --- a/include/uapi/linux/virtio_ids.h +++ b/include/uapi/linux/virtio_ids.h @@ -48,5 +48,6 @@ #define VIRTIO_ID_FS 26 /* virtio filesystem */ #define VIRTIO_ID_PMEM 27 /* virtio pmem */ #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ #endif /* _LINUX_VIRTIO_IDS_H */ diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h new file mode 100644 index 000000000000..9f21b3dbbfe2 --- /dev/null +++ b/include/uapi/linux/virtio_scmi.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* + * Copyright (C) 2020 OpenSynergy GmbH + */ + +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H +#define _UAPI_LINUX_VIRTIO_SCMI_H + +#include <linux/virtio_types.h> + +/* Feature bits */ + +/* Device implements some SCMI notifications, or delayed responses. */ +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 + +/* Device implements any SCMI statistics shared memory region */ +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 + +/* Virtqueues */ + +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ +#define VIRTIO_SCMI_VQ_MAX_CNT 2 + +struct virtio_scmi_request { + __virtio32 hdr; + __u8 data[]; +}; + +struct virtio_scmi_response { + __virtio32 hdr; + __virtio32 status; + __u8 data[]; +}; + +struct virtio_scmi_notification { + __virtio32 hdr; + __u8 data[]; +}; + +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport 2020-11-05 21:21 ` Peter Hilber @ 2020-11-10 21:32 ` Cristian Marussi -1 siblings, 0 replies; 34+ messages in thread From: Cristian Marussi @ 2020-11-10 21:32 UTC (permalink / raw) To: Peter Hilber Cc: linux-arm-kernel, devicetree, virtualization, virtio-dev, jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, Michael S. Tsirkin, Jason Wang, linux-kernel, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee Hi Peter/Igor, I went through this series while trying to grasp a bit more of all the virtio inner workings and needs and I'll leave a few detailed comments down below. In short at first, I think I can say that there are a couple of places where I noticed you had to play all sort of tricks to fit into the current SCMI transport layer frameowrk or to avoid adding DT references. It's clear that we'll have to somehow extend/fix/abstract the SCMI transport layer (in my opinion), because I doubt that some of the tricks you played would be well received for upstream ever (...but it's worth noting it's not up to me saying the last...) So in these days I'm trying to play with this series and the SCMI stack to see if I can extend it in a more sensible way to fit some of the observations I make down below (now transport core layer is pretty much shmem/mailbox oriented)...still nothing to share anyway as of now. On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: > From: Igor Skalkin <igor.skalkin@opensynergy.com> > > This transport enables accessing an SCMI platform as a virtio device. > > Implement an SCMI virtio driver according to the virtio SCMI device spec > patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device > [2]. > > The virtio transport has one tx channel (virtio cmdq, A2P channel) and > at most one rx channel (virtio eventq, P2A channel). > > The following feature bit defined in [1] is not implemented: > VIRTIO_SCMI_F_SHARED_MEMORY. > > After the preparatory patches, implement the virtio transport as > paraphrased: > > Only support a single arm-scmi device (which is consistent with the SCMI > spec). Call scmi-virtio init from arm-scmi module init. During the > arm-scmi probing, link to the first probed scmi-virtio device. Defer > arm-scmi probing if no scmi-virtio device is bound yet. > > Use the scmi_xfer tx/rx buffers for data exchange with the virtio device > in order to avoid redundant maintenance of additional buffers. Allocate > the buffers in the SCMI transport, and prepend room for a small header > used by the virtio transport to the tx/rx buffers. > > For simplicity, restrict the number of messages which can be pending > simultaneously according to the virtqueue capacity. (The virtqueue sizes > are negotiated with the virtio device.) > > As soon as rx channel message buffers are allocated or have been read > out by the arm-scmi driver, feed them to the virtio device. > > Since some virtio devices may not have the short response time exhibited > by SCMI platforms using other transports, set a generous response > timeout. > > Limitations: > > Do not adjust the other SCMI timeouts for delayed response and polling > for now, since these timeouts are only relevant in special cases which > are not yet deemed relevant for this transport. > > To do (as discussed in the cover letter): > > - Avoid re-use of buffers still being used by the virtio device on > timeouts. > > - Avoid race conditions on receiving messages during/after channel free > on driver probe failure or remove. > > [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > [2] https://www.oasis-open.org/committees/ballot.php?id=3496 > > Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> > Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> > Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> > --- > MAINTAINERS | 1 + > drivers/firmware/Kconfig | 12 +- > drivers/firmware/arm_scmi/Makefile | 1 + > drivers/firmware/arm_scmi/common.h | 14 + > drivers/firmware/arm_scmi/driver.c | 11 + > drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ > include/uapi/linux/virtio_ids.h | 1 + > include/uapi/linux/virtio_scmi.h | 41 +++ > 8 files changed, 573 insertions(+), 1 deletion(-) > create mode 100644 drivers/firmware/arm_scmi/virtio.c > create mode 100644 include/uapi/linux/virtio_scmi.h > > diff --git a/MAINTAINERS b/MAINTAINERS > index deaafb617361..8df73d6ddfc1 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c > F: drivers/reset/reset-scmi.c > F: include/linux/sc[mp]i_protocol.h > F: include/trace/events/scmi.h > +F: include/uapi/linux/virtio_scmi.h > > SYSTEM RESET/SHUTDOWN DRIVERS > M: Sebastian Reichel <sre@kernel.org> > diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig > index bdde51adb267..c4bdd84f7405 100644 > --- a/drivers/firmware/Kconfig > +++ b/drivers/firmware/Kconfig > @@ -9,7 +9,7 @@ menu "Firmware Drivers" > config ARM_SCMI_PROTOCOL > tristate "ARM System Control and Management Interface (SCMI) Message Protocol" > depends on ARM || ARM64 || COMPILE_TEST > - depends on ARM_SCMI_HAVE_SHMEM > + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI > help > ARM System Control and Management Interface (SCMI) protocol is a > set of operating system-independent software interfaces that are > @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM > This declares whether a shared memory based transport for SCMI is > available. > > +config VIRTIO_SCMI > + bool "Virtio transport for SCMI" > + default n > + depends on VIRTIO > + help > + This enables the virtio based transport for SCMI. > + > + If you want to use the ARM SCMI protocol between the virtio guest and > + a host providing a virtio SCMI device, answer Y. > + > config ARM_SCMI_POWER_DOMAIN > tristate "SCMI power domain driver" > depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) > diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile > index 3cc7fa40a464..25caea5e1969 100644 > --- a/drivers/firmware/arm_scmi/Makefile > +++ b/drivers/firmware/arm_scmi/Makefile > @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o > scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o > scmi-transport-$(CONFIG_MAILBOX) += mailbox.o > scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o > +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o > scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o > scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ > $(scmi-transport-y) > diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h > index 13c9ac176b23..b46dfe84e78b 100644 > --- a/drivers/firmware/arm_scmi/common.h > +++ b/drivers/firmware/arm_scmi/common.h > @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); > int __init scmi_bus_init(void); > void __exit scmi_bus_exit(void); > > +#ifdef CONFIG_VIRTIO_SCMI > +int __init virtio_scmi_init(void); > +void __exit virtio_scmi_exit(void); > +#else > +static inline int __init virtio_scmi_init(void) > +{ > + return 0; > +} > +#define virtio_scmi_exit() do { } while (0) > +#endif > + > #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ > int __init scmi_##func##_register(void); \ > void __exit scmi_##func##_unregister(void) > @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; > #ifdef CONFIG_HAVE_ARM_SMCCC > extern const struct scmi_desc scmi_smc_desc; > #endif > +#ifdef CONFIG_VIRTIO_SCMI > +extern const struct scmi_desc scmi_virtio_desc; > +#endif > > int scmi_set_transport_info(struct device *dev, void *transport_info); > void *scmi_get_transport_info(struct device *dev); > diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c > index 244141e45e88..923ba526e829 100644 > --- a/drivers/firmware/arm_scmi/driver.c > +++ b/drivers/firmware/arm_scmi/driver.c > @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { > #endif > #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY > { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, > +#endif > +#ifdef CONFIG_VIRTIO_SCMI > + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, > #endif > { /* Sentinel */ }, > }; > @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { > > static int __init scmi_driver_init(void) > { > + int ret; > + > scmi_bus_init(); > > + ret = virtio_scmi_init(); > + if (ret) > + return ret; > + > scmi_clock_register(); > scmi_perf_register(); > scmi_power_register(); > @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) > scmi_sensors_unregister(); > scmi_system_unregister(); > > + virtio_scmi_exit(); > + These virtio init/exit functions which are called by the platform driver init code in fact introduce a very transport specific non-general piece of code in the common init path: this is one of the things I'd like to abstract better, so that any available transport can just register itself with the core in the same way and be initialized if needed in an uniform way, without having to extend the core driver init. (not saying that now it is not anyway already done for other matters....I'd like to remove those too in the future.) > platform_driver_unregister(&scmi_driver); > } > module_exit(scmi_driver_exit); > diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c > new file mode 100644 > index 000000000000..f70aa72f34f1 > --- /dev/null > +++ b/drivers/firmware/arm_scmi/virtio.c > @@ -0,0 +1,493 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Virtio Transport driver for Arm System Control and Management Interface > + * (SCMI). > + * > + * Copyright (C) 2020 OpenSynergy. > + */ > + > +/** > + * DOC: Theory of Operation > + * > + * The scmi-virtio transport implements a driver for the virtio SCMI device > + * proposed in virtio spec patch v5[1]. > + * > + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx > + * channel (virtio eventq, P2A channel). Each channel is implemented through a > + * virtqueue. Access to each virtqueue is protected by a spinlock. > + * > + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with > + * the virtio device to avoid maintenance of additional buffers. > + * > + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > + */ > + > +#include <linux/errno.h> > +#include <linux/of.h> > +#include <linux/of_platform.h> > +#include <linux/platform_device.h> > +#include <linux/module.h> > +#include <linux/slab.h> > +#include <linux/virtio.h> > +#include <linux/virtio_config.h> > +#include <uapi/linux/virtio_ids.h> > +#include <uapi/linux/virtio_scmi.h> > + > +#include "common.h" > + > +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ > +#define DESCR_PER_TX_MSG 2 > + > +struct scmi_vio_channel { > + spinlock_t lock; > + struct virtqueue *vqueue; > + struct scmi_chan_info *cinfo; > + u8 is_rx; > +}; > + > +union virtio_scmi_input { > + __virtio32 hdr; > + struct virtio_scmi_response response; > + struct virtio_scmi_notification notification; > +}; > + > +struct scmi_vio_msg { > + struct virtio_scmi_request *request; > + union virtio_scmi_input *input; > + u8 completed; > +}; > + > +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, > + struct scmi_xfer *xfer) > +{ > + struct scatterlist sg_in; > + struct scmi_vio_msg *msg = xfer->extra_data; > + int rc; > + > + msg->completed = false; > + > + sg_init_one(&sg_in, msg->input, > + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); > + > + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); > + if (rc) > + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > + else > + virtqueue_kick(vioch->vqueue); > + > + return rc; > +} > + > +static void scmi_vio_complete_cb(struct virtqueue *vqueue) > +{ > + struct scmi_vio_channel *vioch = vqueue->priv; > + unsigned long iflags; > + unsigned int length; > + > + spin_lock_irqsave(&vioch->lock, iflags); > + > + do { > + struct scmi_xfer *xfer; > + > + virtqueue_disable_cb(vqueue); > + > + while ((xfer = virtqueue_get_buf(vqueue, &length))) { > + struct scmi_vio_msg *msg = xfer->extra_data; > + u32 msg_hdr = > + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); > + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); > + > + if (!vioch->is_rx) { /* tx queue response */ > + msg->completed = true; > + xfer->rx.len = > + length - sizeof(msg->input->response); > + if (!xfer->hdr.poll_completion) > + scmi_rx_callback(vioch->cinfo, msg_hdr); > + continue; > + } > + So this is one of the places where SCMI transport seems not to accomodate so well the way virtio works as of now; it seems to me to be down to the fact that while virtio returns the message as a single chunk in one shot (since it carries the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads since the original shmem transport has the message size embedded in its header (which so has to be read at first), and so you have to stretch the usage of scmi_rx_callback introducing here some of its logic while leaving some other callbacks as empty dummies (like notifications callback): I think the transport should NOT have any knowledge of the structure/content of the stuff that it receives (like down below detecting which MSG_TYPE it is and acting accordingly). It clearly cannot work like it is now (so you have to play the tricks), but I think the answer is reviewing accordingly the SCMI transport layer, not trying to fooli it while polluting the transport layer with knowlegde of SCMI internals like message's structure. > + /* rx queue - notification or delayed response */ > + switch (msg_type) { > + case MSG_TYPE_NOTIFICATION: > + xfer->rx.len = length - > + sizeof(msg->input->notification); > + xfer->rx.buf = msg->input->notification.data; > + break; > + case MSG_TYPE_DELAYED_RESP: > + xfer->rx.len = > + length - sizeof(msg->input->response); > + xfer->rx.buf = msg->input->response.data; > + break; > + default: > + dev_warn_once(vioch->cinfo->dev, > + "rx: unknown message_type %d\n", > + msg_type); > + scmi_vio_populate_vq_rx(vioch, xfer); > + continue; > + } > + > + scmi_rx_callback(vioch->cinfo, msg_hdr); > + scmi_vio_populate_vq_rx(vioch, xfer); > + } > + > + if (unlikely(virtqueue_is_broken(vqueue))) > + break; > + } while (!virtqueue_enable_cb(vqueue)); > + > + spin_unlock_irqrestore(&vioch->lock, iflags); > +} > + > +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; > + > +static vq_callback_t *scmi_vio_complete_callbacks[] = { > + scmi_vio_complete_cb, > + scmi_vio_complete_cb > +}; > + > +static int scmi_vio_match_any_dev(struct device *dev, const void *data) > +{ > + (void)dev; > + (void)data; > + > + return 1; > +} > + > +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ > + > +static int virtio_link_supplier(struct device *dev) > +{ > + struct device *vdev = driver_find_device( > + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); > + > + if (!vdev) { > + dev_notice_once( > + dev, > + "Deferring probe after not finding a bound scmi-virtio device\n"); > + return -EPROBE_DEFER; > + } > + > + /* > + * Add plain device link for completeness. It might have no effect > + * beyond sysfs. > + */ > + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { > + put_device(vdev); > + dev_err(dev, "Adding link to supplier virtio device failed\n"); > + return -ECANCELED; > + } > + > + put_device(vdev); > + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); > +} > + I understand that here the attempt is to grab the virtio device upon which we are working and make it known to the core, while handling the case in which the MMIO transport is not ready with -EPROBE_DEFER, but I don't think such full-scale search with a dummy match function would be so well accepted upstream, and I don't understand really what's the point of the device_link_add() at the end...but maybe I'm missing something. Moreover looking at other virtio drivers (not that I'm an expert though...) it seems very much to me that this SCMI Virtio driver is sort of built/probed/initialized upside-down respect how the other virtio drivers are made, and this seems to be down again to some missing support in our SCMI transport layer (...plus the need to avoid any virtio-related DT addition). That's the other thing that would be in my plan to rectify and unify in the SCMI transport layer (...at least in my hopes :D) > +static bool virtio_chan_available(struct device *dev, int idx) > +{ > + struct virtio_device *vdev; > + struct scmi_vio_channel **vioch; > + > + /* scmi-virtio doesn't support per-protocol channels */ > + if (is_scmi_protocol_device(dev)) > + return false; > + > + vdev = scmi_get_transport_info(dev); > + if (!vdev) > + return false; > + > + vioch = vdev->priv; > + if (!vioch) > + return false; > + > + return vioch[idx] && vioch[idx]->vqueue; > +} > + > +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, > + bool tx) > +{ > + struct virtio_device *vdev; > + struct scmi_vio_channel **vioch; > + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; > + > + /* scmi-virtio doesn't support per-protocol channels */ > + if (is_scmi_protocol_device(dev)) > + return -1; > + > + vdev = scmi_get_transport_info(dev); > + if (!vdev) > + return -1; > + > + vioch = vdev->priv; > + if (!vioch) { > + dev_err(dev, "Data from scmi-virtio probe not found\n"); > + return -1; > + } > + cinfo->transport_info = vioch[vioch_index]; > + vioch[vioch_index]->cinfo = cinfo; > + > + return 0; > +} > + Same goes, from my point of view, for these two channel related callbacks that now have to perform some very odd interactions and checks with the SCMI core (at least in my opinion). Does the lack of support for multiple per-protocol channels derive from the lack of a way (now that the DT entry has gone) to determine the association between a channel and its user ? > +static int virtio_chan_free(int id, void *p, void *data) > +{ > + struct scmi_chan_info *cinfo = p; > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + > + if (vioch) { > + cinfo->transport_info = NULL; > + kfree(vioch); > + } > + > + scmi_free_channel(cinfo, data, id); > + return 0; > +} > + > +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, > + int *max_msg) > +{ > + struct scmi_vio_channel *vioch = base_cinfo->transport_info; > + > + *max_msg = virtqueue_get_vring_size(vioch->vqueue); > + > + /* Tx messages need multiple descriptors. */ > + if (tx) > + *max_msg /= DESCR_PER_TX_MSG; > + > + if (*max_msg > MSG_TOKEN_MAX) { > + dev_notice( > + base_cinfo->dev, > + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", > + MSG_TOKEN_MAX, *max_msg); > + *max_msg = MSG_TOKEN_MAX; > + } > + > + return 0; > +} > + > +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer, int max_msg_size) > +{ > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct scmi_vio_msg *msg; > + > + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); > + if (!msg) > + return -ENOMEM; > + > + xfer->extra_data = msg; > + > + if (vioch->is_rx) { > + int rc; > + unsigned long iflags; > + > + msg->input = devm_kzalloc(cinfo->dev, > + sizeof(*msg->input) + max_msg_size, > + GFP_KERNEL); > + if (!msg->input) > + return -ENOMEM; > + > + /* > + * xfer->rx.buf will be set to notification or delayed response > + * specific values in the receive callback, according to the > + * type of the received message. > + */ > + > + spin_lock_irqsave(&vioch->lock, iflags); > + rc = scmi_vio_populate_vq_rx(vioch, xfer); > + spin_unlock_irqrestore(&vioch->lock, iflags); > + if (rc) > + return rc; > + } else { > + msg->request = > + devm_kzalloc(cinfo->dev, > + sizeof(*msg->request) + max_msg_size, > + GFP_KERNEL); > + if (!msg->request) > + return -ENOMEM; > + > + xfer->tx.buf = msg->request->data; > + > + msg->input = devm_kzalloc( > + cinfo->dev, sizeof(msg->input->response) + max_msg_size, > + GFP_KERNEL); > + if (!msg->input) > + return -ENOMEM; > + > + xfer->rx.buf = msg->input->response.data; > + } > + > + return 0; > +} > + > +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) > +{ > + struct scatterlist sg_out; > + struct scatterlist sg_in; > + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; > + struct scmi_vio_msg *msg = xfer->extra_data; > + unsigned long iflags; > + int rc; > + > + msg->completed = false; > + > + sg_init_one(&sg_out, msg->request, > + sizeof(*msg->request) + xfer->tx.len); > + sg_init_one(&sg_in, &msg->input->response, > + sizeof(msg->input->response) + xfer->rx.len); > + > + spin_lock_irqsave(&vioch->lock, iflags); > + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); > + if (rc) > + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > + else > + virtqueue_kick(vioch->vqueue); > + spin_unlock_irqrestore(&vioch->lock, iflags); > + > + return rc; > +} > + > +static int virtio_send_message(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer) > +{ > + uint32_t hdr; > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct virtio_device *vdev = vioch->vqueue->vdev; > + struct scmi_vio_msg *msg = xfer->extra_data; > + > + hdr = pack_scmi_header(&xfer->hdr); > + > + msg->request->hdr = cpu_to_virtio32(vdev, hdr); > + > + return scmi_vio_send(vioch, xfer); > +} > + > +static void virtio_fetch_response(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer) > +{ > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct scmi_vio_msg *msg = xfer->extra_data; > + > + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, > + msg->input->response.status); > +} > + > +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, > + size_t max_len, struct scmi_xfer *xfer) > +{ > + (void)cinfo; > + (void)max_len; > + (void)xfer; > +} > + > +static void dummy_clear_channel(struct scmi_chan_info *cinfo) > +{ > + (void)cinfo; > +} > + > +static bool virtio_poll_done(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer) > +{ > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct scmi_vio_msg *msg = xfer->extra_data; > + unsigned long iflags; > + bool completed; > + > + spin_lock_irqsave(&vioch->lock, iflags); > + completed = msg->completed; > + spin_unlock_irqrestore(&vioch->lock, iflags); > + > + return completed; > +} > + > +static const struct scmi_transport_ops scmi_virtio_ops = { > + .link_supplier = virtio_link_supplier, > + .chan_available = virtio_chan_available, > + .chan_setup = virtio_chan_setup, > + .chan_free = virtio_chan_free, > + .get_max_msg = virtio_get_max_msg, > + .send_message = virtio_send_message, > + .fetch_response = virtio_fetch_response, > + .fetch_notification = dummy_fetch_notification, > + .clear_channel = dummy_clear_channel, > + .poll_done = virtio_poll_done, > + .xfer_init_buffers = virtio_xfer_init_buffers, > +}; > + > +const struct scmi_desc scmi_virtio_desc = { > + .ops = &scmi_virtio_ops, > + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ > + .max_msg = 0, /* overridden by virtio_get_max_msg() */ > + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, > +}; > + > +static int scmi_vio_probe(struct virtio_device *vdev) > +{ > + struct device *dev = &vdev->dev; > + struct scmi_vio_channel **vioch; > + bool have_vq_rx; > + int vq_cnt; > + int i; > + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; > + > + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), > + GFP_KERNEL); > + if (!vioch) > + return -ENOMEM; > + > + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); > + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; > + > + for (i = 0; i < vq_cnt; i++) { > + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); > + if (!vioch[i]) > + return -ENOMEM; > + } > + > + if (have_vq_rx) > + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; > + > + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, > + scmi_vio_vqueue_names, NULL)) { > + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); > + return -1; > + } > + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); > + > + for (i = 0; i < vq_cnt; i++) { > + spin_lock_init(&vioch[i]->lock); > + vioch[i]->vqueue = vqs[i]; > + vioch[i]->vqueue->priv = vioch[i]; > + } > + > + vdev->priv = vioch; > + As mentioned before, in other Virtio drivers this is the place where we should register somehow this transports/channels with the core (before marking the device ready), and may be the place where we can just share the core device for this tranport with the SCMI core without then having to play the above link_supplier chan_available/setup dance. But again I still have to fully tries to change the SCMI transport interface at this point. I've not really gone into all the details across all series because I see the above issues/limitations in the driver/SCMI_transport as sort of blocking. I'll let you now when I'll have something sensible on the SCMI core to share for better accomodate this transport. Thanks Cristian > + virtio_device_ready(vdev); > + > + return 0; > +} > + > +static unsigned int features[] = { > + VIRTIO_SCMI_F_P2A_CHANNELS, > +}; > + > +static const struct virtio_device_id id_table[] = { > + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, > + { 0 } > +}; > + > +static struct virtio_driver virtio_scmi_driver = { > + .driver.name = "scmi-virtio", > + .driver.owner = THIS_MODULE, > + .feature_table = features, > + .feature_table_size = ARRAY_SIZE(features), > + .id_table = id_table, > + .probe = scmi_vio_probe, > +}; > + > +int __init virtio_scmi_init(void) > +{ > + return register_virtio_driver(&virtio_scmi_driver); > +} > + > +void __exit virtio_scmi_exit(void) > +{ > + unregister_virtio_driver(&virtio_scmi_driver); > +} > diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h > index b052355ac7a3..57d233c02720 100644 > --- a/include/uapi/linux/virtio_ids.h > +++ b/include/uapi/linux/virtio_ids.h > @@ -48,5 +48,6 @@ > #define VIRTIO_ID_FS 26 /* virtio filesystem */ > #define VIRTIO_ID_PMEM 27 /* virtio pmem */ > #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ > +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ > > #endif /* _LINUX_VIRTIO_IDS_H */ > diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h > new file mode 100644 > index 000000000000..9f21b3dbbfe2 > --- /dev/null > +++ b/include/uapi/linux/virtio_scmi.h > @@ -0,0 +1,41 @@ > +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ > +/* > + * Copyright (C) 2020 OpenSynergy GmbH > + */ > + > +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H > +#define _UAPI_LINUX_VIRTIO_SCMI_H > + > +#include <linux/virtio_types.h> > + > +/* Feature bits */ > + > +/* Device implements some SCMI notifications, or delayed responses. */ > +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 > + > +/* Device implements any SCMI statistics shared memory region */ > +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 > + > +/* Virtqueues */ > + > +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ > +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ > +#define VIRTIO_SCMI_VQ_MAX_CNT 2 > + > +struct virtio_scmi_request { > + __virtio32 hdr; > + __u8 data[]; > +}; > + > +struct virtio_scmi_response { > + __virtio32 hdr; > + __virtio32 status; > + __u8 data[]; > +}; > + > +struct virtio_scmi_notification { > + __virtio32 hdr; > + __u8 data[]; > +}; > + > +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ > -- > 2.25.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2020-11-10 21:32 ` Cristian Marussi 0 siblings, 0 replies; 34+ messages in thread From: Cristian Marussi @ 2020-11-10 21:32 UTC (permalink / raw) To: Peter Hilber Cc: devicetree, mikhail.golubev, souvik.chakravarty, Igor Skalkin, jean-philippe, Jason Wang, Michael S. Tsirkin, linux-kernel, virtualization, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee, virtio-dev, linux-arm-kernel Hi Peter/Igor, I went through this series while trying to grasp a bit more of all the virtio inner workings and needs and I'll leave a few detailed comments down below. In short at first, I think I can say that there are a couple of places where I noticed you had to play all sort of tricks to fit into the current SCMI transport layer frameowrk or to avoid adding DT references. It's clear that we'll have to somehow extend/fix/abstract the SCMI transport layer (in my opinion), because I doubt that some of the tricks you played would be well received for upstream ever (...but it's worth noting it's not up to me saying the last...) So in these days I'm trying to play with this series and the SCMI stack to see if I can extend it in a more sensible way to fit some of the observations I make down below (now transport core layer is pretty much shmem/mailbox oriented)...still nothing to share anyway as of now. On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: > From: Igor Skalkin <igor.skalkin@opensynergy.com> > > This transport enables accessing an SCMI platform as a virtio device. > > Implement an SCMI virtio driver according to the virtio SCMI device spec > patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device > [2]. > > The virtio transport has one tx channel (virtio cmdq, A2P channel) and > at most one rx channel (virtio eventq, P2A channel). > > The following feature bit defined in [1] is not implemented: > VIRTIO_SCMI_F_SHARED_MEMORY. > > After the preparatory patches, implement the virtio transport as > paraphrased: > > Only support a single arm-scmi device (which is consistent with the SCMI > spec). Call scmi-virtio init from arm-scmi module init. During the > arm-scmi probing, link to the first probed scmi-virtio device. Defer > arm-scmi probing if no scmi-virtio device is bound yet. > > Use the scmi_xfer tx/rx buffers for data exchange with the virtio device > in order to avoid redundant maintenance of additional buffers. Allocate > the buffers in the SCMI transport, and prepend room for a small header > used by the virtio transport to the tx/rx buffers. > > For simplicity, restrict the number of messages which can be pending > simultaneously according to the virtqueue capacity. (The virtqueue sizes > are negotiated with the virtio device.) > > As soon as rx channel message buffers are allocated or have been read > out by the arm-scmi driver, feed them to the virtio device. > > Since some virtio devices may not have the short response time exhibited > by SCMI platforms using other transports, set a generous response > timeout. > > Limitations: > > Do not adjust the other SCMI timeouts for delayed response and polling > for now, since these timeouts are only relevant in special cases which > are not yet deemed relevant for this transport. > > To do (as discussed in the cover letter): > > - Avoid re-use of buffers still being used by the virtio device on > timeouts. > > - Avoid race conditions on receiving messages during/after channel free > on driver probe failure or remove. > > [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > [2] https://www.oasis-open.org/committees/ballot.php?id=3496 > > Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> > Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> > Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> > --- > MAINTAINERS | 1 + > drivers/firmware/Kconfig | 12 +- > drivers/firmware/arm_scmi/Makefile | 1 + > drivers/firmware/arm_scmi/common.h | 14 + > drivers/firmware/arm_scmi/driver.c | 11 + > drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ > include/uapi/linux/virtio_ids.h | 1 + > include/uapi/linux/virtio_scmi.h | 41 +++ > 8 files changed, 573 insertions(+), 1 deletion(-) > create mode 100644 drivers/firmware/arm_scmi/virtio.c > create mode 100644 include/uapi/linux/virtio_scmi.h > > diff --git a/MAINTAINERS b/MAINTAINERS > index deaafb617361..8df73d6ddfc1 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c > F: drivers/reset/reset-scmi.c > F: include/linux/sc[mp]i_protocol.h > F: include/trace/events/scmi.h > +F: include/uapi/linux/virtio_scmi.h > > SYSTEM RESET/SHUTDOWN DRIVERS > M: Sebastian Reichel <sre@kernel.org> > diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig > index bdde51adb267..c4bdd84f7405 100644 > --- a/drivers/firmware/Kconfig > +++ b/drivers/firmware/Kconfig > @@ -9,7 +9,7 @@ menu "Firmware Drivers" > config ARM_SCMI_PROTOCOL > tristate "ARM System Control and Management Interface (SCMI) Message Protocol" > depends on ARM || ARM64 || COMPILE_TEST > - depends on ARM_SCMI_HAVE_SHMEM > + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI > help > ARM System Control and Management Interface (SCMI) protocol is a > set of operating system-independent software interfaces that are > @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM > This declares whether a shared memory based transport for SCMI is > available. > > +config VIRTIO_SCMI > + bool "Virtio transport for SCMI" > + default n > + depends on VIRTIO > + help > + This enables the virtio based transport for SCMI. > + > + If you want to use the ARM SCMI protocol between the virtio guest and > + a host providing a virtio SCMI device, answer Y. > + > config ARM_SCMI_POWER_DOMAIN > tristate "SCMI power domain driver" > depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) > diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile > index 3cc7fa40a464..25caea5e1969 100644 > --- a/drivers/firmware/arm_scmi/Makefile > +++ b/drivers/firmware/arm_scmi/Makefile > @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o > scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o > scmi-transport-$(CONFIG_MAILBOX) += mailbox.o > scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o > +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o > scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o > scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ > $(scmi-transport-y) > diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h > index 13c9ac176b23..b46dfe84e78b 100644 > --- a/drivers/firmware/arm_scmi/common.h > +++ b/drivers/firmware/arm_scmi/common.h > @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); > int __init scmi_bus_init(void); > void __exit scmi_bus_exit(void); > > +#ifdef CONFIG_VIRTIO_SCMI > +int __init virtio_scmi_init(void); > +void __exit virtio_scmi_exit(void); > +#else > +static inline int __init virtio_scmi_init(void) > +{ > + return 0; > +} > +#define virtio_scmi_exit() do { } while (0) > +#endif > + > #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ > int __init scmi_##func##_register(void); \ > void __exit scmi_##func##_unregister(void) > @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; > #ifdef CONFIG_HAVE_ARM_SMCCC > extern const struct scmi_desc scmi_smc_desc; > #endif > +#ifdef CONFIG_VIRTIO_SCMI > +extern const struct scmi_desc scmi_virtio_desc; > +#endif > > int scmi_set_transport_info(struct device *dev, void *transport_info); > void *scmi_get_transport_info(struct device *dev); > diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c > index 244141e45e88..923ba526e829 100644 > --- a/drivers/firmware/arm_scmi/driver.c > +++ b/drivers/firmware/arm_scmi/driver.c > @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { > #endif > #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY > { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, > +#endif > +#ifdef CONFIG_VIRTIO_SCMI > + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, > #endif > { /* Sentinel */ }, > }; > @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { > > static int __init scmi_driver_init(void) > { > + int ret; > + > scmi_bus_init(); > > + ret = virtio_scmi_init(); > + if (ret) > + return ret; > + > scmi_clock_register(); > scmi_perf_register(); > scmi_power_register(); > @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) > scmi_sensors_unregister(); > scmi_system_unregister(); > > + virtio_scmi_exit(); > + These virtio init/exit functions which are called by the platform driver init code in fact introduce a very transport specific non-general piece of code in the common init path: this is one of the things I'd like to abstract better, so that any available transport can just register itself with the core in the same way and be initialized if needed in an uniform way, without having to extend the core driver init. (not saying that now it is not anyway already done for other matters....I'd like to remove those too in the future.) > platform_driver_unregister(&scmi_driver); > } > module_exit(scmi_driver_exit); > diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c > new file mode 100644 > index 000000000000..f70aa72f34f1 > --- /dev/null > +++ b/drivers/firmware/arm_scmi/virtio.c > @@ -0,0 +1,493 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Virtio Transport driver for Arm System Control and Management Interface > + * (SCMI). > + * > + * Copyright (C) 2020 OpenSynergy. > + */ > + > +/** > + * DOC: Theory of Operation > + * > + * The scmi-virtio transport implements a driver for the virtio SCMI device > + * proposed in virtio spec patch v5[1]. > + * > + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx > + * channel (virtio eventq, P2A channel). Each channel is implemented through a > + * virtqueue. Access to each virtqueue is protected by a spinlock. > + * > + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with > + * the virtio device to avoid maintenance of additional buffers. > + * > + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > + */ > + > +#include <linux/errno.h> > +#include <linux/of.h> > +#include <linux/of_platform.h> > +#include <linux/platform_device.h> > +#include <linux/module.h> > +#include <linux/slab.h> > +#include <linux/virtio.h> > +#include <linux/virtio_config.h> > +#include <uapi/linux/virtio_ids.h> > +#include <uapi/linux/virtio_scmi.h> > + > +#include "common.h" > + > +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ > +#define DESCR_PER_TX_MSG 2 > + > +struct scmi_vio_channel { > + spinlock_t lock; > + struct virtqueue *vqueue; > + struct scmi_chan_info *cinfo; > + u8 is_rx; > +}; > + > +union virtio_scmi_input { > + __virtio32 hdr; > + struct virtio_scmi_response response; > + struct virtio_scmi_notification notification; > +}; > + > +struct scmi_vio_msg { > + struct virtio_scmi_request *request; > + union virtio_scmi_input *input; > + u8 completed; > +}; > + > +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, > + struct scmi_xfer *xfer) > +{ > + struct scatterlist sg_in; > + struct scmi_vio_msg *msg = xfer->extra_data; > + int rc; > + > + msg->completed = false; > + > + sg_init_one(&sg_in, msg->input, > + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); > + > + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); > + if (rc) > + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > + else > + virtqueue_kick(vioch->vqueue); > + > + return rc; > +} > + > +static void scmi_vio_complete_cb(struct virtqueue *vqueue) > +{ > + struct scmi_vio_channel *vioch = vqueue->priv; > + unsigned long iflags; > + unsigned int length; > + > + spin_lock_irqsave(&vioch->lock, iflags); > + > + do { > + struct scmi_xfer *xfer; > + > + virtqueue_disable_cb(vqueue); > + > + while ((xfer = virtqueue_get_buf(vqueue, &length))) { > + struct scmi_vio_msg *msg = xfer->extra_data; > + u32 msg_hdr = > + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); > + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); > + > + if (!vioch->is_rx) { /* tx queue response */ > + msg->completed = true; > + xfer->rx.len = > + length - sizeof(msg->input->response); > + if (!xfer->hdr.poll_completion) > + scmi_rx_callback(vioch->cinfo, msg_hdr); > + continue; > + } > + So this is one of the places where SCMI transport seems not to accomodate so well the way virtio works as of now; it seems to me to be down to the fact that while virtio returns the message as a single chunk in one shot (since it carries the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads since the original shmem transport has the message size embedded in its header (which so has to be read at first), and so you have to stretch the usage of scmi_rx_callback introducing here some of its logic while leaving some other callbacks as empty dummies (like notifications callback): I think the transport should NOT have any knowledge of the structure/content of the stuff that it receives (like down below detecting which MSG_TYPE it is and acting accordingly). It clearly cannot work like it is now (so you have to play the tricks), but I think the answer is reviewing accordingly the SCMI transport layer, not trying to fooli it while polluting the transport layer with knowlegde of SCMI internals like message's structure. > + /* rx queue - notification or delayed response */ > + switch (msg_type) { > + case MSG_TYPE_NOTIFICATION: > + xfer->rx.len = length - > + sizeof(msg->input->notification); > + xfer->rx.buf = msg->input->notification.data; > + break; > + case MSG_TYPE_DELAYED_RESP: > + xfer->rx.len = > + length - sizeof(msg->input->response); > + xfer->rx.buf = msg->input->response.data; > + break; > + default: > + dev_warn_once(vioch->cinfo->dev, > + "rx: unknown message_type %d\n", > + msg_type); > + scmi_vio_populate_vq_rx(vioch, xfer); > + continue; > + } > + > + scmi_rx_callback(vioch->cinfo, msg_hdr); > + scmi_vio_populate_vq_rx(vioch, xfer); > + } > + > + if (unlikely(virtqueue_is_broken(vqueue))) > + break; > + } while (!virtqueue_enable_cb(vqueue)); > + > + spin_unlock_irqrestore(&vioch->lock, iflags); > +} > + > +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; > + > +static vq_callback_t *scmi_vio_complete_callbacks[] = { > + scmi_vio_complete_cb, > + scmi_vio_complete_cb > +}; > + > +static int scmi_vio_match_any_dev(struct device *dev, const void *data) > +{ > + (void)dev; > + (void)data; > + > + return 1; > +} > + > +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ > + > +static int virtio_link_supplier(struct device *dev) > +{ > + struct device *vdev = driver_find_device( > + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); > + > + if (!vdev) { > + dev_notice_once( > + dev, > + "Deferring probe after not finding a bound scmi-virtio device\n"); > + return -EPROBE_DEFER; > + } > + > + /* > + * Add plain device link for completeness. It might have no effect > + * beyond sysfs. > + */ > + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { > + put_device(vdev); > + dev_err(dev, "Adding link to supplier virtio device failed\n"); > + return -ECANCELED; > + } > + > + put_device(vdev); > + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); > +} > + I understand that here the attempt is to grab the virtio device upon which we are working and make it known to the core, while handling the case in which the MMIO transport is not ready with -EPROBE_DEFER, but I don't think such full-scale search with a dummy match function would be so well accepted upstream, and I don't understand really what's the point of the device_link_add() at the end...but maybe I'm missing something. Moreover looking at other virtio drivers (not that I'm an expert though...) it seems very much to me that this SCMI Virtio driver is sort of built/probed/initialized upside-down respect how the other virtio drivers are made, and this seems to be down again to some missing support in our SCMI transport layer (...plus the need to avoid any virtio-related DT addition). That's the other thing that would be in my plan to rectify and unify in the SCMI transport layer (...at least in my hopes :D) > +static bool virtio_chan_available(struct device *dev, int idx) > +{ > + struct virtio_device *vdev; > + struct scmi_vio_channel **vioch; > + > + /* scmi-virtio doesn't support per-protocol channels */ > + if (is_scmi_protocol_device(dev)) > + return false; > + > + vdev = scmi_get_transport_info(dev); > + if (!vdev) > + return false; > + > + vioch = vdev->priv; > + if (!vioch) > + return false; > + > + return vioch[idx] && vioch[idx]->vqueue; > +} > + > +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, > + bool tx) > +{ > + struct virtio_device *vdev; > + struct scmi_vio_channel **vioch; > + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; > + > + /* scmi-virtio doesn't support per-protocol channels */ > + if (is_scmi_protocol_device(dev)) > + return -1; > + > + vdev = scmi_get_transport_info(dev); > + if (!vdev) > + return -1; > + > + vioch = vdev->priv; > + if (!vioch) { > + dev_err(dev, "Data from scmi-virtio probe not found\n"); > + return -1; > + } > + cinfo->transport_info = vioch[vioch_index]; > + vioch[vioch_index]->cinfo = cinfo; > + > + return 0; > +} > + Same goes, from my point of view, for these two channel related callbacks that now have to perform some very odd interactions and checks with the SCMI core (at least in my opinion). Does the lack of support for multiple per-protocol channels derive from the lack of a way (now that the DT entry has gone) to determine the association between a channel and its user ? > +static int virtio_chan_free(int id, void *p, void *data) > +{ > + struct scmi_chan_info *cinfo = p; > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + > + if (vioch) { > + cinfo->transport_info = NULL; > + kfree(vioch); > + } > + > + scmi_free_channel(cinfo, data, id); > + return 0; > +} > + > +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, > + int *max_msg) > +{ > + struct scmi_vio_channel *vioch = base_cinfo->transport_info; > + > + *max_msg = virtqueue_get_vring_size(vioch->vqueue); > + > + /* Tx messages need multiple descriptors. */ > + if (tx) > + *max_msg /= DESCR_PER_TX_MSG; > + > + if (*max_msg > MSG_TOKEN_MAX) { > + dev_notice( > + base_cinfo->dev, > + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", > + MSG_TOKEN_MAX, *max_msg); > + *max_msg = MSG_TOKEN_MAX; > + } > + > + return 0; > +} > + > +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer, int max_msg_size) > +{ > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct scmi_vio_msg *msg; > + > + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); > + if (!msg) > + return -ENOMEM; > + > + xfer->extra_data = msg; > + > + if (vioch->is_rx) { > + int rc; > + unsigned long iflags; > + > + msg->input = devm_kzalloc(cinfo->dev, > + sizeof(*msg->input) + max_msg_size, > + GFP_KERNEL); > + if (!msg->input) > + return -ENOMEM; > + > + /* > + * xfer->rx.buf will be set to notification or delayed response > + * specific values in the receive callback, according to the > + * type of the received message. > + */ > + > + spin_lock_irqsave(&vioch->lock, iflags); > + rc = scmi_vio_populate_vq_rx(vioch, xfer); > + spin_unlock_irqrestore(&vioch->lock, iflags); > + if (rc) > + return rc; > + } else { > + msg->request = > + devm_kzalloc(cinfo->dev, > + sizeof(*msg->request) + max_msg_size, > + GFP_KERNEL); > + if (!msg->request) > + return -ENOMEM; > + > + xfer->tx.buf = msg->request->data; > + > + msg->input = devm_kzalloc( > + cinfo->dev, sizeof(msg->input->response) + max_msg_size, > + GFP_KERNEL); > + if (!msg->input) > + return -ENOMEM; > + > + xfer->rx.buf = msg->input->response.data; > + } > + > + return 0; > +} > + > +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) > +{ > + struct scatterlist sg_out; > + struct scatterlist sg_in; > + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; > + struct scmi_vio_msg *msg = xfer->extra_data; > + unsigned long iflags; > + int rc; > + > + msg->completed = false; > + > + sg_init_one(&sg_out, msg->request, > + sizeof(*msg->request) + xfer->tx.len); > + sg_init_one(&sg_in, &msg->input->response, > + sizeof(msg->input->response) + xfer->rx.len); > + > + spin_lock_irqsave(&vioch->lock, iflags); > + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); > + if (rc) > + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > + else > + virtqueue_kick(vioch->vqueue); > + spin_unlock_irqrestore(&vioch->lock, iflags); > + > + return rc; > +} > + > +static int virtio_send_message(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer) > +{ > + uint32_t hdr; > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct virtio_device *vdev = vioch->vqueue->vdev; > + struct scmi_vio_msg *msg = xfer->extra_data; > + > + hdr = pack_scmi_header(&xfer->hdr); > + > + msg->request->hdr = cpu_to_virtio32(vdev, hdr); > + > + return scmi_vio_send(vioch, xfer); > +} > + > +static void virtio_fetch_response(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer) > +{ > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct scmi_vio_msg *msg = xfer->extra_data; > + > + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, > + msg->input->response.status); > +} > + > +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, > + size_t max_len, struct scmi_xfer *xfer) > +{ > + (void)cinfo; > + (void)max_len; > + (void)xfer; > +} > + > +static void dummy_clear_channel(struct scmi_chan_info *cinfo) > +{ > + (void)cinfo; > +} > + > +static bool virtio_poll_done(struct scmi_chan_info *cinfo, > + struct scmi_xfer *xfer) > +{ > + struct scmi_vio_channel *vioch = cinfo->transport_info; > + struct scmi_vio_msg *msg = xfer->extra_data; > + unsigned long iflags; > + bool completed; > + > + spin_lock_irqsave(&vioch->lock, iflags); > + completed = msg->completed; > + spin_unlock_irqrestore(&vioch->lock, iflags); > + > + return completed; > +} > + > +static const struct scmi_transport_ops scmi_virtio_ops = { > + .link_supplier = virtio_link_supplier, > + .chan_available = virtio_chan_available, > + .chan_setup = virtio_chan_setup, > + .chan_free = virtio_chan_free, > + .get_max_msg = virtio_get_max_msg, > + .send_message = virtio_send_message, > + .fetch_response = virtio_fetch_response, > + .fetch_notification = dummy_fetch_notification, > + .clear_channel = dummy_clear_channel, > + .poll_done = virtio_poll_done, > + .xfer_init_buffers = virtio_xfer_init_buffers, > +}; > + > +const struct scmi_desc scmi_virtio_desc = { > + .ops = &scmi_virtio_ops, > + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ > + .max_msg = 0, /* overridden by virtio_get_max_msg() */ > + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, > +}; > + > +static int scmi_vio_probe(struct virtio_device *vdev) > +{ > + struct device *dev = &vdev->dev; > + struct scmi_vio_channel **vioch; > + bool have_vq_rx; > + int vq_cnt; > + int i; > + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; > + > + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), > + GFP_KERNEL); > + if (!vioch) > + return -ENOMEM; > + > + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); > + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; > + > + for (i = 0; i < vq_cnt; i++) { > + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); > + if (!vioch[i]) > + return -ENOMEM; > + } > + > + if (have_vq_rx) > + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; > + > + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, > + scmi_vio_vqueue_names, NULL)) { > + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); > + return -1; > + } > + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); > + > + for (i = 0; i < vq_cnt; i++) { > + spin_lock_init(&vioch[i]->lock); > + vioch[i]->vqueue = vqs[i]; > + vioch[i]->vqueue->priv = vioch[i]; > + } > + > + vdev->priv = vioch; > + As mentioned before, in other Virtio drivers this is the place where we should register somehow this transports/channels with the core (before marking the device ready), and may be the place where we can just share the core device for this tranport with the SCMI core without then having to play the above link_supplier chan_available/setup dance. But again I still have to fully tries to change the SCMI transport interface at this point. I've not really gone into all the details across all series because I see the above issues/limitations in the driver/SCMI_transport as sort of blocking. I'll let you now when I'll have something sensible on the SCMI core to share for better accomodate this transport. Thanks Cristian > + virtio_device_ready(vdev); > + > + return 0; > +} > + > +static unsigned int features[] = { > + VIRTIO_SCMI_F_P2A_CHANNELS, > +}; > + > +static const struct virtio_device_id id_table[] = { > + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, > + { 0 } > +}; > + > +static struct virtio_driver virtio_scmi_driver = { > + .driver.name = "scmi-virtio", > + .driver.owner = THIS_MODULE, > + .feature_table = features, > + .feature_table_size = ARRAY_SIZE(features), > + .id_table = id_table, > + .probe = scmi_vio_probe, > +}; > + > +int __init virtio_scmi_init(void) > +{ > + return register_virtio_driver(&virtio_scmi_driver); > +} > + > +void __exit virtio_scmi_exit(void) > +{ > + unregister_virtio_driver(&virtio_scmi_driver); > +} > diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h > index b052355ac7a3..57d233c02720 100644 > --- a/include/uapi/linux/virtio_ids.h > +++ b/include/uapi/linux/virtio_ids.h > @@ -48,5 +48,6 @@ > #define VIRTIO_ID_FS 26 /* virtio filesystem */ > #define VIRTIO_ID_PMEM 27 /* virtio pmem */ > #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ > +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ > > #endif /* _LINUX_VIRTIO_IDS_H */ > diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h > new file mode 100644 > index 000000000000..9f21b3dbbfe2 > --- /dev/null > +++ b/include/uapi/linux/virtio_scmi.h > @@ -0,0 +1,41 @@ > +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ > +/* > + * Copyright (C) 2020 OpenSynergy GmbH > + */ > + > +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H > +#define _UAPI_LINUX_VIRTIO_SCMI_H > + > +#include <linux/virtio_types.h> > + > +/* Feature bits */ > + > +/* Device implements some SCMI notifications, or delayed responses. */ > +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 > + > +/* Device implements any SCMI statistics shared memory region */ > +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 > + > +/* Virtqueues */ > + > +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ > +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ > +#define VIRTIO_SCMI_VQ_MAX_CNT 2 > + > +struct virtio_scmi_request { > + __virtio32 hdr; > + __u8 data[]; > +}; > + > +struct virtio_scmi_response { > + __virtio32 hdr; > + __virtio32 status; > + __u8 data[]; > +}; > + > +struct virtio_scmi_notification { > + __virtio32 hdr; > + __u8 data[]; > +}; > + > +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ > -- > 2.25.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport 2020-11-10 21:32 ` Cristian Marussi (?) @ 2020-11-12 10:57 ` Peter Hilber -1 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-12 10:57 UTC (permalink / raw) To: Cristian Marussi Cc: linux-arm-kernel, devicetree, virtualization, virtio-dev, jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, Michael S. Tsirkin, Jason Wang, linux-kernel, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee On 10.11.20 22:32, Cristian Marussi wrote: > Hi Peter/Igor, > > I went through this series while trying to grasp a bit more of all the > virtio inner workings and needs and I'll leave a few detailed comments > down below. > > In short at first, I think I can say that there are a couple of places > where I noticed you had to play all sort of tricks to fit into the current > SCMI transport layer frameowrk or to avoid adding DT references. > > It's clear that we'll have to somehow extend/fix/abstract the SCMI > transport layer (in my opinion), because I doubt that some of the tricks > you played would be well received for upstream ever (...but it's worth > noting it's not up to me saying the last...) > > So in these days I'm trying to play with this series and the SCMI > stack to see if I can extend it in a more sensible way to fit some of > the observations I make down below (now transport core layer is pretty > much shmem/mailbox oriented)...still nothing to share anyway as of now. Hi Cristian, thanks for your review. I agree that some changes to the patch series are needed to move it beyond RFC state. Please see individual responses below. Best regards, Peter > > > On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: >> From: Igor Skalkin <igor.skalkin@opensynergy.com> >> >> This transport enables accessing an SCMI platform as a virtio device. >> >> Implement an SCMI virtio driver according to the virtio SCMI device spec >> patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device >> [2]. >> >> The virtio transport has one tx channel (virtio cmdq, A2P channel) and >> at most one rx channel (virtio eventq, P2A channel). >> >> The following feature bit defined in [1] is not implemented: >> VIRTIO_SCMI_F_SHARED_MEMORY. >> >> After the preparatory patches, implement the virtio transport as >> paraphrased: >> >> Only support a single arm-scmi device (which is consistent with the SCMI >> spec). Call scmi-virtio init from arm-scmi module init. During the >> arm-scmi probing, link to the first probed scmi-virtio device. Defer >> arm-scmi probing if no scmi-virtio device is bound yet. >> >> Use the scmi_xfer tx/rx buffers for data exchange with the virtio device >> in order to avoid redundant maintenance of additional buffers. Allocate >> the buffers in the SCMI transport, and prepend room for a small header >> used by the virtio transport to the tx/rx buffers. >> >> For simplicity, restrict the number of messages which can be pending >> simultaneously according to the virtqueue capacity. (The virtqueue sizes >> are negotiated with the virtio device.) >> >> As soon as rx channel message buffers are allocated or have been read >> out by the arm-scmi driver, feed them to the virtio device. >> >> Since some virtio devices may not have the short response time exhibited >> by SCMI platforms using other transports, set a generous response >> timeout. >> >> Limitations: >> >> Do not adjust the other SCMI timeouts for delayed response and polling >> for now, since these timeouts are only relevant in special cases which >> are not yet deemed relevant for this transport. >> >> To do (as discussed in the cover letter): >> >> - Avoid re-use of buffers still being used by the virtio device on >> timeouts. >> >> - Avoid race conditions on receiving messages during/after channel free >> on driver probe failure or remove. >> >> [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html >> [2] https://www.oasis-open.org/committees/ballot.php?id=3496 >> >> Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> >> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> >> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> >> --- >> MAINTAINERS | 1 + >> drivers/firmware/Kconfig | 12 +- >> drivers/firmware/arm_scmi/Makefile | 1 + >> drivers/firmware/arm_scmi/common.h | 14 + >> drivers/firmware/arm_scmi/driver.c | 11 + >> drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ >> include/uapi/linux/virtio_ids.h | 1 + >> include/uapi/linux/virtio_scmi.h | 41 +++ >> 8 files changed, 573 insertions(+), 1 deletion(-) >> create mode 100644 drivers/firmware/arm_scmi/virtio.c >> create mode 100644 include/uapi/linux/virtio_scmi.h >> >> diff --git a/MAINTAINERS b/MAINTAINERS >> index deaafb617361..8df73d6ddfc1 100644 >> --- a/MAINTAINERS >> +++ b/MAINTAINERS >> @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c >> F: drivers/reset/reset-scmi.c >> F: include/linux/sc[mp]i_protocol.h >> F: include/trace/events/scmi.h >> +F: include/uapi/linux/virtio_scmi.h >> >> SYSTEM RESET/SHUTDOWN DRIVERS >> M: Sebastian Reichel <sre@kernel.org> >> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig >> index bdde51adb267..c4bdd84f7405 100644 >> --- a/drivers/firmware/Kconfig >> +++ b/drivers/firmware/Kconfig >> @@ -9,7 +9,7 @@ menu "Firmware Drivers" >> config ARM_SCMI_PROTOCOL >> tristate "ARM System Control and Management Interface (SCMI) Message Protocol" >> depends on ARM || ARM64 || COMPILE_TEST >> - depends on ARM_SCMI_HAVE_SHMEM >> + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI >> help >> ARM System Control and Management Interface (SCMI) protocol is a >> set of operating system-independent software interfaces that are >> @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM >> This declares whether a shared memory based transport for SCMI is >> available. >> >> +config VIRTIO_SCMI >> + bool "Virtio transport for SCMI" >> + default n >> + depends on VIRTIO >> + help >> + This enables the virtio based transport for SCMI. >> + >> + If you want to use the ARM SCMI protocol between the virtio guest and >> + a host providing a virtio SCMI device, answer Y. >> + >> config ARM_SCMI_POWER_DOMAIN >> tristate "SCMI power domain driver" >> depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) >> diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile >> index 3cc7fa40a464..25caea5e1969 100644 >> --- a/drivers/firmware/arm_scmi/Makefile >> +++ b/drivers/firmware/arm_scmi/Makefile >> @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o >> scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o >> scmi-transport-$(CONFIG_MAILBOX) += mailbox.o >> scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o >> +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o >> scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o >> scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ >> $(scmi-transport-y) >> diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h >> index 13c9ac176b23..b46dfe84e78b 100644 >> --- a/drivers/firmware/arm_scmi/common.h >> +++ b/drivers/firmware/arm_scmi/common.h >> @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); >> int __init scmi_bus_init(void); >> void __exit scmi_bus_exit(void); >> >> +#ifdef CONFIG_VIRTIO_SCMI >> +int __init virtio_scmi_init(void); >> +void __exit virtio_scmi_exit(void); >> +#else >> +static inline int __init virtio_scmi_init(void) >> +{ >> + return 0; >> +} >> +#define virtio_scmi_exit() do { } while (0) >> +#endif >> + >> #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ >> int __init scmi_##func##_register(void); \ >> void __exit scmi_##func##_unregister(void) >> @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; >> #ifdef CONFIG_HAVE_ARM_SMCCC >> extern const struct scmi_desc scmi_smc_desc; >> #endif >> +#ifdef CONFIG_VIRTIO_SCMI >> +extern const struct scmi_desc scmi_virtio_desc; >> +#endif >> >> int scmi_set_transport_info(struct device *dev, void *transport_info); >> void *scmi_get_transport_info(struct device *dev); >> diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c >> index 244141e45e88..923ba526e829 100644 >> --- a/drivers/firmware/arm_scmi/driver.c >> +++ b/drivers/firmware/arm_scmi/driver.c >> @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { >> #endif >> #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY >> { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, >> +#endif >> +#ifdef CONFIG_VIRTIO_SCMI >> + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, >> #endif >> { /* Sentinel */ }, >> }; >> @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { >> >> static int __init scmi_driver_init(void) >> { >> + int ret; >> + >> scmi_bus_init(); >> >> + ret = virtio_scmi_init(); >> + if (ret) >> + return ret; >> + >> scmi_clock_register(); >> scmi_perf_register(); >> scmi_power_register(); >> @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) >> scmi_sensors_unregister(); >> scmi_system_unregister(); >> >> + virtio_scmi_exit(); >> + > > These virtio init/exit functions which are called by the platform driver > init code in fact introduce a very transport specific non-general piece > of code in the common init path: this is one of the things I'd like to > abstract better, so that any available transport can just register itself > with the core in the same way and be initialized if needed in an uniform > way, without having to extend the core driver init. > (not saying that now it is not anyway already done for other > matters....I'd like to remove those too in the future.) Agree. > > >> platform_driver_unregister(&scmi_driver); >> } >> module_exit(scmi_driver_exit); >> diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c >> new file mode 100644 >> index 000000000000..f70aa72f34f1 >> --- /dev/null >> +++ b/drivers/firmware/arm_scmi/virtio.c >> @@ -0,0 +1,493 @@ >> +// SPDX-License-Identifier: GPL-2.0 >> +/* >> + * Virtio Transport driver for Arm System Control and Management Interface >> + * (SCMI). >> + * >> + * Copyright (C) 2020 OpenSynergy. >> + */ >> + >> +/** >> + * DOC: Theory of Operation >> + * >> + * The scmi-virtio transport implements a driver for the virtio SCMI device >> + * proposed in virtio spec patch v5[1]. >> + * >> + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx >> + * channel (virtio eventq, P2A channel). Each channel is implemented through a >> + * virtqueue. Access to each virtqueue is protected by a spinlock. >> + * >> + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with >> + * the virtio device to avoid maintenance of additional buffers. >> + * >> + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html >> + */ >> + >> +#include <linux/errno.h> >> +#include <linux/of.h> >> +#include <linux/of_platform.h> >> +#include <linux/platform_device.h> >> +#include <linux/module.h> >> +#include <linux/slab.h> >> +#include <linux/virtio.h> >> +#include <linux/virtio_config.h> >> +#include <uapi/linux/virtio_ids.h> >> +#include <uapi/linux/virtio_scmi.h> >> + >> +#include "common.h" >> + >> +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ >> +#define DESCR_PER_TX_MSG 2 >> + >> +struct scmi_vio_channel { >> + spinlock_t lock; >> + struct virtqueue *vqueue; >> + struct scmi_chan_info *cinfo; >> + u8 is_rx; >> +}; >> + >> +union virtio_scmi_input { >> + __virtio32 hdr; >> + struct virtio_scmi_response response; >> + struct virtio_scmi_notification notification; >> +}; >> + >> +struct scmi_vio_msg { >> + struct virtio_scmi_request *request; >> + union virtio_scmi_input *input; >> + u8 completed; >> +}; >> + >> +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, >> + struct scmi_xfer *xfer) >> +{ >> + struct scatterlist sg_in; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + int rc; >> + >> + msg->completed = false; >> + >> + sg_init_one(&sg_in, msg->input, >> + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); >> + >> + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); >> + if (rc) >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); >> + else >> + virtqueue_kick(vioch->vqueue); >> + >> + return rc; >> +} >> + >> +static void scmi_vio_complete_cb(struct virtqueue *vqueue) >> +{ >> + struct scmi_vio_channel *vioch = vqueue->priv; >> + unsigned long iflags; >> + unsigned int length; >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + >> + do { >> + struct scmi_xfer *xfer; >> + >> + virtqueue_disable_cb(vqueue); >> + >> + while ((xfer = virtqueue_get_buf(vqueue, &length))) { >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + u32 msg_hdr = >> + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); >> + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); >> + >> + if (!vioch->is_rx) { /* tx queue response */ >> + msg->completed = true; >> + xfer->rx.len = >> + length - sizeof(msg->input->response); >> + if (!xfer->hdr.poll_completion) >> + scmi_rx_callback(vioch->cinfo, msg_hdr); >> + continue; >> + } >> + > > So this is one of the places where SCMI transport seems not to accomodate so > well the way virtio works as of now; it seems to me to be down to the fact that > while virtio returns the message as a single chunk in one shot (since it carries > the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads > since the original shmem transport has the message size embedded in its header > (which so has to be read at first), and so you have to stretch the usage of > scmi_rx_callback introducing here some of its logic while leaving some other > callbacks as empty dummies (like notifications callback): I think the transport > should NOT have any knowledge of the structure/content of the stuff that it > receives (like down below detecting which MSG_TYPE it is and acting accordingly). > > It clearly cannot work like it is now (so you have to play the tricks), but I > think the answer is reviewing accordingly the SCMI transport layer, not trying > to fooli it while polluting the transport layer with knowlegde of SCMI internals > like message's structure. > I think I should remove the xfer->[tr]x.buf manipulation from the patch. In my understanding, this is one, or even the, major source of problems. It would have saved a few kiB of memory and avoided copying between scmi_xfer buffers and dedicated virtio queue buffers. But we can just introduce dedicated virtio buffers and copy, in lieu of a better idea. As for the rest, if you don't like any of the scmi_xfer related logic to be inside virtio.c, maybe we can move that logic into a new msg.c for message-based communication, similar to shm.c for shmem-based communication? The msg.c logic would then mostly be called from the callback ops, just like shm.c. I think we should retain some transport private pointer in scmi_xfer to help with potential concurrent or out-of-order message processing by the virtio device (access to the private pointer can be abstracted). Otherwise I'm concerned that in some callback ops it would be difficult for the virtio transport to determine which message is being addressed. If you agree on this, do you think that OpenSynergy could already start implementing the aforementioned msg.c, or should we better wait for your refactored interface? >> + /* rx queue - notification or delayed response */ >> + switch (msg_type) { >> + case MSG_TYPE_NOTIFICATION: >> + xfer->rx.len = length - >> + sizeof(msg->input->notification); >> + xfer->rx.buf = msg->input->notification.data; >> + break; >> + case MSG_TYPE_DELAYED_RESP: >> + xfer->rx.len = >> + length - sizeof(msg->input->response); >> + xfer->rx.buf = msg->input->response.data; >> + break; >> + default: >> + dev_warn_once(vioch->cinfo->dev, >> + "rx: unknown message_type %d\n", >> + msg_type); >> + scmi_vio_populate_vq_rx(vioch, xfer); >> + continue; >> + } >> + >> + scmi_rx_callback(vioch->cinfo, msg_hdr); >> + scmi_vio_populate_vq_rx(vioch, xfer); >> + } >> + >> + if (unlikely(virtqueue_is_broken(vqueue))) >> + break; >> + } while (!virtqueue_enable_cb(vqueue)); >> + >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> +} >> + >> +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; >> + >> +static vq_callback_t *scmi_vio_complete_callbacks[] = { >> + scmi_vio_complete_cb, >> + scmi_vio_complete_cb >> +}; >> + >> +static int scmi_vio_match_any_dev(struct device *dev, const void *data) >> +{ >> + (void)dev; >> + (void)data; >> + >> + return 1; >> +} >> + >> +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ >> + >> +static int virtio_link_supplier(struct device *dev) >> +{ >> + struct device *vdev = driver_find_device( >> + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); >> + >> + if (!vdev) { >> + dev_notice_once( >> + dev, >> + "Deferring probe after not finding a bound scmi-virtio device\n"); >> + return -EPROBE_DEFER; >> + } >> + >> + /* >> + * Add plain device link for completeness. It might have no effect >> + * beyond sysfs. >> + */ >> + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { >> + put_device(vdev); >> + dev_err(dev, "Adding link to supplier virtio device failed\n"); >> + return -ECANCELED; >> + } >> + >> + put_device(vdev); >> + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); >> +} >> + > > I understand that here the attempt is to grab the virtio device upon > which we are working and make it known to the core, while handling the > case in which the MMIO transport is not ready with -EPROBE_DEFER, but I > don't think such full-scale search with a dummy match function would be > so well accepted upstream, and I don't understand really what's the > point of the device_link_add() at the end...but maybe I'm missing > something. The point of the device_link_add() is just to make the relation between the devices apparent in sysfs. It should show up in sysfs as described here: https://lore.kernel.org/patchwork/patch/1245696/ > > Moreover looking at other virtio drivers (not that I'm an expert > though...) it seems very much to me that this SCMI Virtio driver is > sort of built/probed/initialized upside-down respect how the other > virtio drivers are made, and this seems to be down again to some missing > support in our SCMI transport layer (...plus the need to avoid any > virtio-related DT addition). > > That's the other thing that would be in my plan to rectify and unify in > the SCMI transport layer (...at least in my hopes :D) OK. > >> +static bool virtio_chan_available(struct device *dev, int idx) >> +{ >> + struct virtio_device *vdev; >> + struct scmi_vio_channel **vioch; >> + >> + /* scmi-virtio doesn't support per-protocol channels */ >> + if (is_scmi_protocol_device(dev)) >> + return false; >> + >> + vdev = scmi_get_transport_info(dev); >> + if (!vdev) >> + return false; >> + >> + vioch = vdev->priv; >> + if (!vioch) >> + return false; >> + >> + return vioch[idx] && vioch[idx]->vqueue; >> +} >> + >> +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, >> + bool tx) >> +{ >> + struct virtio_device *vdev; >> + struct scmi_vio_channel **vioch; >> + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; >> + >> + /* scmi-virtio doesn't support per-protocol channels */ >> + if (is_scmi_protocol_device(dev)) >> + return -1; >> + >> + vdev = scmi_get_transport_info(dev); >> + if (!vdev) >> + return -1; >> + >> + vioch = vdev->priv; >> + if (!vioch) { >> + dev_err(dev, "Data from scmi-virtio probe not found\n"); >> + return -1; >> + } >> + cinfo->transport_info = vioch[vioch_index]; >> + vioch[vioch_index]->cinfo = cinfo; >> + >> + return 0; >> +} >> + > > Same goes, from my point of view, for these two channel related > callbacks that now have to perform some very odd interactions and checks > with the SCMI core (at least in my opinion). > ATM the SCMI core will not call these callbacks before link_supplier() was successful, so we could indeed omit the second and third early return in each of the callbacks. It seemed more robust to leave them in, though. In case it doesn't get obsoleted by interface changes, I will add some additional comments to make clearer what is going on here. > Does the lack of support for multiple per-protocol channels derive from > the lack of a way (now that the DT entry has gone) to determine the > association between a channel and its user ? In my understanding that association could still be determined. Per-protocol channels were deliberately omitted, since we saw little benefit for virtio. In my understanding, it would only help to avoid that one protocol monopolizes the channel by sending many messages or causing many notifications. > >> +static int virtio_chan_free(int id, void *p, void *data) >> +{ >> + struct scmi_chan_info *cinfo = p; >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + >> + if (vioch) { >> + cinfo->transport_info = NULL; >> + kfree(vioch); >> + } >> + >> + scmi_free_channel(cinfo, data, id); >> + return 0; >> +} >> + >> +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, >> + int *max_msg) >> +{ >> + struct scmi_vio_channel *vioch = base_cinfo->transport_info; >> + >> + *max_msg = virtqueue_get_vring_size(vioch->vqueue); >> + >> + /* Tx messages need multiple descriptors. */ >> + if (tx) >> + *max_msg /= DESCR_PER_TX_MSG; >> + >> + if (*max_msg > MSG_TOKEN_MAX) { >> + dev_notice( >> + base_cinfo->dev, >> + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", >> + MSG_TOKEN_MAX, *max_msg); >> + *max_msg = MSG_TOKEN_MAX; >> + } >> + >> + return 0; >> +} >> + >> +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer, int max_msg_size) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg; >> + >> + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); >> + if (!msg) >> + return -ENOMEM; >> + >> + xfer->extra_data = msg; >> + >> + if (vioch->is_rx) { >> + int rc; >> + unsigned long iflags; >> + >> + msg->input = devm_kzalloc(cinfo->dev, >> + sizeof(*msg->input) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->input) >> + return -ENOMEM; >> + >> + /* >> + * xfer->rx.buf will be set to notification or delayed response >> + * specific values in the receive callback, according to the >> + * type of the received message. >> + */ >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + rc = scmi_vio_populate_vq_rx(vioch, xfer); >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + if (rc) >> + return rc; >> + } else { >> + msg->request = >> + devm_kzalloc(cinfo->dev, >> + sizeof(*msg->request) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->request) >> + return -ENOMEM; >> + >> + xfer->tx.buf = msg->request->data; >> + >> + msg->input = devm_kzalloc( >> + cinfo->dev, sizeof(msg->input->response) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->input) >> + return -ENOMEM; >> + >> + xfer->rx.buf = msg->input->response.data; >> + } >> + >> + return 0; >> +} >> + >> +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) >> +{ >> + struct scatterlist sg_out; >> + struct scatterlist sg_in; >> + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + unsigned long iflags; >> + int rc; >> + >> + msg->completed = false; >> + >> + sg_init_one(&sg_out, msg->request, >> + sizeof(*msg->request) + xfer->tx.len); >> + sg_init_one(&sg_in, &msg->input->response, >> + sizeof(msg->input->response) + xfer->rx.len); >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); >> + if (rc) >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); >> + else >> + virtqueue_kick(vioch->vqueue); >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + >> + return rc; >> +} >> + >> +static int virtio_send_message(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + uint32_t hdr; >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct virtio_device *vdev = vioch->vqueue->vdev; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + >> + hdr = pack_scmi_header(&xfer->hdr); >> + >> + msg->request->hdr = cpu_to_virtio32(vdev, hdr); >> + >> + return scmi_vio_send(vioch, xfer); >> +} >> + >> +static void virtio_fetch_response(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + >> + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, >> + msg->input->response.status); >> +} >> + >> +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, >> + size_t max_len, struct scmi_xfer *xfer) >> +{ >> + (void)cinfo; >> + (void)max_len; >> + (void)xfer; >> +} >> + >> +static void dummy_clear_channel(struct scmi_chan_info *cinfo) >> +{ >> + (void)cinfo; >> +} >> + >> +static bool virtio_poll_done(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + unsigned long iflags; >> + bool completed; >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + completed = msg->completed; >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + >> + return completed; >> +} >> + >> +static const struct scmi_transport_ops scmi_virtio_ops = { >> + .link_supplier = virtio_link_supplier, >> + .chan_available = virtio_chan_available, >> + .chan_setup = virtio_chan_setup, >> + .chan_free = virtio_chan_free, >> + .get_max_msg = virtio_get_max_msg, >> + .send_message = virtio_send_message, >> + .fetch_response = virtio_fetch_response, >> + .fetch_notification = dummy_fetch_notification, >> + .clear_channel = dummy_clear_channel, >> + .poll_done = virtio_poll_done, >> + .xfer_init_buffers = virtio_xfer_init_buffers, >> +}; >> + >> +const struct scmi_desc scmi_virtio_desc = { >> + .ops = &scmi_virtio_ops, >> + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ >> + .max_msg = 0, /* overridden by virtio_get_max_msg() */ >> + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, >> +}; >> + >> +static int scmi_vio_probe(struct virtio_device *vdev) >> +{ >> + struct device *dev = &vdev->dev; >> + struct scmi_vio_channel **vioch; >> + bool have_vq_rx; >> + int vq_cnt; >> + int i; >> + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; >> + >> + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), >> + GFP_KERNEL); >> + if (!vioch) >> + return -ENOMEM; >> + >> + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); >> + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; >> + >> + for (i = 0; i < vq_cnt; i++) { >> + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); >> + if (!vioch[i]) >> + return -ENOMEM; >> + } >> + >> + if (have_vq_rx) >> + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; >> + >> + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, >> + scmi_vio_vqueue_names, NULL)) { >> + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); >> + return -1; >> + } >> + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); >> + >> + for (i = 0; i < vq_cnt; i++) { >> + spin_lock_init(&vioch[i]->lock); >> + vioch[i]->vqueue = vqs[i]; >> + vioch[i]->vqueue->priv = vioch[i]; >> + } >> + >> + vdev->priv = vioch; >> + > > As mentioned before, in other Virtio drivers this is the place where we > should register somehow this transports/channels with the core (before > marking the device ready), and may be the place where we can just share > the core device for this tranport with the SCMI core without then having > to play the above link_supplier chan_available/setup dance. > > But again I still have to fully tries to change the SCMI transport > interface at this point. > > I've not really gone into all the details across all series because I > see the above issues/limitations in the driver/SCMI_transport as sort of > blocking. I think some of the patches will no longer be needed in the future. I would expect at least the following to go away: [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op > > I'll let you now when I'll have something sensible on the SCMI core to > share for better accomodate this transport. > > Thanks > > Cristian > > >> + virtio_device_ready(vdev); >> + >> + return 0; >> +} >> + >> +static unsigned int features[] = { >> + VIRTIO_SCMI_F_P2A_CHANNELS, >> +}; >> + >> +static const struct virtio_device_id id_table[] = { >> + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, >> + { 0 } >> +}; >> + >> +static struct virtio_driver virtio_scmi_driver = { >> + .driver.name = "scmi-virtio", >> + .driver.owner = THIS_MODULE, >> + .feature_table = features, >> + .feature_table_size = ARRAY_SIZE(features), >> + .id_table = id_table, >> + .probe = scmi_vio_probe, >> +}; >> + >> +int __init virtio_scmi_init(void) >> +{ >> + return register_virtio_driver(&virtio_scmi_driver); >> +} >> + >> +void __exit virtio_scmi_exit(void) >> +{ >> + unregister_virtio_driver(&virtio_scmi_driver); >> +} >> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h >> index b052355ac7a3..57d233c02720 100644 >> --- a/include/uapi/linux/virtio_ids.h >> +++ b/include/uapi/linux/virtio_ids.h >> @@ -48,5 +48,6 @@ >> #define VIRTIO_ID_FS 26 /* virtio filesystem */ >> #define VIRTIO_ID_PMEM 27 /* virtio pmem */ >> #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ >> +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ >> >> #endif /* _LINUX_VIRTIO_IDS_H */ >> diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h >> new file mode 100644 >> index 000000000000..9f21b3dbbfe2 >> --- /dev/null >> +++ b/include/uapi/linux/virtio_scmi.h >> @@ -0,0 +1,41 @@ >> +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ >> +/* >> + * Copyright (C) 2020 OpenSynergy GmbH >> + */ >> + >> +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H >> +#define _UAPI_LINUX_VIRTIO_SCMI_H >> + >> +#include <linux/virtio_types.h> >> + >> +/* Feature bits */ >> + >> +/* Device implements some SCMI notifications, or delayed responses. */ >> +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 >> + >> +/* Device implements any SCMI statistics shared memory region */ >> +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 >> + >> +/* Virtqueues */ >> + >> +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ >> +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ >> +#define VIRTIO_SCMI_VQ_MAX_CNT 2 >> + >> +struct virtio_scmi_request { >> + __virtio32 hdr; >> + __u8 data[]; >> +}; >> + >> +struct virtio_scmi_response { >> + __virtio32 hdr; >> + __virtio32 status; >> + __u8 data[]; >> +}; >> + >> +struct virtio_scmi_notification { >> + __virtio32 hdr; >> + __u8 data[]; >> +}; >> + >> +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ >> -- >> 2.25.1 >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > ^ permalink raw reply [flat|nested] 34+ messages in thread
* [virtio-dev] Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2020-11-12 10:57 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-12 10:57 UTC (permalink / raw) To: Cristian Marussi Cc: linux-arm-kernel, devicetree, virtualization, virtio-dev, jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, Michael S. Tsirkin, Jason Wang, linux-kernel, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee On 10.11.20 22:32, Cristian Marussi wrote: > Hi Peter/Igor, > > I went through this series while trying to grasp a bit more of all the > virtio inner workings and needs and I'll leave a few detailed comments > down below. > > In short at first, I think I can say that there are a couple of places > where I noticed you had to play all sort of tricks to fit into the current > SCMI transport layer frameowrk or to avoid adding DT references. > > It's clear that we'll have to somehow extend/fix/abstract the SCMI > transport layer (in my opinion), because I doubt that some of the tricks > you played would be well received for upstream ever (...but it's worth > noting it's not up to me saying the last...) > > So in these days I'm trying to play with this series and the SCMI > stack to see if I can extend it in a more sensible way to fit some of > the observations I make down below (now transport core layer is pretty > much shmem/mailbox oriented)...still nothing to share anyway as of now. Hi Cristian, thanks for your review. I agree that some changes to the patch series are needed to move it beyond RFC state. Please see individual responses below. Best regards, Peter > > > On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: >> From: Igor Skalkin <igor.skalkin@opensynergy.com> >> >> This transport enables accessing an SCMI platform as a virtio device. >> >> Implement an SCMI virtio driver according to the virtio SCMI device spec >> patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device >> [2]. >> >> The virtio transport has one tx channel (virtio cmdq, A2P channel) and >> at most one rx channel (virtio eventq, P2A channel). >> >> The following feature bit defined in [1] is not implemented: >> VIRTIO_SCMI_F_SHARED_MEMORY. >> >> After the preparatory patches, implement the virtio transport as >> paraphrased: >> >> Only support a single arm-scmi device (which is consistent with the SCMI >> spec). Call scmi-virtio init from arm-scmi module init. During the >> arm-scmi probing, link to the first probed scmi-virtio device. Defer >> arm-scmi probing if no scmi-virtio device is bound yet. >> >> Use the scmi_xfer tx/rx buffers for data exchange with the virtio device >> in order to avoid redundant maintenance of additional buffers. Allocate >> the buffers in the SCMI transport, and prepend room for a small header >> used by the virtio transport to the tx/rx buffers. >> >> For simplicity, restrict the number of messages which can be pending >> simultaneously according to the virtqueue capacity. (The virtqueue sizes >> are negotiated with the virtio device.) >> >> As soon as rx channel message buffers are allocated or have been read >> out by the arm-scmi driver, feed them to the virtio device. >> >> Since some virtio devices may not have the short response time exhibited >> by SCMI platforms using other transports, set a generous response >> timeout. >> >> Limitations: >> >> Do not adjust the other SCMI timeouts for delayed response and polling >> for now, since these timeouts are only relevant in special cases which >> are not yet deemed relevant for this transport. >> >> To do (as discussed in the cover letter): >> >> - Avoid re-use of buffers still being used by the virtio device on >> timeouts. >> >> - Avoid race conditions on receiving messages during/after channel free >> on driver probe failure or remove. >> >> [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html >> [2] https://www.oasis-open.org/committees/ballot.php?id=3496 >> >> Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> >> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> >> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> >> --- >> MAINTAINERS | 1 + >> drivers/firmware/Kconfig | 12 +- >> drivers/firmware/arm_scmi/Makefile | 1 + >> drivers/firmware/arm_scmi/common.h | 14 + >> drivers/firmware/arm_scmi/driver.c | 11 + >> drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ >> include/uapi/linux/virtio_ids.h | 1 + >> include/uapi/linux/virtio_scmi.h | 41 +++ >> 8 files changed, 573 insertions(+), 1 deletion(-) >> create mode 100644 drivers/firmware/arm_scmi/virtio.c >> create mode 100644 include/uapi/linux/virtio_scmi.h >> >> diff --git a/MAINTAINERS b/MAINTAINERS >> index deaafb617361..8df73d6ddfc1 100644 >> --- a/MAINTAINERS >> +++ b/MAINTAINERS >> @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c >> F: drivers/reset/reset-scmi.c >> F: include/linux/sc[mp]i_protocol.h >> F: include/trace/events/scmi.h >> +F: include/uapi/linux/virtio_scmi.h >> >> SYSTEM RESET/SHUTDOWN DRIVERS >> M: Sebastian Reichel <sre@kernel.org> >> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig >> index bdde51adb267..c4bdd84f7405 100644 >> --- a/drivers/firmware/Kconfig >> +++ b/drivers/firmware/Kconfig >> @@ -9,7 +9,7 @@ menu "Firmware Drivers" >> config ARM_SCMI_PROTOCOL >> tristate "ARM System Control and Management Interface (SCMI) Message Protocol" >> depends on ARM || ARM64 || COMPILE_TEST >> - depends on ARM_SCMI_HAVE_SHMEM >> + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI >> help >> ARM System Control and Management Interface (SCMI) protocol is a >> set of operating system-independent software interfaces that are >> @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM >> This declares whether a shared memory based transport for SCMI is >> available. >> >> +config VIRTIO_SCMI >> + bool "Virtio transport for SCMI" >> + default n >> + depends on VIRTIO >> + help >> + This enables the virtio based transport for SCMI. >> + >> + If you want to use the ARM SCMI protocol between the virtio guest and >> + a host providing a virtio SCMI device, answer Y. >> + >> config ARM_SCMI_POWER_DOMAIN >> tristate "SCMI power domain driver" >> depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) >> diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile >> index 3cc7fa40a464..25caea5e1969 100644 >> --- a/drivers/firmware/arm_scmi/Makefile >> +++ b/drivers/firmware/arm_scmi/Makefile >> @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o >> scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o >> scmi-transport-$(CONFIG_MAILBOX) += mailbox.o >> scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o >> +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o >> scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o >> scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ >> $(scmi-transport-y) >> diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h >> index 13c9ac176b23..b46dfe84e78b 100644 >> --- a/drivers/firmware/arm_scmi/common.h >> +++ b/drivers/firmware/arm_scmi/common.h >> @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); >> int __init scmi_bus_init(void); >> void __exit scmi_bus_exit(void); >> >> +#ifdef CONFIG_VIRTIO_SCMI >> +int __init virtio_scmi_init(void); >> +void __exit virtio_scmi_exit(void); >> +#else >> +static inline int __init virtio_scmi_init(void) >> +{ >> + return 0; >> +} >> +#define virtio_scmi_exit() do { } while (0) >> +#endif >> + >> #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ >> int __init scmi_##func##_register(void); \ >> void __exit scmi_##func##_unregister(void) >> @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; >> #ifdef CONFIG_HAVE_ARM_SMCCC >> extern const struct scmi_desc scmi_smc_desc; >> #endif >> +#ifdef CONFIG_VIRTIO_SCMI >> +extern const struct scmi_desc scmi_virtio_desc; >> +#endif >> >> int scmi_set_transport_info(struct device *dev, void *transport_info); >> void *scmi_get_transport_info(struct device *dev); >> diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c >> index 244141e45e88..923ba526e829 100644 >> --- a/drivers/firmware/arm_scmi/driver.c >> +++ b/drivers/firmware/arm_scmi/driver.c >> @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { >> #endif >> #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY >> { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, >> +#endif >> +#ifdef CONFIG_VIRTIO_SCMI >> + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, >> #endif >> { /* Sentinel */ }, >> }; >> @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { >> >> static int __init scmi_driver_init(void) >> { >> + int ret; >> + >> scmi_bus_init(); >> >> + ret = virtio_scmi_init(); >> + if (ret) >> + return ret; >> + >> scmi_clock_register(); >> scmi_perf_register(); >> scmi_power_register(); >> @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) >> scmi_sensors_unregister(); >> scmi_system_unregister(); >> >> + virtio_scmi_exit(); >> + > > These virtio init/exit functions which are called by the platform driver > init code in fact introduce a very transport specific non-general piece > of code in the common init path: this is one of the things I'd like to > abstract better, so that any available transport can just register itself > with the core in the same way and be initialized if needed in an uniform > way, without having to extend the core driver init. > (not saying that now it is not anyway already done for other > matters....I'd like to remove those too in the future.) Agree. > > >> platform_driver_unregister(&scmi_driver); >> } >> module_exit(scmi_driver_exit); >> diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c >> new file mode 100644 >> index 000000000000..f70aa72f34f1 >> --- /dev/null >> +++ b/drivers/firmware/arm_scmi/virtio.c >> @@ -0,0 +1,493 @@ >> +// SPDX-License-Identifier: GPL-2.0 >> +/* >> + * Virtio Transport driver for Arm System Control and Management Interface >> + * (SCMI). >> + * >> + * Copyright (C) 2020 OpenSynergy. >> + */ >> + >> +/** >> + * DOC: Theory of Operation >> + * >> + * The scmi-virtio transport implements a driver for the virtio SCMI device >> + * proposed in virtio spec patch v5[1]. >> + * >> + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx >> + * channel (virtio eventq, P2A channel). Each channel is implemented through a >> + * virtqueue. Access to each virtqueue is protected by a spinlock. >> + * >> + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with >> + * the virtio device to avoid maintenance of additional buffers. >> + * >> + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html >> + */ >> + >> +#include <linux/errno.h> >> +#include <linux/of.h> >> +#include <linux/of_platform.h> >> +#include <linux/platform_device.h> >> +#include <linux/module.h> >> +#include <linux/slab.h> >> +#include <linux/virtio.h> >> +#include <linux/virtio_config.h> >> +#include <uapi/linux/virtio_ids.h> >> +#include <uapi/linux/virtio_scmi.h> >> + >> +#include "common.h" >> + >> +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ >> +#define DESCR_PER_TX_MSG 2 >> + >> +struct scmi_vio_channel { >> + spinlock_t lock; >> + struct virtqueue *vqueue; >> + struct scmi_chan_info *cinfo; >> + u8 is_rx; >> +}; >> + >> +union virtio_scmi_input { >> + __virtio32 hdr; >> + struct virtio_scmi_response response; >> + struct virtio_scmi_notification notification; >> +}; >> + >> +struct scmi_vio_msg { >> + struct virtio_scmi_request *request; >> + union virtio_scmi_input *input; >> + u8 completed; >> +}; >> + >> +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, >> + struct scmi_xfer *xfer) >> +{ >> + struct scatterlist sg_in; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + int rc; >> + >> + msg->completed = false; >> + >> + sg_init_one(&sg_in, msg->input, >> + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); >> + >> + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); >> + if (rc) >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); >> + else >> + virtqueue_kick(vioch->vqueue); >> + >> + return rc; >> +} >> + >> +static void scmi_vio_complete_cb(struct virtqueue *vqueue) >> +{ >> + struct scmi_vio_channel *vioch = vqueue->priv; >> + unsigned long iflags; >> + unsigned int length; >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + >> + do { >> + struct scmi_xfer *xfer; >> + >> + virtqueue_disable_cb(vqueue); >> + >> + while ((xfer = virtqueue_get_buf(vqueue, &length))) { >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + u32 msg_hdr = >> + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); >> + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); >> + >> + if (!vioch->is_rx) { /* tx queue response */ >> + msg->completed = true; >> + xfer->rx.len = >> + length - sizeof(msg->input->response); >> + if (!xfer->hdr.poll_completion) >> + scmi_rx_callback(vioch->cinfo, msg_hdr); >> + continue; >> + } >> + > > So this is one of the places where SCMI transport seems not to accomodate so > well the way virtio works as of now; it seems to me to be down to the fact that > while virtio returns the message as a single chunk in one shot (since it carries > the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads > since the original shmem transport has the message size embedded in its header > (which so has to be read at first), and so you have to stretch the usage of > scmi_rx_callback introducing here some of its logic while leaving some other > callbacks as empty dummies (like notifications callback): I think the transport > should NOT have any knowledge of the structure/content of the stuff that it > receives (like down below detecting which MSG_TYPE it is and acting accordingly). > > It clearly cannot work like it is now (so you have to play the tricks), but I > think the answer is reviewing accordingly the SCMI transport layer, not trying > to fooli it while polluting the transport layer with knowlegde of SCMI internals > like message's structure. > I think I should remove the xfer->[tr]x.buf manipulation from the patch. In my understanding, this is one, or even the, major source of problems. It would have saved a few kiB of memory and avoided copying between scmi_xfer buffers and dedicated virtio queue buffers. But we can just introduce dedicated virtio buffers and copy, in lieu of a better idea. As for the rest, if you don't like any of the scmi_xfer related logic to be inside virtio.c, maybe we can move that logic into a new msg.c for message-based communication, similar to shm.c for shmem-based communication? The msg.c logic would then mostly be called from the callback ops, just like shm.c. I think we should retain some transport private pointer in scmi_xfer to help with potential concurrent or out-of-order message processing by the virtio device (access to the private pointer can be abstracted). Otherwise I'm concerned that in some callback ops it would be difficult for the virtio transport to determine which message is being addressed. If you agree on this, do you think that OpenSynergy could already start implementing the aforementioned msg.c, or should we better wait for your refactored interface? >> + /* rx queue - notification or delayed response */ >> + switch (msg_type) { >> + case MSG_TYPE_NOTIFICATION: >> + xfer->rx.len = length - >> + sizeof(msg->input->notification); >> + xfer->rx.buf = msg->input->notification.data; >> + break; >> + case MSG_TYPE_DELAYED_RESP: >> + xfer->rx.len = >> + length - sizeof(msg->input->response); >> + xfer->rx.buf = msg->input->response.data; >> + break; >> + default: >> + dev_warn_once(vioch->cinfo->dev, >> + "rx: unknown message_type %d\n", >> + msg_type); >> + scmi_vio_populate_vq_rx(vioch, xfer); >> + continue; >> + } >> + >> + scmi_rx_callback(vioch->cinfo, msg_hdr); >> + scmi_vio_populate_vq_rx(vioch, xfer); >> + } >> + >> + if (unlikely(virtqueue_is_broken(vqueue))) >> + break; >> + } while (!virtqueue_enable_cb(vqueue)); >> + >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> +} >> + >> +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; >> + >> +static vq_callback_t *scmi_vio_complete_callbacks[] = { >> + scmi_vio_complete_cb, >> + scmi_vio_complete_cb >> +}; >> + >> +static int scmi_vio_match_any_dev(struct device *dev, const void *data) >> +{ >> + (void)dev; >> + (void)data; >> + >> + return 1; >> +} >> + >> +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ >> + >> +static int virtio_link_supplier(struct device *dev) >> +{ >> + struct device *vdev = driver_find_device( >> + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); >> + >> + if (!vdev) { >> + dev_notice_once( >> + dev, >> + "Deferring probe after not finding a bound scmi-virtio device\n"); >> + return -EPROBE_DEFER; >> + } >> + >> + /* >> + * Add plain device link for completeness. It might have no effect >> + * beyond sysfs. >> + */ >> + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { >> + put_device(vdev); >> + dev_err(dev, "Adding link to supplier virtio device failed\n"); >> + return -ECANCELED; >> + } >> + >> + put_device(vdev); >> + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); >> +} >> + > > I understand that here the attempt is to grab the virtio device upon > which we are working and make it known to the core, while handling the > case in which the MMIO transport is not ready with -EPROBE_DEFER, but I > don't think such full-scale search with a dummy match function would be > so well accepted upstream, and I don't understand really what's the > point of the device_link_add() at the end...but maybe I'm missing > something. The point of the device_link_add() is just to make the relation between the devices apparent in sysfs. It should show up in sysfs as described here: https://lore.kernel.org/patchwork/patch/1245696/ > > Moreover looking at other virtio drivers (not that I'm an expert > though...) it seems very much to me that this SCMI Virtio driver is > sort of built/probed/initialized upside-down respect how the other > virtio drivers are made, and this seems to be down again to some missing > support in our SCMI transport layer (...plus the need to avoid any > virtio-related DT addition). > > That's the other thing that would be in my plan to rectify and unify in > the SCMI transport layer (...at least in my hopes :D) OK. > >> +static bool virtio_chan_available(struct device *dev, int idx) >> +{ >> + struct virtio_device *vdev; >> + struct scmi_vio_channel **vioch; >> + >> + /* scmi-virtio doesn't support per-protocol channels */ >> + if (is_scmi_protocol_device(dev)) >> + return false; >> + >> + vdev = scmi_get_transport_info(dev); >> + if (!vdev) >> + return false; >> + >> + vioch = vdev->priv; >> + if (!vioch) >> + return false; >> + >> + return vioch[idx] && vioch[idx]->vqueue; >> +} >> + >> +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, >> + bool tx) >> +{ >> + struct virtio_device *vdev; >> + struct scmi_vio_channel **vioch; >> + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; >> + >> + /* scmi-virtio doesn't support per-protocol channels */ >> + if (is_scmi_protocol_device(dev)) >> + return -1; >> + >> + vdev = scmi_get_transport_info(dev); >> + if (!vdev) >> + return -1; >> + >> + vioch = vdev->priv; >> + if (!vioch) { >> + dev_err(dev, "Data from scmi-virtio probe not found\n"); >> + return -1; >> + } >> + cinfo->transport_info = vioch[vioch_index]; >> + vioch[vioch_index]->cinfo = cinfo; >> + >> + return 0; >> +} >> + > > Same goes, from my point of view, for these two channel related > callbacks that now have to perform some very odd interactions and checks > with the SCMI core (at least in my opinion). > ATM the SCMI core will not call these callbacks before link_supplier() was successful, so we could indeed omit the second and third early return in each of the callbacks. It seemed more robust to leave them in, though. In case it doesn't get obsoleted by interface changes, I will add some additional comments to make clearer what is going on here. > Does the lack of support for multiple per-protocol channels derive from > the lack of a way (now that the DT entry has gone) to determine the > association between a channel and its user ? In my understanding that association could still be determined. Per-protocol channels were deliberately omitted, since we saw little benefit for virtio. In my understanding, it would only help to avoid that one protocol monopolizes the channel by sending many messages or causing many notifications. > >> +static int virtio_chan_free(int id, void *p, void *data) >> +{ >> + struct scmi_chan_info *cinfo = p; >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + >> + if (vioch) { >> + cinfo->transport_info = NULL; >> + kfree(vioch); >> + } >> + >> + scmi_free_channel(cinfo, data, id); >> + return 0; >> +} >> + >> +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, >> + int *max_msg) >> +{ >> + struct scmi_vio_channel *vioch = base_cinfo->transport_info; >> + >> + *max_msg = virtqueue_get_vring_size(vioch->vqueue); >> + >> + /* Tx messages need multiple descriptors. */ >> + if (tx) >> + *max_msg /= DESCR_PER_TX_MSG; >> + >> + if (*max_msg > MSG_TOKEN_MAX) { >> + dev_notice( >> + base_cinfo->dev, >> + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", >> + MSG_TOKEN_MAX, *max_msg); >> + *max_msg = MSG_TOKEN_MAX; >> + } >> + >> + return 0; >> +} >> + >> +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer, int max_msg_size) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg; >> + >> + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); >> + if (!msg) >> + return -ENOMEM; >> + >> + xfer->extra_data = msg; >> + >> + if (vioch->is_rx) { >> + int rc; >> + unsigned long iflags; >> + >> + msg->input = devm_kzalloc(cinfo->dev, >> + sizeof(*msg->input) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->input) >> + return -ENOMEM; >> + >> + /* >> + * xfer->rx.buf will be set to notification or delayed response >> + * specific values in the receive callback, according to the >> + * type of the received message. >> + */ >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + rc = scmi_vio_populate_vq_rx(vioch, xfer); >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + if (rc) >> + return rc; >> + } else { >> + msg->request = >> + devm_kzalloc(cinfo->dev, >> + sizeof(*msg->request) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->request) >> + return -ENOMEM; >> + >> + xfer->tx.buf = msg->request->data; >> + >> + msg->input = devm_kzalloc( >> + cinfo->dev, sizeof(msg->input->response) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->input) >> + return -ENOMEM; >> + >> + xfer->rx.buf = msg->input->response.data; >> + } >> + >> + return 0; >> +} >> + >> +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) >> +{ >> + struct scatterlist sg_out; >> + struct scatterlist sg_in; >> + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + unsigned long iflags; >> + int rc; >> + >> + msg->completed = false; >> + >> + sg_init_one(&sg_out, msg->request, >> + sizeof(*msg->request) + xfer->tx.len); >> + sg_init_one(&sg_in, &msg->input->response, >> + sizeof(msg->input->response) + xfer->rx.len); >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); >> + if (rc) >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); >> + else >> + virtqueue_kick(vioch->vqueue); >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + >> + return rc; >> +} >> + >> +static int virtio_send_message(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + uint32_t hdr; >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct virtio_device *vdev = vioch->vqueue->vdev; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + >> + hdr = pack_scmi_header(&xfer->hdr); >> + >> + msg->request->hdr = cpu_to_virtio32(vdev, hdr); >> + >> + return scmi_vio_send(vioch, xfer); >> +} >> + >> +static void virtio_fetch_response(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + >> + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, >> + msg->input->response.status); >> +} >> + >> +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, >> + size_t max_len, struct scmi_xfer *xfer) >> +{ >> + (void)cinfo; >> + (void)max_len; >> + (void)xfer; >> +} >> + >> +static void dummy_clear_channel(struct scmi_chan_info *cinfo) >> +{ >> + (void)cinfo; >> +} >> + >> +static bool virtio_poll_done(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + unsigned long iflags; >> + bool completed; >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + completed = msg->completed; >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + >> + return completed; >> +} >> + >> +static const struct scmi_transport_ops scmi_virtio_ops = { >> + .link_supplier = virtio_link_supplier, >> + .chan_available = virtio_chan_available, >> + .chan_setup = virtio_chan_setup, >> + .chan_free = virtio_chan_free, >> + .get_max_msg = virtio_get_max_msg, >> + .send_message = virtio_send_message, >> + .fetch_response = virtio_fetch_response, >> + .fetch_notification = dummy_fetch_notification, >> + .clear_channel = dummy_clear_channel, >> + .poll_done = virtio_poll_done, >> + .xfer_init_buffers = virtio_xfer_init_buffers, >> +}; >> + >> +const struct scmi_desc scmi_virtio_desc = { >> + .ops = &scmi_virtio_ops, >> + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ >> + .max_msg = 0, /* overridden by virtio_get_max_msg() */ >> + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, >> +}; >> + >> +static int scmi_vio_probe(struct virtio_device *vdev) >> +{ >> + struct device *dev = &vdev->dev; >> + struct scmi_vio_channel **vioch; >> + bool have_vq_rx; >> + int vq_cnt; >> + int i; >> + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; >> + >> + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), >> + GFP_KERNEL); >> + if (!vioch) >> + return -ENOMEM; >> + >> + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); >> + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; >> + >> + for (i = 0; i < vq_cnt; i++) { >> + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); >> + if (!vioch[i]) >> + return -ENOMEM; >> + } >> + >> + if (have_vq_rx) >> + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; >> + >> + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, >> + scmi_vio_vqueue_names, NULL)) { >> + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); >> + return -1; >> + } >> + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); >> + >> + for (i = 0; i < vq_cnt; i++) { >> + spin_lock_init(&vioch[i]->lock); >> + vioch[i]->vqueue = vqs[i]; >> + vioch[i]->vqueue->priv = vioch[i]; >> + } >> + >> + vdev->priv = vioch; >> + > > As mentioned before, in other Virtio drivers this is the place where we > should register somehow this transports/channels with the core (before > marking the device ready), and may be the place where we can just share > the core device for this tranport with the SCMI core without then having > to play the above link_supplier chan_available/setup dance. > > But again I still have to fully tries to change the SCMI transport > interface at this point. > > I've not really gone into all the details across all series because I > see the above issues/limitations in the driver/SCMI_transport as sort of > blocking. I think some of the patches will no longer be needed in the future. I would expect at least the following to go away: [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op > > I'll let you now when I'll have something sensible on the SCMI core to > share for better accomodate this transport. > > Thanks > > Cristian > > >> + virtio_device_ready(vdev); >> + >> + return 0; >> +} >> + >> +static unsigned int features[] = { >> + VIRTIO_SCMI_F_P2A_CHANNELS, >> +}; >> + >> +static const struct virtio_device_id id_table[] = { >> + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, >> + { 0 } >> +}; >> + >> +static struct virtio_driver virtio_scmi_driver = { >> + .driver.name = "scmi-virtio", >> + .driver.owner = THIS_MODULE, >> + .feature_table = features, >> + .feature_table_size = ARRAY_SIZE(features), >> + .id_table = id_table, >> + .probe = scmi_vio_probe, >> +}; >> + >> +int __init virtio_scmi_init(void) >> +{ >> + return register_virtio_driver(&virtio_scmi_driver); >> +} >> + >> +void __exit virtio_scmi_exit(void) >> +{ >> + unregister_virtio_driver(&virtio_scmi_driver); >> +} >> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h >> index b052355ac7a3..57d233c02720 100644 >> --- a/include/uapi/linux/virtio_ids.h >> +++ b/include/uapi/linux/virtio_ids.h >> @@ -48,5 +48,6 @@ >> #define VIRTIO_ID_FS 26 /* virtio filesystem */ >> #define VIRTIO_ID_PMEM 27 /* virtio pmem */ >> #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ >> +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ >> >> #endif /* _LINUX_VIRTIO_IDS_H */ >> diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h >> new file mode 100644 >> index 000000000000..9f21b3dbbfe2 >> --- /dev/null >> +++ b/include/uapi/linux/virtio_scmi.h >> @@ -0,0 +1,41 @@ >> +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ >> +/* >> + * Copyright (C) 2020 OpenSynergy GmbH >> + */ >> + >> +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H >> +#define _UAPI_LINUX_VIRTIO_SCMI_H >> + >> +#include <linux/virtio_types.h> >> + >> +/* Feature bits */ >> + >> +/* Device implements some SCMI notifications, or delayed responses. */ >> +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 >> + >> +/* Device implements any SCMI statistics shared memory region */ >> +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 >> + >> +/* Virtqueues */ >> + >> +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ >> +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ >> +#define VIRTIO_SCMI_VQ_MAX_CNT 2 >> + >> +struct virtio_scmi_request { >> + __virtio32 hdr; >> + __u8 data[]; >> +}; >> + >> +struct virtio_scmi_response { >> + __virtio32 hdr; >> + __virtio32 status; >> + __u8 data[]; >> +}; >> + >> +struct virtio_scmi_notification { >> + __virtio32 hdr; >> + __u8 data[]; >> +}; >> + >> +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ >> -- >> 2.25.1 >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org ^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2020-11-12 10:57 ` Peter Hilber 0 siblings, 0 replies; 34+ messages in thread From: Peter Hilber @ 2020-11-12 10:57 UTC (permalink / raw) To: Cristian Marussi Cc: devicetree, mikhail.golubev, souvik.chakravarty, Igor Skalkin, jean-philippe, Jason Wang, Michael S. Tsirkin, linux-kernel, virtualization, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee, virtio-dev, linux-arm-kernel On 10.11.20 22:32, Cristian Marussi wrote: > Hi Peter/Igor, > > I went through this series while trying to grasp a bit more of all the > virtio inner workings and needs and I'll leave a few detailed comments > down below. > > In short at first, I think I can say that there are a couple of places > where I noticed you had to play all sort of tricks to fit into the current > SCMI transport layer frameowrk or to avoid adding DT references. > > It's clear that we'll have to somehow extend/fix/abstract the SCMI > transport layer (in my opinion), because I doubt that some of the tricks > you played would be well received for upstream ever (...but it's worth > noting it's not up to me saying the last...) > > So in these days I'm trying to play with this series and the SCMI > stack to see if I can extend it in a more sensible way to fit some of > the observations I make down below (now transport core layer is pretty > much shmem/mailbox oriented)...still nothing to share anyway as of now. Hi Cristian, thanks for your review. I agree that some changes to the patch series are needed to move it beyond RFC state. Please see individual responses below. Best regards, Peter > > > On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: >> From: Igor Skalkin <igor.skalkin@opensynergy.com> >> >> This transport enables accessing an SCMI platform as a virtio device. >> >> Implement an SCMI virtio driver according to the virtio SCMI device spec >> patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device >> [2]. >> >> The virtio transport has one tx channel (virtio cmdq, A2P channel) and >> at most one rx channel (virtio eventq, P2A channel). >> >> The following feature bit defined in [1] is not implemented: >> VIRTIO_SCMI_F_SHARED_MEMORY. >> >> After the preparatory patches, implement the virtio transport as >> paraphrased: >> >> Only support a single arm-scmi device (which is consistent with the SCMI >> spec). Call scmi-virtio init from arm-scmi module init. During the >> arm-scmi probing, link to the first probed scmi-virtio device. Defer >> arm-scmi probing if no scmi-virtio device is bound yet. >> >> Use the scmi_xfer tx/rx buffers for data exchange with the virtio device >> in order to avoid redundant maintenance of additional buffers. Allocate >> the buffers in the SCMI transport, and prepend room for a small header >> used by the virtio transport to the tx/rx buffers. >> >> For simplicity, restrict the number of messages which can be pending >> simultaneously according to the virtqueue capacity. (The virtqueue sizes >> are negotiated with the virtio device.) >> >> As soon as rx channel message buffers are allocated or have been read >> out by the arm-scmi driver, feed them to the virtio device. >> >> Since some virtio devices may not have the short response time exhibited >> by SCMI platforms using other transports, set a generous response >> timeout. >> >> Limitations: >> >> Do not adjust the other SCMI timeouts for delayed response and polling >> for now, since these timeouts are only relevant in special cases which >> are not yet deemed relevant for this transport. >> >> To do (as discussed in the cover letter): >> >> - Avoid re-use of buffers still being used by the virtio device on >> timeouts. >> >> - Avoid race conditions on receiving messages during/after channel free >> on driver probe failure or remove. >> >> [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html >> [2] https://www.oasis-open.org/committees/ballot.php?id=3496 >> >> Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> >> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> >> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> >> --- >> MAINTAINERS | 1 + >> drivers/firmware/Kconfig | 12 +- >> drivers/firmware/arm_scmi/Makefile | 1 + >> drivers/firmware/arm_scmi/common.h | 14 + >> drivers/firmware/arm_scmi/driver.c | 11 + >> drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ >> include/uapi/linux/virtio_ids.h | 1 + >> include/uapi/linux/virtio_scmi.h | 41 +++ >> 8 files changed, 573 insertions(+), 1 deletion(-) >> create mode 100644 drivers/firmware/arm_scmi/virtio.c >> create mode 100644 include/uapi/linux/virtio_scmi.h >> >> diff --git a/MAINTAINERS b/MAINTAINERS >> index deaafb617361..8df73d6ddfc1 100644 >> --- a/MAINTAINERS >> +++ b/MAINTAINERS >> @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c >> F: drivers/reset/reset-scmi.c >> F: include/linux/sc[mp]i_protocol.h >> F: include/trace/events/scmi.h >> +F: include/uapi/linux/virtio_scmi.h >> >> SYSTEM RESET/SHUTDOWN DRIVERS >> M: Sebastian Reichel <sre@kernel.org> >> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig >> index bdde51adb267..c4bdd84f7405 100644 >> --- a/drivers/firmware/Kconfig >> +++ b/drivers/firmware/Kconfig >> @@ -9,7 +9,7 @@ menu "Firmware Drivers" >> config ARM_SCMI_PROTOCOL >> tristate "ARM System Control and Management Interface (SCMI) Message Protocol" >> depends on ARM || ARM64 || COMPILE_TEST >> - depends on ARM_SCMI_HAVE_SHMEM >> + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI >> help >> ARM System Control and Management Interface (SCMI) protocol is a >> set of operating system-independent software interfaces that are >> @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM >> This declares whether a shared memory based transport for SCMI is >> available. >> >> +config VIRTIO_SCMI >> + bool "Virtio transport for SCMI" >> + default n >> + depends on VIRTIO >> + help >> + This enables the virtio based transport for SCMI. >> + >> + If you want to use the ARM SCMI protocol between the virtio guest and >> + a host providing a virtio SCMI device, answer Y. >> + >> config ARM_SCMI_POWER_DOMAIN >> tristate "SCMI power domain driver" >> depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) >> diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile >> index 3cc7fa40a464..25caea5e1969 100644 >> --- a/drivers/firmware/arm_scmi/Makefile >> +++ b/drivers/firmware/arm_scmi/Makefile >> @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o >> scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o >> scmi-transport-$(CONFIG_MAILBOX) += mailbox.o >> scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o >> +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o >> scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o >> scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ >> $(scmi-transport-y) >> diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h >> index 13c9ac176b23..b46dfe84e78b 100644 >> --- a/drivers/firmware/arm_scmi/common.h >> +++ b/drivers/firmware/arm_scmi/common.h >> @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); >> int __init scmi_bus_init(void); >> void __exit scmi_bus_exit(void); >> >> +#ifdef CONFIG_VIRTIO_SCMI >> +int __init virtio_scmi_init(void); >> +void __exit virtio_scmi_exit(void); >> +#else >> +static inline int __init virtio_scmi_init(void) >> +{ >> + return 0; >> +} >> +#define virtio_scmi_exit() do { } while (0) >> +#endif >> + >> #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ >> int __init scmi_##func##_register(void); \ >> void __exit scmi_##func##_unregister(void) >> @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; >> #ifdef CONFIG_HAVE_ARM_SMCCC >> extern const struct scmi_desc scmi_smc_desc; >> #endif >> +#ifdef CONFIG_VIRTIO_SCMI >> +extern const struct scmi_desc scmi_virtio_desc; >> +#endif >> >> int scmi_set_transport_info(struct device *dev, void *transport_info); >> void *scmi_get_transport_info(struct device *dev); >> diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c >> index 244141e45e88..923ba526e829 100644 >> --- a/drivers/firmware/arm_scmi/driver.c >> +++ b/drivers/firmware/arm_scmi/driver.c >> @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { >> #endif >> #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY >> { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, >> +#endif >> +#ifdef CONFIG_VIRTIO_SCMI >> + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, >> #endif >> { /* Sentinel */ }, >> }; >> @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { >> >> static int __init scmi_driver_init(void) >> { >> + int ret; >> + >> scmi_bus_init(); >> >> + ret = virtio_scmi_init(); >> + if (ret) >> + return ret; >> + >> scmi_clock_register(); >> scmi_perf_register(); >> scmi_power_register(); >> @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) >> scmi_sensors_unregister(); >> scmi_system_unregister(); >> >> + virtio_scmi_exit(); >> + > > These virtio init/exit functions which are called by the platform driver > init code in fact introduce a very transport specific non-general piece > of code in the common init path: this is one of the things I'd like to > abstract better, so that any available transport can just register itself > with the core in the same way and be initialized if needed in an uniform > way, without having to extend the core driver init. > (not saying that now it is not anyway already done for other > matters....I'd like to remove those too in the future.) Agree. > > >> platform_driver_unregister(&scmi_driver); >> } >> module_exit(scmi_driver_exit); >> diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c >> new file mode 100644 >> index 000000000000..f70aa72f34f1 >> --- /dev/null >> +++ b/drivers/firmware/arm_scmi/virtio.c >> @@ -0,0 +1,493 @@ >> +// SPDX-License-Identifier: GPL-2.0 >> +/* >> + * Virtio Transport driver for Arm System Control and Management Interface >> + * (SCMI). >> + * >> + * Copyright (C) 2020 OpenSynergy. >> + */ >> + >> +/** >> + * DOC: Theory of Operation >> + * >> + * The scmi-virtio transport implements a driver for the virtio SCMI device >> + * proposed in virtio spec patch v5[1]. >> + * >> + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx >> + * channel (virtio eventq, P2A channel). Each channel is implemented through a >> + * virtqueue. Access to each virtqueue is protected by a spinlock. >> + * >> + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with >> + * the virtio device to avoid maintenance of additional buffers. >> + * >> + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html >> + */ >> + >> +#include <linux/errno.h> >> +#include <linux/of.h> >> +#include <linux/of_platform.h> >> +#include <linux/platform_device.h> >> +#include <linux/module.h> >> +#include <linux/slab.h> >> +#include <linux/virtio.h> >> +#include <linux/virtio_config.h> >> +#include <uapi/linux/virtio_ids.h> >> +#include <uapi/linux/virtio_scmi.h> >> + >> +#include "common.h" >> + >> +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ >> +#define DESCR_PER_TX_MSG 2 >> + >> +struct scmi_vio_channel { >> + spinlock_t lock; >> + struct virtqueue *vqueue; >> + struct scmi_chan_info *cinfo; >> + u8 is_rx; >> +}; >> + >> +union virtio_scmi_input { >> + __virtio32 hdr; >> + struct virtio_scmi_response response; >> + struct virtio_scmi_notification notification; >> +}; >> + >> +struct scmi_vio_msg { >> + struct virtio_scmi_request *request; >> + union virtio_scmi_input *input; >> + u8 completed; >> +}; >> + >> +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, >> + struct scmi_xfer *xfer) >> +{ >> + struct scatterlist sg_in; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + int rc; >> + >> + msg->completed = false; >> + >> + sg_init_one(&sg_in, msg->input, >> + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); >> + >> + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); >> + if (rc) >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); >> + else >> + virtqueue_kick(vioch->vqueue); >> + >> + return rc; >> +} >> + >> +static void scmi_vio_complete_cb(struct virtqueue *vqueue) >> +{ >> + struct scmi_vio_channel *vioch = vqueue->priv; >> + unsigned long iflags; >> + unsigned int length; >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + >> + do { >> + struct scmi_xfer *xfer; >> + >> + virtqueue_disable_cb(vqueue); >> + >> + while ((xfer = virtqueue_get_buf(vqueue, &length))) { >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + u32 msg_hdr = >> + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); >> + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); >> + >> + if (!vioch->is_rx) { /* tx queue response */ >> + msg->completed = true; >> + xfer->rx.len = >> + length - sizeof(msg->input->response); >> + if (!xfer->hdr.poll_completion) >> + scmi_rx_callback(vioch->cinfo, msg_hdr); >> + continue; >> + } >> + > > So this is one of the places where SCMI transport seems not to accomodate so > well the way virtio works as of now; it seems to me to be down to the fact that > while virtio returns the message as a single chunk in one shot (since it carries > the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads > since the original shmem transport has the message size embedded in its header > (which so has to be read at first), and so you have to stretch the usage of > scmi_rx_callback introducing here some of its logic while leaving some other > callbacks as empty dummies (like notifications callback): I think the transport > should NOT have any knowledge of the structure/content of the stuff that it > receives (like down below detecting which MSG_TYPE it is and acting accordingly). > > It clearly cannot work like it is now (so you have to play the tricks), but I > think the answer is reviewing accordingly the SCMI transport layer, not trying > to fooli it while polluting the transport layer with knowlegde of SCMI internals > like message's structure. > I think I should remove the xfer->[tr]x.buf manipulation from the patch. In my understanding, this is one, or even the, major source of problems. It would have saved a few kiB of memory and avoided copying between scmi_xfer buffers and dedicated virtio queue buffers. But we can just introduce dedicated virtio buffers and copy, in lieu of a better idea. As for the rest, if you don't like any of the scmi_xfer related logic to be inside virtio.c, maybe we can move that logic into a new msg.c for message-based communication, similar to shm.c for shmem-based communication? The msg.c logic would then mostly be called from the callback ops, just like shm.c. I think we should retain some transport private pointer in scmi_xfer to help with potential concurrent or out-of-order message processing by the virtio device (access to the private pointer can be abstracted). Otherwise I'm concerned that in some callback ops it would be difficult for the virtio transport to determine which message is being addressed. If you agree on this, do you think that OpenSynergy could already start implementing the aforementioned msg.c, or should we better wait for your refactored interface? >> + /* rx queue - notification or delayed response */ >> + switch (msg_type) { >> + case MSG_TYPE_NOTIFICATION: >> + xfer->rx.len = length - >> + sizeof(msg->input->notification); >> + xfer->rx.buf = msg->input->notification.data; >> + break; >> + case MSG_TYPE_DELAYED_RESP: >> + xfer->rx.len = >> + length - sizeof(msg->input->response); >> + xfer->rx.buf = msg->input->response.data; >> + break; >> + default: >> + dev_warn_once(vioch->cinfo->dev, >> + "rx: unknown message_type %d\n", >> + msg_type); >> + scmi_vio_populate_vq_rx(vioch, xfer); >> + continue; >> + } >> + >> + scmi_rx_callback(vioch->cinfo, msg_hdr); >> + scmi_vio_populate_vq_rx(vioch, xfer); >> + } >> + >> + if (unlikely(virtqueue_is_broken(vqueue))) >> + break; >> + } while (!virtqueue_enable_cb(vqueue)); >> + >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> +} >> + >> +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; >> + >> +static vq_callback_t *scmi_vio_complete_callbacks[] = { >> + scmi_vio_complete_cb, >> + scmi_vio_complete_cb >> +}; >> + >> +static int scmi_vio_match_any_dev(struct device *dev, const void *data) >> +{ >> + (void)dev; >> + (void)data; >> + >> + return 1; >> +} >> + >> +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ >> + >> +static int virtio_link_supplier(struct device *dev) >> +{ >> + struct device *vdev = driver_find_device( >> + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); >> + >> + if (!vdev) { >> + dev_notice_once( >> + dev, >> + "Deferring probe after not finding a bound scmi-virtio device\n"); >> + return -EPROBE_DEFER; >> + } >> + >> + /* >> + * Add plain device link for completeness. It might have no effect >> + * beyond sysfs. >> + */ >> + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { >> + put_device(vdev); >> + dev_err(dev, "Adding link to supplier virtio device failed\n"); >> + return -ECANCELED; >> + } >> + >> + put_device(vdev); >> + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); >> +} >> + > > I understand that here the attempt is to grab the virtio device upon > which we are working and make it known to the core, while handling the > case in which the MMIO transport is not ready with -EPROBE_DEFER, but I > don't think such full-scale search with a dummy match function would be > so well accepted upstream, and I don't understand really what's the > point of the device_link_add() at the end...but maybe I'm missing > something. The point of the device_link_add() is just to make the relation between the devices apparent in sysfs. It should show up in sysfs as described here: https://lore.kernel.org/patchwork/patch/1245696/ > > Moreover looking at other virtio drivers (not that I'm an expert > though...) it seems very much to me that this SCMI Virtio driver is > sort of built/probed/initialized upside-down respect how the other > virtio drivers are made, and this seems to be down again to some missing > support in our SCMI transport layer (...plus the need to avoid any > virtio-related DT addition). > > That's the other thing that would be in my plan to rectify and unify in > the SCMI transport layer (...at least in my hopes :D) OK. > >> +static bool virtio_chan_available(struct device *dev, int idx) >> +{ >> + struct virtio_device *vdev; >> + struct scmi_vio_channel **vioch; >> + >> + /* scmi-virtio doesn't support per-protocol channels */ >> + if (is_scmi_protocol_device(dev)) >> + return false; >> + >> + vdev = scmi_get_transport_info(dev); >> + if (!vdev) >> + return false; >> + >> + vioch = vdev->priv; >> + if (!vioch) >> + return false; >> + >> + return vioch[idx] && vioch[idx]->vqueue; >> +} >> + >> +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, >> + bool tx) >> +{ >> + struct virtio_device *vdev; >> + struct scmi_vio_channel **vioch; >> + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; >> + >> + /* scmi-virtio doesn't support per-protocol channels */ >> + if (is_scmi_protocol_device(dev)) >> + return -1; >> + >> + vdev = scmi_get_transport_info(dev); >> + if (!vdev) >> + return -1; >> + >> + vioch = vdev->priv; >> + if (!vioch) { >> + dev_err(dev, "Data from scmi-virtio probe not found\n"); >> + return -1; >> + } >> + cinfo->transport_info = vioch[vioch_index]; >> + vioch[vioch_index]->cinfo = cinfo; >> + >> + return 0; >> +} >> + > > Same goes, from my point of view, for these two channel related > callbacks that now have to perform some very odd interactions and checks > with the SCMI core (at least in my opinion). > ATM the SCMI core will not call these callbacks before link_supplier() was successful, so we could indeed omit the second and third early return in each of the callbacks. It seemed more robust to leave them in, though. In case it doesn't get obsoleted by interface changes, I will add some additional comments to make clearer what is going on here. > Does the lack of support for multiple per-protocol channels derive from > the lack of a way (now that the DT entry has gone) to determine the > association between a channel and its user ? In my understanding that association could still be determined. Per-protocol channels were deliberately omitted, since we saw little benefit for virtio. In my understanding, it would only help to avoid that one protocol monopolizes the channel by sending many messages or causing many notifications. > >> +static int virtio_chan_free(int id, void *p, void *data) >> +{ >> + struct scmi_chan_info *cinfo = p; >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + >> + if (vioch) { >> + cinfo->transport_info = NULL; >> + kfree(vioch); >> + } >> + >> + scmi_free_channel(cinfo, data, id); >> + return 0; >> +} >> + >> +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, >> + int *max_msg) >> +{ >> + struct scmi_vio_channel *vioch = base_cinfo->transport_info; >> + >> + *max_msg = virtqueue_get_vring_size(vioch->vqueue); >> + >> + /* Tx messages need multiple descriptors. */ >> + if (tx) >> + *max_msg /= DESCR_PER_TX_MSG; >> + >> + if (*max_msg > MSG_TOKEN_MAX) { >> + dev_notice( >> + base_cinfo->dev, >> + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", >> + MSG_TOKEN_MAX, *max_msg); >> + *max_msg = MSG_TOKEN_MAX; >> + } >> + >> + return 0; >> +} >> + >> +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer, int max_msg_size) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg; >> + >> + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); >> + if (!msg) >> + return -ENOMEM; >> + >> + xfer->extra_data = msg; >> + >> + if (vioch->is_rx) { >> + int rc; >> + unsigned long iflags; >> + >> + msg->input = devm_kzalloc(cinfo->dev, >> + sizeof(*msg->input) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->input) >> + return -ENOMEM; >> + >> + /* >> + * xfer->rx.buf will be set to notification or delayed response >> + * specific values in the receive callback, according to the >> + * type of the received message. >> + */ >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + rc = scmi_vio_populate_vq_rx(vioch, xfer); >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + if (rc) >> + return rc; >> + } else { >> + msg->request = >> + devm_kzalloc(cinfo->dev, >> + sizeof(*msg->request) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->request) >> + return -ENOMEM; >> + >> + xfer->tx.buf = msg->request->data; >> + >> + msg->input = devm_kzalloc( >> + cinfo->dev, sizeof(msg->input->response) + max_msg_size, >> + GFP_KERNEL); >> + if (!msg->input) >> + return -ENOMEM; >> + >> + xfer->rx.buf = msg->input->response.data; >> + } >> + >> + return 0; >> +} >> + >> +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) >> +{ >> + struct scatterlist sg_out; >> + struct scatterlist sg_in; >> + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + unsigned long iflags; >> + int rc; >> + >> + msg->completed = false; >> + >> + sg_init_one(&sg_out, msg->request, >> + sizeof(*msg->request) + xfer->tx.len); >> + sg_init_one(&sg_in, &msg->input->response, >> + sizeof(msg->input->response) + xfer->rx.len); >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); >> + if (rc) >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); >> + else >> + virtqueue_kick(vioch->vqueue); >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + >> + return rc; >> +} >> + >> +static int virtio_send_message(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + uint32_t hdr; >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct virtio_device *vdev = vioch->vqueue->vdev; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + >> + hdr = pack_scmi_header(&xfer->hdr); >> + >> + msg->request->hdr = cpu_to_virtio32(vdev, hdr); >> + >> + return scmi_vio_send(vioch, xfer); >> +} >> + >> +static void virtio_fetch_response(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + >> + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, >> + msg->input->response.status); >> +} >> + >> +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, >> + size_t max_len, struct scmi_xfer *xfer) >> +{ >> + (void)cinfo; >> + (void)max_len; >> + (void)xfer; >> +} >> + >> +static void dummy_clear_channel(struct scmi_chan_info *cinfo) >> +{ >> + (void)cinfo; >> +} >> + >> +static bool virtio_poll_done(struct scmi_chan_info *cinfo, >> + struct scmi_xfer *xfer) >> +{ >> + struct scmi_vio_channel *vioch = cinfo->transport_info; >> + struct scmi_vio_msg *msg = xfer->extra_data; >> + unsigned long iflags; >> + bool completed; >> + >> + spin_lock_irqsave(&vioch->lock, iflags); >> + completed = msg->completed; >> + spin_unlock_irqrestore(&vioch->lock, iflags); >> + >> + return completed; >> +} >> + >> +static const struct scmi_transport_ops scmi_virtio_ops = { >> + .link_supplier = virtio_link_supplier, >> + .chan_available = virtio_chan_available, >> + .chan_setup = virtio_chan_setup, >> + .chan_free = virtio_chan_free, >> + .get_max_msg = virtio_get_max_msg, >> + .send_message = virtio_send_message, >> + .fetch_response = virtio_fetch_response, >> + .fetch_notification = dummy_fetch_notification, >> + .clear_channel = dummy_clear_channel, >> + .poll_done = virtio_poll_done, >> + .xfer_init_buffers = virtio_xfer_init_buffers, >> +}; >> + >> +const struct scmi_desc scmi_virtio_desc = { >> + .ops = &scmi_virtio_ops, >> + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ >> + .max_msg = 0, /* overridden by virtio_get_max_msg() */ >> + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, >> +}; >> + >> +static int scmi_vio_probe(struct virtio_device *vdev) >> +{ >> + struct device *dev = &vdev->dev; >> + struct scmi_vio_channel **vioch; >> + bool have_vq_rx; >> + int vq_cnt; >> + int i; >> + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; >> + >> + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), >> + GFP_KERNEL); >> + if (!vioch) >> + return -ENOMEM; >> + >> + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); >> + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; >> + >> + for (i = 0; i < vq_cnt; i++) { >> + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); >> + if (!vioch[i]) >> + return -ENOMEM; >> + } >> + >> + if (have_vq_rx) >> + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; >> + >> + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, >> + scmi_vio_vqueue_names, NULL)) { >> + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); >> + return -1; >> + } >> + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); >> + >> + for (i = 0; i < vq_cnt; i++) { >> + spin_lock_init(&vioch[i]->lock); >> + vioch[i]->vqueue = vqs[i]; >> + vioch[i]->vqueue->priv = vioch[i]; >> + } >> + >> + vdev->priv = vioch; >> + > > As mentioned before, in other Virtio drivers this is the place where we > should register somehow this transports/channels with the core (before > marking the device ready), and may be the place where we can just share > the core device for this tranport with the SCMI core without then having > to play the above link_supplier chan_available/setup dance. > > But again I still have to fully tries to change the SCMI transport > interface at this point. > > I've not really gone into all the details across all series because I > see the above issues/limitations in the driver/SCMI_transport as sort of > blocking. I think some of the patches will no longer be needed in the future. I would expect at least the following to go away: [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op > > I'll let you now when I'll have something sensible on the SCMI core to > share for better accomodate this transport. > > Thanks > > Cristian > > >> + virtio_device_ready(vdev); >> + >> + return 0; >> +} >> + >> +static unsigned int features[] = { >> + VIRTIO_SCMI_F_P2A_CHANNELS, >> +}; >> + >> +static const struct virtio_device_id id_table[] = { >> + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, >> + { 0 } >> +}; >> + >> +static struct virtio_driver virtio_scmi_driver = { >> + .driver.name = "scmi-virtio", >> + .driver.owner = THIS_MODULE, >> + .feature_table = features, >> + .feature_table_size = ARRAY_SIZE(features), >> + .id_table = id_table, >> + .probe = scmi_vio_probe, >> +}; >> + >> +int __init virtio_scmi_init(void) >> +{ >> + return register_virtio_driver(&virtio_scmi_driver); >> +} >> + >> +void __exit virtio_scmi_exit(void) >> +{ >> + unregister_virtio_driver(&virtio_scmi_driver); >> +} >> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h >> index b052355ac7a3..57d233c02720 100644 >> --- a/include/uapi/linux/virtio_ids.h >> +++ b/include/uapi/linux/virtio_ids.h >> @@ -48,5 +48,6 @@ >> #define VIRTIO_ID_FS 26 /* virtio filesystem */ >> #define VIRTIO_ID_PMEM 27 /* virtio pmem */ >> #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ >> +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ >> >> #endif /* _LINUX_VIRTIO_IDS_H */ >> diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h >> new file mode 100644 >> index 000000000000..9f21b3dbbfe2 >> --- /dev/null >> +++ b/include/uapi/linux/virtio_scmi.h >> @@ -0,0 +1,41 @@ >> +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ >> +/* >> + * Copyright (C) 2020 OpenSynergy GmbH >> + */ >> + >> +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H >> +#define _UAPI_LINUX_VIRTIO_SCMI_H >> + >> +#include <linux/virtio_types.h> >> + >> +/* Feature bits */ >> + >> +/* Device implements some SCMI notifications, or delayed responses. */ >> +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 >> + >> +/* Device implements any SCMI statistics shared memory region */ >> +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 >> + >> +/* Virtqueues */ >> + >> +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ >> +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ >> +#define VIRTIO_SCMI_VQ_MAX_CNT 2 >> + >> +struct virtio_scmi_request { >> + __virtio32 hdr; >> + __u8 data[]; >> +}; >> + >> +struct virtio_scmi_response { >> + __virtio32 hdr; >> + __virtio32 status; >> + __u8 data[]; >> +}; >> + >> +struct virtio_scmi_notification { >> + __virtio32 hdr; >> + __u8 data[]; >> +}; >> + >> +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ >> -- >> 2.25.1 >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport 2020-11-12 10:57 ` Peter Hilber @ 2020-11-17 21:20 ` Cristian Marussi -1 siblings, 0 replies; 34+ messages in thread From: Cristian Marussi @ 2020-11-17 21:20 UTC (permalink / raw) To: Peter Hilber Cc: linux-arm-kernel, devicetree, virtualization, virtio-dev, jean-philippe, mikhail.golubev, souvik.chakravarty, Igor Skalkin, Michael S. Tsirkin, Jason Wang, linux-kernel, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee On Thu, Nov 12, 2020 at 11:57:09AM +0100, Peter Hilber wrote: > On 10.11.20 22:32, Cristian Marussi wrote: > > Hi Peter/Igor, > > > > I went through this series while trying to grasp a bit more of all the > > virtio inner workings and needs and I'll leave a few detailed comments > > down below. > > > > In short at first, I think I can say that there are a couple of places > > where I noticed you had to play all sort of tricks to fit into the current > > SCMI transport layer frameowrk or to avoid adding DT references. > > > > It's clear that we'll have to somehow extend/fix/abstract the SCMI > > transport layer (in my opinion), because I doubt that some of the tricks > > you played would be well received for upstream ever (...but it's worth > > noting it's not up to me saying the last...) > > > > So in these days I'm trying to play with this series and the SCMI > > stack to see if I can extend it in a more sensible way to fit some of > > the observations I make down below (now transport core layer is pretty > > much shmem/mailbox oriented)...still nothing to share anyway as of now. > > Hi Cristian, > > thanks for your review. I agree that some changes to the patch series > are needed to move it beyond RFC state. Please see individual responses > below. > > Best regards, > > Peter > Hi Peter, apologize for the delay in the answer. Yesterday I experimenetd with some changes only related to the initialization and probing in order to expose a better interface from the SCMI core and limit the need for tricks I was talking about. Nothing about the message dispatching still. I tested minimally hacking brutally virtio-mmio to fake the presence of an SCMI device so as to verify I had not done worst :D For this reason I'll send you at first such patches offlist just for you to verify init/probing and timings are still ok and get your general feedback; patches should apply cleanly on your codebase, but they are just an initial experiment so they are not even cleanly split: they both add new core interfaces and use it in your code removing unneded code from virtio-scmi (hopefully). So, yeah, they are as of now a big ball of mud...just for you to experiment or point out if I missed something basic. Sorry for that. Tomorrow I'll get back to look at the rest of the series and the message dispatching to have a better idea what to do. Thanks Cristian > > > > > > On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: > >> From: Igor Skalkin <igor.skalkin@opensynergy.com> > >> > >> This transport enables accessing an SCMI platform as a virtio device. > >> > >> Implement an SCMI virtio driver according to the virtio SCMI device spec > >> patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device > >> [2]. > >> > >> The virtio transport has one tx channel (virtio cmdq, A2P channel) and > >> at most one rx channel (virtio eventq, P2A channel). > >> > >> The following feature bit defined in [1] is not implemented: > >> VIRTIO_SCMI_F_SHARED_MEMORY. > >> > >> After the preparatory patches, implement the virtio transport as > >> paraphrased: > >> > >> Only support a single arm-scmi device (which is consistent with the SCMI > >> spec). Call scmi-virtio init from arm-scmi module init. During the > >> arm-scmi probing, link to the first probed scmi-virtio device. Defer > >> arm-scmi probing if no scmi-virtio device is bound yet. > >> > >> Use the scmi_xfer tx/rx buffers for data exchange with the virtio device > >> in order to avoid redundant maintenance of additional buffers. Allocate > >> the buffers in the SCMI transport, and prepend room for a small header > >> used by the virtio transport to the tx/rx buffers. > >> > >> For simplicity, restrict the number of messages which can be pending > >> simultaneously according to the virtqueue capacity. (The virtqueue sizes > >> are negotiated with the virtio device.) > >> > >> As soon as rx channel message buffers are allocated or have been read > >> out by the arm-scmi driver, feed them to the virtio device. > >> > >> Since some virtio devices may not have the short response time exhibited > >> by SCMI platforms using other transports, set a generous response > >> timeout. > >> > >> Limitations: > >> > >> Do not adjust the other SCMI timeouts for delayed response and polling > >> for now, since these timeouts are only relevant in special cases which > >> are not yet deemed relevant for this transport. > >> > >> To do (as discussed in the cover letter): > >> > >> - Avoid re-use of buffers still being used by the virtio device on > >> timeouts. > >> > >> - Avoid race conditions on receiving messages during/after channel free > >> on driver probe failure or remove. > >> > >> [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > >> [2] https://www.oasis-open.org/committees/ballot.php?id=3496 > >> > >> Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> > >> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> > >> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> > >> --- > >> MAINTAINERS | 1 + > >> drivers/firmware/Kconfig | 12 +- > >> drivers/firmware/arm_scmi/Makefile | 1 + > >> drivers/firmware/arm_scmi/common.h | 14 + > >> drivers/firmware/arm_scmi/driver.c | 11 + > >> drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ > >> include/uapi/linux/virtio_ids.h | 1 + > >> include/uapi/linux/virtio_scmi.h | 41 +++ > >> 8 files changed, 573 insertions(+), 1 deletion(-) > >> create mode 100644 drivers/firmware/arm_scmi/virtio.c > >> create mode 100644 include/uapi/linux/virtio_scmi.h > >> > >> diff --git a/MAINTAINERS b/MAINTAINERS > >> index deaafb617361..8df73d6ddfc1 100644 > >> --- a/MAINTAINERS > >> +++ b/MAINTAINERS > >> @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c > >> F: drivers/reset/reset-scmi.c > >> F: include/linux/sc[mp]i_protocol.h > >> F: include/trace/events/scmi.h > >> +F: include/uapi/linux/virtio_scmi.h > >> > >> SYSTEM RESET/SHUTDOWN DRIVERS > >> M: Sebastian Reichel <sre@kernel.org> > >> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig > >> index bdde51adb267..c4bdd84f7405 100644 > >> --- a/drivers/firmware/Kconfig > >> +++ b/drivers/firmware/Kconfig > >> @@ -9,7 +9,7 @@ menu "Firmware Drivers" > >> config ARM_SCMI_PROTOCOL > >> tristate "ARM System Control and Management Interface (SCMI) Message Protocol" > >> depends on ARM || ARM64 || COMPILE_TEST > >> - depends on ARM_SCMI_HAVE_SHMEM > >> + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI > >> help > >> ARM System Control and Management Interface (SCMI) protocol is a > >> set of operating system-independent software interfaces that are > >> @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM > >> This declares whether a shared memory based transport for SCMI is > >> available. > >> > >> +config VIRTIO_SCMI > >> + bool "Virtio transport for SCMI" > >> + default n > >> + depends on VIRTIO > >> + help > >> + This enables the virtio based transport for SCMI. > >> + > >> + If you want to use the ARM SCMI protocol between the virtio guest and > >> + a host providing a virtio SCMI device, answer Y. > >> + > >> config ARM_SCMI_POWER_DOMAIN > >> tristate "SCMI power domain driver" > >> depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) > >> diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile > >> index 3cc7fa40a464..25caea5e1969 100644 > >> --- a/drivers/firmware/arm_scmi/Makefile > >> +++ b/drivers/firmware/arm_scmi/Makefile > >> @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o > >> scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o > >> scmi-transport-$(CONFIG_MAILBOX) += mailbox.o > >> scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o > >> +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o > >> scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o > >> scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ > >> $(scmi-transport-y) > >> diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h > >> index 13c9ac176b23..b46dfe84e78b 100644 > >> --- a/drivers/firmware/arm_scmi/common.h > >> +++ b/drivers/firmware/arm_scmi/common.h > >> @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); > >> int __init scmi_bus_init(void); > >> void __exit scmi_bus_exit(void); > >> > >> +#ifdef CONFIG_VIRTIO_SCMI > >> +int __init virtio_scmi_init(void); > >> +void __exit virtio_scmi_exit(void); > >> +#else > >> +static inline int __init virtio_scmi_init(void) > >> +{ > >> + return 0; > >> +} > >> +#define virtio_scmi_exit() do { } while (0) > >> +#endif > >> + > >> #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ > >> int __init scmi_##func##_register(void); \ > >> void __exit scmi_##func##_unregister(void) > >> @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; > >> #ifdef CONFIG_HAVE_ARM_SMCCC > >> extern const struct scmi_desc scmi_smc_desc; > >> #endif > >> +#ifdef CONFIG_VIRTIO_SCMI > >> +extern const struct scmi_desc scmi_virtio_desc; > >> +#endif > >> > >> int scmi_set_transport_info(struct device *dev, void *transport_info); > >> void *scmi_get_transport_info(struct device *dev); > >> diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c > >> index 244141e45e88..923ba526e829 100644 > >> --- a/drivers/firmware/arm_scmi/driver.c > >> +++ b/drivers/firmware/arm_scmi/driver.c > >> @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { > >> #endif > >> #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY > >> { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, > >> +#endif > >> +#ifdef CONFIG_VIRTIO_SCMI > >> + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, > >> #endif > >> { /* Sentinel */ }, > >> }; > >> @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { > >> > >> static int __init scmi_driver_init(void) > >> { > >> + int ret; > >> + > >> scmi_bus_init(); > >> > >> + ret = virtio_scmi_init(); > >> + if (ret) > >> + return ret; > >> + > >> scmi_clock_register(); > >> scmi_perf_register(); > >> scmi_power_register(); > >> @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) > >> scmi_sensors_unregister(); > >> scmi_system_unregister(); > >> > >> + virtio_scmi_exit(); > >> + > > > > These virtio init/exit functions which are called by the platform driver > > init code in fact introduce a very transport specific non-general piece > > of code in the common init path: this is one of the things I'd like to > > abstract better, so that any available transport can just register itself > > with the core in the same way and be initialized if needed in an uniform > > way, without having to extend the core driver init. > > (not saying that now it is not anyway already done for other > > matters....I'd like to remove those too in the future.) > > Agree. > > > > > > >> platform_driver_unregister(&scmi_driver); > >> } > >> module_exit(scmi_driver_exit); > >> diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c > >> new file mode 100644 > >> index 000000000000..f70aa72f34f1 > >> --- /dev/null > >> +++ b/drivers/firmware/arm_scmi/virtio.c > >> @@ -0,0 +1,493 @@ > >> +// SPDX-License-Identifier: GPL-2.0 > >> +/* > >> + * Virtio Transport driver for Arm System Control and Management Interface > >> + * (SCMI). > >> + * > >> + * Copyright (C) 2020 OpenSynergy. > >> + */ > >> + > >> +/** > >> + * DOC: Theory of Operation > >> + * > >> + * The scmi-virtio transport implements a driver for the virtio SCMI device > >> + * proposed in virtio spec patch v5[1]. > >> + * > >> + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx > >> + * channel (virtio eventq, P2A channel). Each channel is implemented through a > >> + * virtqueue. Access to each virtqueue is protected by a spinlock. > >> + * > >> + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with > >> + * the virtio device to avoid maintenance of additional buffers. > >> + * > >> + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > >> + */ > >> + > >> +#include <linux/errno.h> > >> +#include <linux/of.h> > >> +#include <linux/of_platform.h> > >> +#include <linux/platform_device.h> > >> +#include <linux/module.h> > >> +#include <linux/slab.h> > >> +#include <linux/virtio.h> > >> +#include <linux/virtio_config.h> > >> +#include <uapi/linux/virtio_ids.h> > >> +#include <uapi/linux/virtio_scmi.h> > >> + > >> +#include "common.h" > >> + > >> +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ > >> +#define DESCR_PER_TX_MSG 2 > >> + > >> +struct scmi_vio_channel { > >> + spinlock_t lock; > >> + struct virtqueue *vqueue; > >> + struct scmi_chan_info *cinfo; > >> + u8 is_rx; > >> +}; > >> + > >> +union virtio_scmi_input { > >> + __virtio32 hdr; > >> + struct virtio_scmi_response response; > >> + struct virtio_scmi_notification notification; > >> +}; > >> + > >> +struct scmi_vio_msg { > >> + struct virtio_scmi_request *request; > >> + union virtio_scmi_input *input; > >> + u8 completed; > >> +}; > >> + > >> +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, > >> + struct scmi_xfer *xfer) > >> +{ > >> + struct scatterlist sg_in; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + int rc; > >> + > >> + msg->completed = false; > >> + > >> + sg_init_one(&sg_in, msg->input, > >> + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); > >> + > >> + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); > >> + if (rc) > >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > >> + else > >> + virtqueue_kick(vioch->vqueue); > >> + > >> + return rc; > >> +} > >> + > >> +static void scmi_vio_complete_cb(struct virtqueue *vqueue) > >> +{ > >> + struct scmi_vio_channel *vioch = vqueue->priv; > >> + unsigned long iflags; > >> + unsigned int length; > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + > >> + do { > >> + struct scmi_xfer *xfer; > >> + > >> + virtqueue_disable_cb(vqueue); > >> + > >> + while ((xfer = virtqueue_get_buf(vqueue, &length))) { > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + u32 msg_hdr = > >> + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); > >> + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); > >> + > >> + if (!vioch->is_rx) { /* tx queue response */ > >> + msg->completed = true; > >> + xfer->rx.len = > >> + length - sizeof(msg->input->response); > >> + if (!xfer->hdr.poll_completion) > >> + scmi_rx_callback(vioch->cinfo, msg_hdr); > >> + continue; > >> + } > >> + > > > > So this is one of the places where SCMI transport seems not to accomodate so > > well the way virtio works as of now; it seems to me to be down to the fact that > > while virtio returns the message as a single chunk in one shot (since it carries > > the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads > > since the original shmem transport has the message size embedded in its header > > (which so has to be read at first), and so you have to stretch the usage of > > scmi_rx_callback introducing here some of its logic while leaving some other > > callbacks as empty dummies (like notifications callback): I think the transport > > should NOT have any knowledge of the structure/content of the stuff that it > > receives (like down below detecting which MSG_TYPE it is and acting accordingly). > > > > It clearly cannot work like it is now (so you have to play the tricks), but I > > think the answer is reviewing accordingly the SCMI transport layer, not trying > > to fooli it while polluting the transport layer with knowlegde of SCMI internals > > like message's structure. > > > > I think I should remove the xfer->[tr]x.buf manipulation from the patch. > In my understanding, this is one, or even the, major source of problems. > It would have saved a few kiB of memory and avoided copying between > scmi_xfer buffers and dedicated virtio queue buffers. But we can just > introduce dedicated virtio buffers and copy, in lieu of a better idea. > > As for the rest, if you don't like any of the scmi_xfer related logic to > be inside virtio.c, maybe we can move that logic into a new msg.c for > message-based communication, similar to shm.c for shmem-based > communication? > > The msg.c logic would then mostly be called from the callback ops, just > like shm.c. I think we should retain some transport private pointer in > scmi_xfer to help with potential concurrent or out-of-order message > processing by the virtio device (access to the private pointer can be > abstracted). Otherwise I'm concerned that in some callback ops it would > be difficult for the virtio transport to determine which message is > being addressed. > > If you agree on this, do you think that OpenSynergy could already start > implementing the aforementioned msg.c, or should we better wait for your > refactored interface? > > >> + /* rx queue - notification or delayed response */ > >> + switch (msg_type) { > >> + case MSG_TYPE_NOTIFICATION: > >> + xfer->rx.len = length - > >> + sizeof(msg->input->notification); > >> + xfer->rx.buf = msg->input->notification.data; > >> + break; > >> + case MSG_TYPE_DELAYED_RESP: > >> + xfer->rx.len = > >> + length - sizeof(msg->input->response); > >> + xfer->rx.buf = msg->input->response.data; > >> + break; > >> + default: > >> + dev_warn_once(vioch->cinfo->dev, > >> + "rx: unknown message_type %d\n", > >> + msg_type); > >> + scmi_vio_populate_vq_rx(vioch, xfer); > >> + continue; > >> + } > >> + > >> + scmi_rx_callback(vioch->cinfo, msg_hdr); > >> + scmi_vio_populate_vq_rx(vioch, xfer); > >> + } > >> + > >> + if (unlikely(virtqueue_is_broken(vqueue))) > >> + break; > >> + } while (!virtqueue_enable_cb(vqueue)); > >> + > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> +} > >> + > >> +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; > >> + > >> +static vq_callback_t *scmi_vio_complete_callbacks[] = { > >> + scmi_vio_complete_cb, > >> + scmi_vio_complete_cb > >> +}; > >> + > >> +static int scmi_vio_match_any_dev(struct device *dev, const void *data) > >> +{ > >> + (void)dev; > >> + (void)data; > >> + > >> + return 1; > >> +} > >> + > >> +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ > >> + > >> +static int virtio_link_supplier(struct device *dev) > >> +{ > >> + struct device *vdev = driver_find_device( > >> + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); > >> + > >> + if (!vdev) { > >> + dev_notice_once( > >> + dev, > >> + "Deferring probe after not finding a bound scmi-virtio device\n"); > >> + return -EPROBE_DEFER; > >> + } > >> + > >> + /* > >> + * Add plain device link for completeness. It might have no effect > >> + * beyond sysfs. > >> + */ > >> + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { > >> + put_device(vdev); > >> + dev_err(dev, "Adding link to supplier virtio device failed\n"); > >> + return -ECANCELED; > >> + } > >> + > >> + put_device(vdev); > >> + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); > >> +} > >> + > > > > I understand that here the attempt is to grab the virtio device upon > > which we are working and make it known to the core, while handling the > > case in which the MMIO transport is not ready with -EPROBE_DEFER, but I > > don't think such full-scale search with a dummy match function would be > > so well accepted upstream, and I don't understand really what's the > > point of the device_link_add() at the end...but maybe I'm missing > > something. > > The point of the device_link_add() is just to make the relation between > the devices apparent in sysfs. It should show up in sysfs as described here: > > https://lore.kernel.org/patchwork/patch/1245696/ > > > > > Moreover looking at other virtio drivers (not that I'm an expert > > though...) it seems very much to me that this SCMI Virtio driver is > > sort of built/probed/initialized upside-down respect how the other > > virtio drivers are made, and this seems to be down again to some missing > > support in our SCMI transport layer (...plus the need to avoid any > > virtio-related DT addition). > > > > That's the other thing that would be in my plan to rectify and unify in > > the SCMI transport layer (...at least in my hopes :D) > > OK. > > > > >> +static bool virtio_chan_available(struct device *dev, int idx) > >> +{ > >> + struct virtio_device *vdev; > >> + struct scmi_vio_channel **vioch; > >> + > >> + /* scmi-virtio doesn't support per-protocol channels */ > >> + if (is_scmi_protocol_device(dev)) > >> + return false; > >> + > >> + vdev = scmi_get_transport_info(dev); > >> + if (!vdev) > >> + return false; > >> + > >> + vioch = vdev->priv; > >> + if (!vioch) > >> + return false; > >> + > >> + return vioch[idx] && vioch[idx]->vqueue; > >> +} > >> + > >> +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, > >> + bool tx) > >> +{ > >> + struct virtio_device *vdev; > >> + struct scmi_vio_channel **vioch; > >> + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; > >> + > >> + /* scmi-virtio doesn't support per-protocol channels */ > >> + if (is_scmi_protocol_device(dev)) > >> + return -1; > >> + > >> + vdev = scmi_get_transport_info(dev); > >> + if (!vdev) > >> + return -1; > >> + > >> + vioch = vdev->priv; > >> + if (!vioch) { > >> + dev_err(dev, "Data from scmi-virtio probe not found\n"); > >> + return -1; > >> + } > >> + cinfo->transport_info = vioch[vioch_index]; > >> + vioch[vioch_index]->cinfo = cinfo; > >> + > >> + return 0; > >> +} > >> + > > > > Same goes, from my point of view, for these two channel related > > callbacks that now have to perform some very odd interactions and checks > > with the SCMI core (at least in my opinion). > > > > ATM the SCMI core will not call these callbacks before link_supplier() > was successful, so we could indeed omit the second and third early > return in each of the callbacks. It seemed more robust to leave them in, > though. > > In case it doesn't get obsoleted by interface changes, I will add some > additional comments to make clearer what is going on here. > > > Does the lack of support for multiple per-protocol channels derive from > > the lack of a way (now that the DT entry has gone) to determine the > > association between a channel and its user ? > > In my understanding that association could still be determined. > > Per-protocol channels were deliberately omitted, since we saw little > benefit for virtio. In my understanding, it would only help to avoid > that one protocol monopolizes the channel by sending many messages or > causing many notifications. > > > > >> +static int virtio_chan_free(int id, void *p, void *data) > >> +{ > >> + struct scmi_chan_info *cinfo = p; > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + > >> + if (vioch) { > >> + cinfo->transport_info = NULL; > >> + kfree(vioch); > >> + } > >> + > >> + scmi_free_channel(cinfo, data, id); > >> + return 0; > >> +} > >> + > >> +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, > >> + int *max_msg) > >> +{ > >> + struct scmi_vio_channel *vioch = base_cinfo->transport_info; > >> + > >> + *max_msg = virtqueue_get_vring_size(vioch->vqueue); > >> + > >> + /* Tx messages need multiple descriptors. */ > >> + if (tx) > >> + *max_msg /= DESCR_PER_TX_MSG; > >> + > >> + if (*max_msg > MSG_TOKEN_MAX) { > >> + dev_notice( > >> + base_cinfo->dev, > >> + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", > >> + MSG_TOKEN_MAX, *max_msg); > >> + *max_msg = MSG_TOKEN_MAX; > >> + } > >> + > >> + return 0; > >> +} > >> + > >> +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer, int max_msg_size) > >> +{ > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct scmi_vio_msg *msg; > >> + > >> + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); > >> + if (!msg) > >> + return -ENOMEM; > >> + > >> + xfer->extra_data = msg; > >> + > >> + if (vioch->is_rx) { > >> + int rc; > >> + unsigned long iflags; > >> + > >> + msg->input = devm_kzalloc(cinfo->dev, > >> + sizeof(*msg->input) + max_msg_size, > >> + GFP_KERNEL); > >> + if (!msg->input) > >> + return -ENOMEM; > >> + > >> + /* > >> + * xfer->rx.buf will be set to notification or delayed response > >> + * specific values in the receive callback, according to the > >> + * type of the received message. > >> + */ > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + rc = scmi_vio_populate_vq_rx(vioch, xfer); > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> + if (rc) > >> + return rc; > >> + } else { > >> + msg->request = > >> + devm_kzalloc(cinfo->dev, > >> + sizeof(*msg->request) + max_msg_size, > >> + GFP_KERNEL); > >> + if (!msg->request) > >> + return -ENOMEM; > >> + > >> + xfer->tx.buf = msg->request->data; > >> + > >> + msg->input = devm_kzalloc( > >> + cinfo->dev, sizeof(msg->input->response) + max_msg_size, > >> + GFP_KERNEL); > >> + if (!msg->input) > >> + return -ENOMEM; > >> + > >> + xfer->rx.buf = msg->input->response.data; > >> + } > >> + > >> + return 0; > >> +} > >> + > >> +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) > >> +{ > >> + struct scatterlist sg_out; > >> + struct scatterlist sg_in; > >> + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + unsigned long iflags; > >> + int rc; > >> + > >> + msg->completed = false; > >> + > >> + sg_init_one(&sg_out, msg->request, > >> + sizeof(*msg->request) + xfer->tx.len); > >> + sg_init_one(&sg_in, &msg->input->response, > >> + sizeof(msg->input->response) + xfer->rx.len); > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); > >> + if (rc) > >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > >> + else > >> + virtqueue_kick(vioch->vqueue); > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> + > >> + return rc; > >> +} > >> + > >> +static int virtio_send_message(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer) > >> +{ > >> + uint32_t hdr; > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct virtio_device *vdev = vioch->vqueue->vdev; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + > >> + hdr = pack_scmi_header(&xfer->hdr); > >> + > >> + msg->request->hdr = cpu_to_virtio32(vdev, hdr); > >> + > >> + return scmi_vio_send(vioch, xfer); > >> +} > >> + > >> +static void virtio_fetch_response(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer) > >> +{ > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + > >> + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, > >> + msg->input->response.status); > >> +} > >> + > >> +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, > >> + size_t max_len, struct scmi_xfer *xfer) > >> +{ > >> + (void)cinfo; > >> + (void)max_len; > >> + (void)xfer; > >> +} > >> + > >> +static void dummy_clear_channel(struct scmi_chan_info *cinfo) > >> +{ > >> + (void)cinfo; > >> +} > >> + > >> +static bool virtio_poll_done(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer) > >> +{ > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + unsigned long iflags; > >> + bool completed; > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + completed = msg->completed; > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> + > >> + return completed; > >> +} > >> + > >> +static const struct scmi_transport_ops scmi_virtio_ops = { > >> + .link_supplier = virtio_link_supplier, > >> + .chan_available = virtio_chan_available, > >> + .chan_setup = virtio_chan_setup, > >> + .chan_free = virtio_chan_free, > >> + .get_max_msg = virtio_get_max_msg, > >> + .send_message = virtio_send_message, > >> + .fetch_response = virtio_fetch_response, > >> + .fetch_notification = dummy_fetch_notification, > >> + .clear_channel = dummy_clear_channel, > >> + .poll_done = virtio_poll_done, > >> + .xfer_init_buffers = virtio_xfer_init_buffers, > >> +}; > >> + > >> +const struct scmi_desc scmi_virtio_desc = { > >> + .ops = &scmi_virtio_ops, > >> + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ > >> + .max_msg = 0, /* overridden by virtio_get_max_msg() */ > >> + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, > >> +}; > >> + > >> +static int scmi_vio_probe(struct virtio_device *vdev) > >> +{ > >> + struct device *dev = &vdev->dev; > >> + struct scmi_vio_channel **vioch; > >> + bool have_vq_rx; > >> + int vq_cnt; > >> + int i; > >> + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; > >> + > >> + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), > >> + GFP_KERNEL); > >> + if (!vioch) > >> + return -ENOMEM; > >> + > >> + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); > >> + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; > >> + > >> + for (i = 0; i < vq_cnt; i++) { > >> + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); > >> + if (!vioch[i]) > >> + return -ENOMEM; > >> + } > >> + > >> + if (have_vq_rx) > >> + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; > >> + > >> + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, > >> + scmi_vio_vqueue_names, NULL)) { > >> + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); > >> + return -1; > >> + } > >> + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); > >> + > >> + for (i = 0; i < vq_cnt; i++) { > >> + spin_lock_init(&vioch[i]->lock); > >> + vioch[i]->vqueue = vqs[i]; > >> + vioch[i]->vqueue->priv = vioch[i]; > >> + } > >> + > >> + vdev->priv = vioch; > >> + > > > > As mentioned before, in other Virtio drivers this is the place where we > > should register somehow this transports/channels with the core (before > > marking the device ready), and may be the place where we can just share > > the core device for this tranport with the SCMI core without then having > > to play the above link_supplier chan_available/setup dance. > > > > But again I still have to fully tries to change the SCMI transport > > interface at this point. > > > > I've not really gone into all the details across all series because I > > see the above issues/limitations in the driver/SCMI_transport as sort of > > blocking. > > I think some of the patches will no longer be needed in the future. I > would expect at least the following to go away: > > [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op > > > > > I'll let you now when I'll have something sensible on the SCMI core to > > share for better accomodate this transport. > > > > Thanks > > > > Cristian > > > > > >> + virtio_device_ready(vdev); > >> + > >> + return 0; > >> +} > >> + > >> +static unsigned int features[] = { > >> + VIRTIO_SCMI_F_P2A_CHANNELS, > >> +}; > >> + > >> +static const struct virtio_device_id id_table[] = { > >> + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, > >> + { 0 } > >> +}; > >> + > >> +static struct virtio_driver virtio_scmi_driver = { > >> + .driver.name = "scmi-virtio", > >> + .driver.owner = THIS_MODULE, > >> + .feature_table = features, > >> + .feature_table_size = ARRAY_SIZE(features), > >> + .id_table = id_table, > >> + .probe = scmi_vio_probe, > >> +}; > >> + > >> +int __init virtio_scmi_init(void) > >> +{ > >> + return register_virtio_driver(&virtio_scmi_driver); > >> +} > >> + > >> +void __exit virtio_scmi_exit(void) > >> +{ > >> + unregister_virtio_driver(&virtio_scmi_driver); > >> +} > >> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h > >> index b052355ac7a3..57d233c02720 100644 > >> --- a/include/uapi/linux/virtio_ids.h > >> +++ b/include/uapi/linux/virtio_ids.h > >> @@ -48,5 +48,6 @@ > >> #define VIRTIO_ID_FS 26 /* virtio filesystem */ > >> #define VIRTIO_ID_PMEM 27 /* virtio pmem */ > >> #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ > >> +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ > >> > >> #endif /* _LINUX_VIRTIO_IDS_H */ > >> diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h > >> new file mode 100644 > >> index 000000000000..9f21b3dbbfe2 > >> --- /dev/null > >> +++ b/include/uapi/linux/virtio_scmi.h > >> @@ -0,0 +1,41 @@ > >> +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ > >> +/* > >> + * Copyright (C) 2020 OpenSynergy GmbH > >> + */ > >> + > >> +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H > >> +#define _UAPI_LINUX_VIRTIO_SCMI_H > >> + > >> +#include <linux/virtio_types.h> > >> + > >> +/* Feature bits */ > >> + > >> +/* Device implements some SCMI notifications, or delayed responses. */ > >> +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 > >> + > >> +/* Device implements any SCMI statistics shared memory region */ > >> +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 > >> + > >> +/* Virtqueues */ > >> + > >> +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ > >> +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ > >> +#define VIRTIO_SCMI_VQ_MAX_CNT 2 > >> + > >> +struct virtio_scmi_request { > >> + __virtio32 hdr; > >> + __u8 data[]; > >> +}; > >> + > >> +struct virtio_scmi_response { > >> + __virtio32 hdr; > >> + __virtio32 status; > >> + __u8 data[]; > >> +}; > >> + > >> +struct virtio_scmi_notification { > >> + __virtio32 hdr; > >> + __u8 data[]; > >> +}; > >> + > >> +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ > >> -- > >> 2.25.1 > >> > >> > >> _______________________________________________ > >> linux-arm-kernel mailing list > >> linux-arm-kernel@lists.infradead.org > >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > ^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2020-11-17 21:20 ` Cristian Marussi 0 siblings, 0 replies; 34+ messages in thread From: Cristian Marussi @ 2020-11-17 21:20 UTC (permalink / raw) To: Peter Hilber Cc: devicetree, mikhail.golubev, souvik.chakravarty, Igor Skalkin, jean-philippe, Jason Wang, Michael S. Tsirkin, linux-kernel, virtualization, Rob Herring, anton.yakovlev, sudeep.holla, alex.bennee, virtio-dev, linux-arm-kernel On Thu, Nov 12, 2020 at 11:57:09AM +0100, Peter Hilber wrote: > On 10.11.20 22:32, Cristian Marussi wrote: > > Hi Peter/Igor, > > > > I went through this series while trying to grasp a bit more of all the > > virtio inner workings and needs and I'll leave a few detailed comments > > down below. > > > > In short at first, I think I can say that there are a couple of places > > where I noticed you had to play all sort of tricks to fit into the current > > SCMI transport layer frameowrk or to avoid adding DT references. > > > > It's clear that we'll have to somehow extend/fix/abstract the SCMI > > transport layer (in my opinion), because I doubt that some of the tricks > > you played would be well received for upstream ever (...but it's worth > > noting it's not up to me saying the last...) > > > > So in these days I'm trying to play with this series and the SCMI > > stack to see if I can extend it in a more sensible way to fit some of > > the observations I make down below (now transport core layer is pretty > > much shmem/mailbox oriented)...still nothing to share anyway as of now. > > Hi Cristian, > > thanks for your review. I agree that some changes to the patch series > are needed to move it beyond RFC state. Please see individual responses > below. > > Best regards, > > Peter > Hi Peter, apologize for the delay in the answer. Yesterday I experimenetd with some changes only related to the initialization and probing in order to expose a better interface from the SCMI core and limit the need for tricks I was talking about. Nothing about the message dispatching still. I tested minimally hacking brutally virtio-mmio to fake the presence of an SCMI device so as to verify I had not done worst :D For this reason I'll send you at first such patches offlist just for you to verify init/probing and timings are still ok and get your general feedback; patches should apply cleanly on your codebase, but they are just an initial experiment so they are not even cleanly split: they both add new core interfaces and use it in your code removing unneded code from virtio-scmi (hopefully). So, yeah, they are as of now a big ball of mud...just for you to experiment or point out if I missed something basic. Sorry for that. Tomorrow I'll get back to look at the rest of the series and the message dispatching to have a better idea what to do. Thanks Cristian > > > > > > On Thu, Nov 05, 2020 at 10:21:16PM +0100, Peter Hilber wrote: > >> From: Igor Skalkin <igor.skalkin@opensynergy.com> > >> > >> This transport enables accessing an SCMI platform as a virtio device. > >> > >> Implement an SCMI virtio driver according to the virtio SCMI device spec > >> patch v5 [1]. Virtio device id 32 has been reserved for the SCMI device > >> [2]. > >> > >> The virtio transport has one tx channel (virtio cmdq, A2P channel) and > >> at most one rx channel (virtio eventq, P2A channel). > >> > >> The following feature bit defined in [1] is not implemented: > >> VIRTIO_SCMI_F_SHARED_MEMORY. > >> > >> After the preparatory patches, implement the virtio transport as > >> paraphrased: > >> > >> Only support a single arm-scmi device (which is consistent with the SCMI > >> spec). Call scmi-virtio init from arm-scmi module init. During the > >> arm-scmi probing, link to the first probed scmi-virtio device. Defer > >> arm-scmi probing if no scmi-virtio device is bound yet. > >> > >> Use the scmi_xfer tx/rx buffers for data exchange with the virtio device > >> in order to avoid redundant maintenance of additional buffers. Allocate > >> the buffers in the SCMI transport, and prepend room for a small header > >> used by the virtio transport to the tx/rx buffers. > >> > >> For simplicity, restrict the number of messages which can be pending > >> simultaneously according to the virtqueue capacity. (The virtqueue sizes > >> are negotiated with the virtio device.) > >> > >> As soon as rx channel message buffers are allocated or have been read > >> out by the arm-scmi driver, feed them to the virtio device. > >> > >> Since some virtio devices may not have the short response time exhibited > >> by SCMI platforms using other transports, set a generous response > >> timeout. > >> > >> Limitations: > >> > >> Do not adjust the other SCMI timeouts for delayed response and polling > >> for now, since these timeouts are only relevant in special cases which > >> are not yet deemed relevant for this transport. > >> > >> To do (as discussed in the cover letter): > >> > >> - Avoid re-use of buffers still being used by the virtio device on > >> timeouts. > >> > >> - Avoid race conditions on receiving messages during/after channel free > >> on driver probe failure or remove. > >> > >> [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > >> [2] https://www.oasis-open.org/committees/ballot.php?id=3496 > >> > >> Co-developed-by: Peter Hilber <peter.hilber@opensynergy.com> > >> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> > >> Signed-off-by: Igor Skalkin <igor.skalkin@opensynergy.com> > >> --- > >> MAINTAINERS | 1 + > >> drivers/firmware/Kconfig | 12 +- > >> drivers/firmware/arm_scmi/Makefile | 1 + > >> drivers/firmware/arm_scmi/common.h | 14 + > >> drivers/firmware/arm_scmi/driver.c | 11 + > >> drivers/firmware/arm_scmi/virtio.c | 493 +++++++++++++++++++++++++++++ > >> include/uapi/linux/virtio_ids.h | 1 + > >> include/uapi/linux/virtio_scmi.h | 41 +++ > >> 8 files changed, 573 insertions(+), 1 deletion(-) > >> create mode 100644 drivers/firmware/arm_scmi/virtio.c > >> create mode 100644 include/uapi/linux/virtio_scmi.h > >> > >> diff --git a/MAINTAINERS b/MAINTAINERS > >> index deaafb617361..8df73d6ddfc1 100644 > >> --- a/MAINTAINERS > >> +++ b/MAINTAINERS > >> @@ -16772,6 +16772,7 @@ F: drivers/firmware/arm_scpi.c > >> F: drivers/reset/reset-scmi.c > >> F: include/linux/sc[mp]i_protocol.h > >> F: include/trace/events/scmi.h > >> +F: include/uapi/linux/virtio_scmi.h > >> > >> SYSTEM RESET/SHUTDOWN DRIVERS > >> M: Sebastian Reichel <sre@kernel.org> > >> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig > >> index bdde51adb267..c4bdd84f7405 100644 > >> --- a/drivers/firmware/Kconfig > >> +++ b/drivers/firmware/Kconfig > >> @@ -9,7 +9,7 @@ menu "Firmware Drivers" > >> config ARM_SCMI_PROTOCOL > >> tristate "ARM System Control and Management Interface (SCMI) Message Protocol" > >> depends on ARM || ARM64 || COMPILE_TEST > >> - depends on ARM_SCMI_HAVE_SHMEM > >> + depends on ARM_SCMI_HAVE_SHMEM || VIRTIO_SCMI > >> help > >> ARM System Control and Management Interface (SCMI) protocol is a > >> set of operating system-independent software interfaces that are > >> @@ -34,6 +34,16 @@ config ARM_SCMI_HAVE_SHMEM > >> This declares whether a shared memory based transport for SCMI is > >> available. > >> > >> +config VIRTIO_SCMI > >> + bool "Virtio transport for SCMI" > >> + default n > >> + depends on VIRTIO > >> + help > >> + This enables the virtio based transport for SCMI. > >> + > >> + If you want to use the ARM SCMI protocol between the virtio guest and > >> + a host providing a virtio SCMI device, answer Y. > >> + > >> config ARM_SCMI_POWER_DOMAIN > >> tristate "SCMI power domain driver" > >> depends on ARM_SCMI_PROTOCOL || (COMPILE_TEST && OF) > >> diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile > >> index 3cc7fa40a464..25caea5e1969 100644 > >> --- a/drivers/firmware/arm_scmi/Makefile > >> +++ b/drivers/firmware/arm_scmi/Makefile > >> @@ -4,6 +4,7 @@ scmi-driver-y = driver.o notify.o > >> scmi-transport-$(CONFIG_ARM_SCMI_HAVE_SHMEM) = shmem.o > >> scmi-transport-$(CONFIG_MAILBOX) += mailbox.o > >> scmi-transport-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smc.o > >> +scmi-transport-$(CONFIG_VIRTIO_SCMI) += virtio.o > >> scmi-protocols-y = base.o clock.o perf.o power.o reset.o sensors.o system.o > >> scmi-module-objs := $(scmi-bus-y) $(scmi-driver-y) $(scmi-protocols-y) \ > >> $(scmi-transport-y) > >> diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h > >> index 13c9ac176b23..b46dfe84e78b 100644 > >> --- a/drivers/firmware/arm_scmi/common.h > >> +++ b/drivers/firmware/arm_scmi/common.h > >> @@ -165,6 +165,17 @@ int scmi_base_protocol_init(struct scmi_handle *h); > >> int __init scmi_bus_init(void); > >> void __exit scmi_bus_exit(void); > >> > >> +#ifdef CONFIG_VIRTIO_SCMI > >> +int __init virtio_scmi_init(void); > >> +void __exit virtio_scmi_exit(void); > >> +#else > >> +static inline int __init virtio_scmi_init(void) > >> +{ > >> + return 0; > >> +} > >> +#define virtio_scmi_exit() do { } while (0) > >> +#endif > >> + > >> #define DECLARE_SCMI_REGISTER_UNREGISTER(func) \ > >> int __init scmi_##func##_register(void); \ > >> void __exit scmi_##func##_unregister(void) > >> @@ -263,6 +274,9 @@ extern const struct scmi_desc scmi_mailbox_desc; > >> #ifdef CONFIG_HAVE_ARM_SMCCC > >> extern const struct scmi_desc scmi_smc_desc; > >> #endif > >> +#ifdef CONFIG_VIRTIO_SCMI > >> +extern const struct scmi_desc scmi_virtio_desc; > >> +#endif > >> > >> int scmi_set_transport_info(struct device *dev, void *transport_info); > >> void *scmi_get_transport_info(struct device *dev); > >> diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c > >> index 244141e45e88..923ba526e829 100644 > >> --- a/drivers/firmware/arm_scmi/driver.c > >> +++ b/drivers/firmware/arm_scmi/driver.c > >> @@ -996,6 +996,9 @@ static const struct of_device_id scmi_of_match[] = { > >> #endif > >> #ifdef CONFIG_HAVE_ARM_SMCCC_DISCOVERY > >> { .compatible = "arm,scmi-smc", .data = &scmi_smc_desc}, > >> +#endif > >> +#ifdef CONFIG_VIRTIO_SCMI > >> + { .compatible = "arm,scmi-virtio", .data = &scmi_virtio_desc}, > >> #endif > >> { /* Sentinel */ }, > >> }; > >> @@ -1014,8 +1017,14 @@ static struct platform_driver scmi_driver = { > >> > >> static int __init scmi_driver_init(void) > >> { > >> + int ret; > >> + > >> scmi_bus_init(); > >> > >> + ret = virtio_scmi_init(); > >> + if (ret) > >> + return ret; > >> + > >> scmi_clock_register(); > >> scmi_perf_register(); > >> scmi_power_register(); > >> @@ -1038,6 +1047,8 @@ static void __exit scmi_driver_exit(void) > >> scmi_sensors_unregister(); > >> scmi_system_unregister(); > >> > >> + virtio_scmi_exit(); > >> + > > > > These virtio init/exit functions which are called by the platform driver > > init code in fact introduce a very transport specific non-general piece > > of code in the common init path: this is one of the things I'd like to > > abstract better, so that any available transport can just register itself > > with the core in the same way and be initialized if needed in an uniform > > way, without having to extend the core driver init. > > (not saying that now it is not anyway already done for other > > matters....I'd like to remove those too in the future.) > > Agree. > > > > > > >> platform_driver_unregister(&scmi_driver); > >> } > >> module_exit(scmi_driver_exit); > >> diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c > >> new file mode 100644 > >> index 000000000000..f70aa72f34f1 > >> --- /dev/null > >> +++ b/drivers/firmware/arm_scmi/virtio.c > >> @@ -0,0 +1,493 @@ > >> +// SPDX-License-Identifier: GPL-2.0 > >> +/* > >> + * Virtio Transport driver for Arm System Control and Management Interface > >> + * (SCMI). > >> + * > >> + * Copyright (C) 2020 OpenSynergy. > >> + */ > >> + > >> +/** > >> + * DOC: Theory of Operation > >> + * > >> + * The scmi-virtio transport implements a driver for the virtio SCMI device > >> + * proposed in virtio spec patch v5[1]. > >> + * > >> + * There is one tx channel (virtio cmdq, A2P channel) and at most one rx > >> + * channel (virtio eventq, P2A channel). Each channel is implemented through a > >> + * virtqueue. Access to each virtqueue is protected by a spinlock. > >> + * > >> + * This SCMI transport uses the scmi_xfer tx/rx buffers for data exchange with > >> + * the virtio device to avoid maintenance of additional buffers. > >> + * > >> + * [1] https://lists.oasis-open.org/archives/virtio-comment/202005/msg00096.html > >> + */ > >> + > >> +#include <linux/errno.h> > >> +#include <linux/of.h> > >> +#include <linux/of_platform.h> > >> +#include <linux/platform_device.h> > >> +#include <linux/module.h> > >> +#include <linux/slab.h> > >> +#include <linux/virtio.h> > >> +#include <linux/virtio_config.h> > >> +#include <uapi/linux/virtio_ids.h> > >> +#include <uapi/linux/virtio_scmi.h> > >> + > >> +#include "common.h" > >> + > >> +#define VIRTIO_SCMI_MAX_MSG_SIZE 128 /* Value may be increased. */ > >> +#define DESCR_PER_TX_MSG 2 > >> + > >> +struct scmi_vio_channel { > >> + spinlock_t lock; > >> + struct virtqueue *vqueue; > >> + struct scmi_chan_info *cinfo; > >> + u8 is_rx; > >> +}; > >> + > >> +union virtio_scmi_input { > >> + __virtio32 hdr; > >> + struct virtio_scmi_response response; > >> + struct virtio_scmi_notification notification; > >> +}; > >> + > >> +struct scmi_vio_msg { > >> + struct virtio_scmi_request *request; > >> + union virtio_scmi_input *input; > >> + u8 completed; > >> +}; > >> + > >> +static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, > >> + struct scmi_xfer *xfer) > >> +{ > >> + struct scatterlist sg_in; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + int rc; > >> + > >> + msg->completed = false; > >> + > >> + sg_init_one(&sg_in, msg->input, > >> + sizeof(*msg->input) + VIRTIO_SCMI_MAX_MSG_SIZE); > >> + > >> + rc = virtqueue_add_inbuf(vioch->vqueue, &sg_in, 1, xfer, GFP_ATOMIC); > >> + if (rc) > >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > >> + else > >> + virtqueue_kick(vioch->vqueue); > >> + > >> + return rc; > >> +} > >> + > >> +static void scmi_vio_complete_cb(struct virtqueue *vqueue) > >> +{ > >> + struct scmi_vio_channel *vioch = vqueue->priv; > >> + unsigned long iflags; > >> + unsigned int length; > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + > >> + do { > >> + struct scmi_xfer *xfer; > >> + > >> + virtqueue_disable_cb(vqueue); > >> + > >> + while ((xfer = virtqueue_get_buf(vqueue, &length))) { > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + u32 msg_hdr = > >> + virtio32_to_cpu(vqueue->vdev, msg->input->hdr); > >> + u8 msg_type = MSG_XTRACT_TYPE(msg_hdr); > >> + > >> + if (!vioch->is_rx) { /* tx queue response */ > >> + msg->completed = true; > >> + xfer->rx.len = > >> + length - sizeof(msg->input->response); > >> + if (!xfer->hdr.poll_completion) > >> + scmi_rx_callback(vioch->cinfo, msg_hdr); > >> + continue; > >> + } > >> + > > > > So this is one of the places where SCMI transport seems not to accomodate so > > well the way virtio works as of now; it seems to me to be down to the fact that > > while virtio returns the message as a single chunk in one shot (since it carries > > the lenght 'out-of-band'), the SCMI transport expects a sort of 2 stage reads > > since the original shmem transport has the message size embedded in its header > > (which so has to be read at first), and so you have to stretch the usage of > > scmi_rx_callback introducing here some of its logic while leaving some other > > callbacks as empty dummies (like notifications callback): I think the transport > > should NOT have any knowledge of the structure/content of the stuff that it > > receives (like down below detecting which MSG_TYPE it is and acting accordingly). > > > > It clearly cannot work like it is now (so you have to play the tricks), but I > > think the answer is reviewing accordingly the SCMI transport layer, not trying > > to fooli it while polluting the transport layer with knowlegde of SCMI internals > > like message's structure. > > > > I think I should remove the xfer->[tr]x.buf manipulation from the patch. > In my understanding, this is one, or even the, major source of problems. > It would have saved a few kiB of memory and avoided copying between > scmi_xfer buffers and dedicated virtio queue buffers. But we can just > introduce dedicated virtio buffers and copy, in lieu of a better idea. > > As for the rest, if you don't like any of the scmi_xfer related logic to > be inside virtio.c, maybe we can move that logic into a new msg.c for > message-based communication, similar to shm.c for shmem-based > communication? > > The msg.c logic would then mostly be called from the callback ops, just > like shm.c. I think we should retain some transport private pointer in > scmi_xfer to help with potential concurrent or out-of-order message > processing by the virtio device (access to the private pointer can be > abstracted). Otherwise I'm concerned that in some callback ops it would > be difficult for the virtio transport to determine which message is > being addressed. > > If you agree on this, do you think that OpenSynergy could already start > implementing the aforementioned msg.c, or should we better wait for your > refactored interface? > > >> + /* rx queue - notification or delayed response */ > >> + switch (msg_type) { > >> + case MSG_TYPE_NOTIFICATION: > >> + xfer->rx.len = length - > >> + sizeof(msg->input->notification); > >> + xfer->rx.buf = msg->input->notification.data; > >> + break; > >> + case MSG_TYPE_DELAYED_RESP: > >> + xfer->rx.len = > >> + length - sizeof(msg->input->response); > >> + xfer->rx.buf = msg->input->response.data; > >> + break; > >> + default: > >> + dev_warn_once(vioch->cinfo->dev, > >> + "rx: unknown message_type %d\n", > >> + msg_type); > >> + scmi_vio_populate_vq_rx(vioch, xfer); > >> + continue; > >> + } > >> + > >> + scmi_rx_callback(vioch->cinfo, msg_hdr); > >> + scmi_vio_populate_vq_rx(vioch, xfer); > >> + } > >> + > >> + if (unlikely(virtqueue_is_broken(vqueue))) > >> + break; > >> + } while (!virtqueue_enable_cb(vqueue)); > >> + > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> +} > >> + > >> +static const char *const scmi_vio_vqueue_names[] = { "tx", "rx" }; > >> + > >> +static vq_callback_t *scmi_vio_complete_callbacks[] = { > >> + scmi_vio_complete_cb, > >> + scmi_vio_complete_cb > >> +}; > >> + > >> +static int scmi_vio_match_any_dev(struct device *dev, const void *data) > >> +{ > >> + (void)dev; > >> + (void)data; > >> + > >> + return 1; > >> +} > >> + > >> +static struct virtio_driver virtio_scmi_driver; /* Forward declaration */ > >> + > >> +static int virtio_link_supplier(struct device *dev) > >> +{ > >> + struct device *vdev = driver_find_device( > >> + &virtio_scmi_driver.driver, NULL, NULL, scmi_vio_match_any_dev); > >> + > >> + if (!vdev) { > >> + dev_notice_once( > >> + dev, > >> + "Deferring probe after not finding a bound scmi-virtio device\n"); > >> + return -EPROBE_DEFER; > >> + } > >> + > >> + /* > >> + * Add plain device link for completeness. It might have no effect > >> + * beyond sysfs. > >> + */ > >> + if (!device_link_add(dev, vdev, DL_FLAG_AUTOREMOVE_CONSUMER)) { > >> + put_device(vdev); > >> + dev_err(dev, "Adding link to supplier virtio device failed\n"); > >> + return -ECANCELED; > >> + } > >> + > >> + put_device(vdev); > >> + return scmi_set_transport_info(dev, dev_to_virtio(vdev)); > >> +} > >> + > > > > I understand that here the attempt is to grab the virtio device upon > > which we are working and make it known to the core, while handling the > > case in which the MMIO transport is not ready with -EPROBE_DEFER, but I > > don't think such full-scale search with a dummy match function would be > > so well accepted upstream, and I don't understand really what's the > > point of the device_link_add() at the end...but maybe I'm missing > > something. > > The point of the device_link_add() is just to make the relation between > the devices apparent in sysfs. It should show up in sysfs as described here: > > https://lore.kernel.org/patchwork/patch/1245696/ > > > > > Moreover looking at other virtio drivers (not that I'm an expert > > though...) it seems very much to me that this SCMI Virtio driver is > > sort of built/probed/initialized upside-down respect how the other > > virtio drivers are made, and this seems to be down again to some missing > > support in our SCMI transport layer (...plus the need to avoid any > > virtio-related DT addition). > > > > That's the other thing that would be in my plan to rectify and unify in > > the SCMI transport layer (...at least in my hopes :D) > > OK. > > > > >> +static bool virtio_chan_available(struct device *dev, int idx) > >> +{ > >> + struct virtio_device *vdev; > >> + struct scmi_vio_channel **vioch; > >> + > >> + /* scmi-virtio doesn't support per-protocol channels */ > >> + if (is_scmi_protocol_device(dev)) > >> + return false; > >> + > >> + vdev = scmi_get_transport_info(dev); > >> + if (!vdev) > >> + return false; > >> + > >> + vioch = vdev->priv; > >> + if (!vioch) > >> + return false; > >> + > >> + return vioch[idx] && vioch[idx]->vqueue; > >> +} > >> + > >> +static int virtio_chan_setup(struct scmi_chan_info *cinfo, struct device *dev, > >> + bool tx) > >> +{ > >> + struct virtio_device *vdev; > >> + struct scmi_vio_channel **vioch; > >> + int vioch_index = tx ? VIRTIO_SCMI_VQ_TX : VIRTIO_SCMI_VQ_RX; > >> + > >> + /* scmi-virtio doesn't support per-protocol channels */ > >> + if (is_scmi_protocol_device(dev)) > >> + return -1; > >> + > >> + vdev = scmi_get_transport_info(dev); > >> + if (!vdev) > >> + return -1; > >> + > >> + vioch = vdev->priv; > >> + if (!vioch) { > >> + dev_err(dev, "Data from scmi-virtio probe not found\n"); > >> + return -1; > >> + } > >> + cinfo->transport_info = vioch[vioch_index]; > >> + vioch[vioch_index]->cinfo = cinfo; > >> + > >> + return 0; > >> +} > >> + > > > > Same goes, from my point of view, for these two channel related > > callbacks that now have to perform some very odd interactions and checks > > with the SCMI core (at least in my opinion). > > > > ATM the SCMI core will not call these callbacks before link_supplier() > was successful, so we could indeed omit the second and third early > return in each of the callbacks. It seemed more robust to leave them in, > though. > > In case it doesn't get obsoleted by interface changes, I will add some > additional comments to make clearer what is going on here. > > > Does the lack of support for multiple per-protocol channels derive from > > the lack of a way (now that the DT entry has gone) to determine the > > association between a channel and its user ? > > In my understanding that association could still be determined. > > Per-protocol channels were deliberately omitted, since we saw little > benefit for virtio. In my understanding, it would only help to avoid > that one protocol monopolizes the channel by sending many messages or > causing many notifications. > > > > >> +static int virtio_chan_free(int id, void *p, void *data) > >> +{ > >> + struct scmi_chan_info *cinfo = p; > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + > >> + if (vioch) { > >> + cinfo->transport_info = NULL; > >> + kfree(vioch); > >> + } > >> + > >> + scmi_free_channel(cinfo, data, id); > >> + return 0; > >> +} > >> + > >> +static int virtio_get_max_msg(bool tx, struct scmi_chan_info *base_cinfo, > >> + int *max_msg) > >> +{ > >> + struct scmi_vio_channel *vioch = base_cinfo->transport_info; > >> + > >> + *max_msg = virtqueue_get_vring_size(vioch->vqueue); > >> + > >> + /* Tx messages need multiple descriptors. */ > >> + if (tx) > >> + *max_msg /= DESCR_PER_TX_MSG; > >> + > >> + if (*max_msg > MSG_TOKEN_MAX) { > >> + dev_notice( > >> + base_cinfo->dev, > >> + "Only %ld messages can be pending simultaneously, while the virtqueue could hold %d\n", > >> + MSG_TOKEN_MAX, *max_msg); > >> + *max_msg = MSG_TOKEN_MAX; > >> + } > >> + > >> + return 0; > >> +} > >> + > >> +static int virtio_xfer_init_buffers(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer, int max_msg_size) > >> +{ > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct scmi_vio_msg *msg; > >> + > >> + msg = devm_kzalloc(cinfo->dev, sizeof(*msg), GFP_KERNEL); > >> + if (!msg) > >> + return -ENOMEM; > >> + > >> + xfer->extra_data = msg; > >> + > >> + if (vioch->is_rx) { > >> + int rc; > >> + unsigned long iflags; > >> + > >> + msg->input = devm_kzalloc(cinfo->dev, > >> + sizeof(*msg->input) + max_msg_size, > >> + GFP_KERNEL); > >> + if (!msg->input) > >> + return -ENOMEM; > >> + > >> + /* > >> + * xfer->rx.buf will be set to notification or delayed response > >> + * specific values in the receive callback, according to the > >> + * type of the received message. > >> + */ > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + rc = scmi_vio_populate_vq_rx(vioch, xfer); > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> + if (rc) > >> + return rc; > >> + } else { > >> + msg->request = > >> + devm_kzalloc(cinfo->dev, > >> + sizeof(*msg->request) + max_msg_size, > >> + GFP_KERNEL); > >> + if (!msg->request) > >> + return -ENOMEM; > >> + > >> + xfer->tx.buf = msg->request->data; > >> + > >> + msg->input = devm_kzalloc( > >> + cinfo->dev, sizeof(msg->input->response) + max_msg_size, > >> + GFP_KERNEL); > >> + if (!msg->input) > >> + return -ENOMEM; > >> + > >> + xfer->rx.buf = msg->input->response.data; > >> + } > >> + > >> + return 0; > >> +} > >> + > >> +static int scmi_vio_send(struct scmi_vio_channel *vioch, struct scmi_xfer *xfer) > >> +{ > >> + struct scatterlist sg_out; > >> + struct scatterlist sg_in; > >> + struct scatterlist *sgs[DESCR_PER_TX_MSG] = { &sg_out, &sg_in }; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + unsigned long iflags; > >> + int rc; > >> + > >> + msg->completed = false; > >> + > >> + sg_init_one(&sg_out, msg->request, > >> + sizeof(*msg->request) + xfer->tx.len); > >> + sg_init_one(&sg_in, &msg->input->response, > >> + sizeof(msg->input->response) + xfer->rx.len); > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + rc = virtqueue_add_sgs(vioch->vqueue, sgs, 1, 1, xfer, GFP_ATOMIC); > >> + if (rc) > >> + dev_err(vioch->cinfo->dev, "%s() rc=%d\n", __func__, rc); > >> + else > >> + virtqueue_kick(vioch->vqueue); > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> + > >> + return rc; > >> +} > >> + > >> +static int virtio_send_message(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer) > >> +{ > >> + uint32_t hdr; > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct virtio_device *vdev = vioch->vqueue->vdev; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + > >> + hdr = pack_scmi_header(&xfer->hdr); > >> + > >> + msg->request->hdr = cpu_to_virtio32(vdev, hdr); > >> + > >> + return scmi_vio_send(vioch, xfer); > >> +} > >> + > >> +static void virtio_fetch_response(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer) > >> +{ > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + > >> + xfer->hdr.status = virtio32_to_cpu(vioch->vqueue->vdev, > >> + msg->input->response.status); > >> +} > >> + > >> +static void dummy_fetch_notification(struct scmi_chan_info *cinfo, > >> + size_t max_len, struct scmi_xfer *xfer) > >> +{ > >> + (void)cinfo; > >> + (void)max_len; > >> + (void)xfer; > >> +} > >> + > >> +static void dummy_clear_channel(struct scmi_chan_info *cinfo) > >> +{ > >> + (void)cinfo; > >> +} > >> + > >> +static bool virtio_poll_done(struct scmi_chan_info *cinfo, > >> + struct scmi_xfer *xfer) > >> +{ > >> + struct scmi_vio_channel *vioch = cinfo->transport_info; > >> + struct scmi_vio_msg *msg = xfer->extra_data; > >> + unsigned long iflags; > >> + bool completed; > >> + > >> + spin_lock_irqsave(&vioch->lock, iflags); > >> + completed = msg->completed; > >> + spin_unlock_irqrestore(&vioch->lock, iflags); > >> + > >> + return completed; > >> +} > >> + > >> +static const struct scmi_transport_ops scmi_virtio_ops = { > >> + .link_supplier = virtio_link_supplier, > >> + .chan_available = virtio_chan_available, > >> + .chan_setup = virtio_chan_setup, > >> + .chan_free = virtio_chan_free, > >> + .get_max_msg = virtio_get_max_msg, > >> + .send_message = virtio_send_message, > >> + .fetch_response = virtio_fetch_response, > >> + .fetch_notification = dummy_fetch_notification, > >> + .clear_channel = dummy_clear_channel, > >> + .poll_done = virtio_poll_done, > >> + .xfer_init_buffers = virtio_xfer_init_buffers, > >> +}; > >> + > >> +const struct scmi_desc scmi_virtio_desc = { > >> + .ops = &scmi_virtio_ops, > >> + .max_rx_timeout_ms = 60000, /* for non-realtime virtio devices */ > >> + .max_msg = 0, /* overridden by virtio_get_max_msg() */ > >> + .max_msg_size = VIRTIO_SCMI_MAX_MSG_SIZE, > >> +}; > >> + > >> +static int scmi_vio_probe(struct virtio_device *vdev) > >> +{ > >> + struct device *dev = &vdev->dev; > >> + struct scmi_vio_channel **vioch; > >> + bool have_vq_rx; > >> + int vq_cnt; > >> + int i; > >> + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; > >> + > >> + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), > >> + GFP_KERNEL); > >> + if (!vioch) > >> + return -ENOMEM; > >> + > >> + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); > >> + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; > >> + > >> + for (i = 0; i < vq_cnt; i++) { > >> + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); > >> + if (!vioch[i]) > >> + return -ENOMEM; > >> + } > >> + > >> + if (have_vq_rx) > >> + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; > >> + > >> + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, > >> + scmi_vio_vqueue_names, NULL)) { > >> + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); > >> + return -1; > >> + } > >> + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); > >> + > >> + for (i = 0; i < vq_cnt; i++) { > >> + spin_lock_init(&vioch[i]->lock); > >> + vioch[i]->vqueue = vqs[i]; > >> + vioch[i]->vqueue->priv = vioch[i]; > >> + } > >> + > >> + vdev->priv = vioch; > >> + > > > > As mentioned before, in other Virtio drivers this is the place where we > > should register somehow this transports/channels with the core (before > > marking the device ready), and may be the place where we can just share > > the core device for this tranport with the SCMI core without then having > > to play the above link_supplier chan_available/setup dance. > > > > But again I still have to fully tries to change the SCMI transport > > interface at this point. > > > > I've not really gone into all the details across all series because I > > see the above issues/limitations in the driver/SCMI_transport as sort of > > blocking. > > I think some of the patches will no longer be needed in the future. I > would expect at least the following to go away: > > [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op > > > > > I'll let you now when I'll have something sensible on the SCMI core to > > share for better accomodate this transport. > > > > Thanks > > > > Cristian > > > > > >> + virtio_device_ready(vdev); > >> + > >> + return 0; > >> +} > >> + > >> +static unsigned int features[] = { > >> + VIRTIO_SCMI_F_P2A_CHANNELS, > >> +}; > >> + > >> +static const struct virtio_device_id id_table[] = { > >> + { VIRTIO_ID_SCMI, VIRTIO_DEV_ANY_ID }, > >> + { 0 } > >> +}; > >> + > >> +static struct virtio_driver virtio_scmi_driver = { > >> + .driver.name = "scmi-virtio", > >> + .driver.owner = THIS_MODULE, > >> + .feature_table = features, > >> + .feature_table_size = ARRAY_SIZE(features), > >> + .id_table = id_table, > >> + .probe = scmi_vio_probe, > >> +}; > >> + > >> +int __init virtio_scmi_init(void) > >> +{ > >> + return register_virtio_driver(&virtio_scmi_driver); > >> +} > >> + > >> +void __exit virtio_scmi_exit(void) > >> +{ > >> + unregister_virtio_driver(&virtio_scmi_driver); > >> +} > >> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h > >> index b052355ac7a3..57d233c02720 100644 > >> --- a/include/uapi/linux/virtio_ids.h > >> +++ b/include/uapi/linux/virtio_ids.h > >> @@ -48,5 +48,6 @@ > >> #define VIRTIO_ID_FS 26 /* virtio filesystem */ > >> #define VIRTIO_ID_PMEM 27 /* virtio pmem */ > >> #define VIRTIO_ID_MAC80211_HWSIM 29 /* virtio mac80211-hwsim */ > >> +#define VIRTIO_ID_SCMI 32 /* virtio SCMI */ > >> > >> #endif /* _LINUX_VIRTIO_IDS_H */ > >> diff --git a/include/uapi/linux/virtio_scmi.h b/include/uapi/linux/virtio_scmi.h > >> new file mode 100644 > >> index 000000000000..9f21b3dbbfe2 > >> --- /dev/null > >> +++ b/include/uapi/linux/virtio_scmi.h > >> @@ -0,0 +1,41 @@ > >> +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ > >> +/* > >> + * Copyright (C) 2020 OpenSynergy GmbH > >> + */ > >> + > >> +#ifndef _UAPI_LINUX_VIRTIO_SCMI_H > >> +#define _UAPI_LINUX_VIRTIO_SCMI_H > >> + > >> +#include <linux/virtio_types.h> > >> + > >> +/* Feature bits */ > >> + > >> +/* Device implements some SCMI notifications, or delayed responses. */ > >> +#define VIRTIO_SCMI_F_P2A_CHANNELS 0 > >> + > >> +/* Device implements any SCMI statistics shared memory region */ > >> +#define VIRTIO_SCMI_F_SHARED_MEMORY 1 > >> + > >> +/* Virtqueues */ > >> + > >> +#define VIRTIO_SCMI_VQ_TX 0 /* cmdq */ > >> +#define VIRTIO_SCMI_VQ_RX 1 /* eventq */ > >> +#define VIRTIO_SCMI_VQ_MAX_CNT 2 > >> + > >> +struct virtio_scmi_request { > >> + __virtio32 hdr; > >> + __u8 data[]; > >> +}; > >> + > >> +struct virtio_scmi_response { > >> + __virtio32 hdr; > >> + __virtio32 status; > >> + __u8 data[]; > >> +}; > >> + > >> +struct virtio_scmi_notification { > >> + __virtio32 hdr; > >> + __u8 data[]; > >> +}; > >> + > >> +#endif /* _UAPI_LINUX_VIRTIO_SCMI_H */ > >> -- > >> 2.25.1 > >> > >> > >> _______________________________________________ > >> linux-arm-kernel mailing list > >> linux-arm-kernel@lists.infradead.org > >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport 2020-11-05 21:21 ` Peter Hilber @ 2021-04-08 3:45 ` Viresh Kumar -1 siblings, 0 replies; 34+ messages in thread From: Viresh Kumar @ 2021-04-08 3:45 UTC (permalink / raw) To: Peter Hilber Cc: Linux ARM, DTML, virtualization, virtio-dev, Igor Skalkin, Rob Herring, Linux Kernel Mailing List, Sudeep Holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Michael S. Tsirkin, Jason Wang, Vincent Guittot On Fri, Nov 6, 2020 at 2:59 AM Peter Hilber <peter.hilber@opensynergy.com> wrote: > +static int scmi_vio_probe(struct virtio_device *vdev) > +{ > + struct device *dev = &vdev->dev; > + struct scmi_vio_channel **vioch; > + bool have_vq_rx; > + int vq_cnt; > + int i; > + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; > + > + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), > + GFP_KERNEL); > + if (!vioch) > + return -ENOMEM; > + > + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); > + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; > + > + for (i = 0; i < vq_cnt; i++) { > + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); > + if (!vioch[i]) > + return -ENOMEM; > + } > + > + if (have_vq_rx) > + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; > + > + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, > + scmi_vio_vqueue_names, NULL)) { > + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); > + return -1; > + } > + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); > + > + for (i = 0; i < vq_cnt; i++) { > + spin_lock_init(&vioch[i]->lock); > + vioch[i]->vqueue = vqs[i]; > + vioch[i]->vqueue->priv = vioch[i]; The vqueue->priv field is used by core, you can't update it else notifications won't work. > + } > + > + vdev->priv = vioch; > + > + virtio_device_ready(vdev); > + > + return 0; > +} diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c index f70aa72f34f1..b1af77341b30 100644 --- a/drivers/firmware/arm_scmi/virtio.c +++ b/drivers/firmware/arm_scmi/virtio.c @@ -80,7 +80,8 @@ static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, static void scmi_vio_complete_cb(struct virtqueue *vqueue) { - struct scmi_vio_channel *vioch = vqueue->priv; + struct scmi_vio_channel **_vioch = vqueue->vdev->priv; + struct scmi_vio_channel *vioch = _vioch[vqueue->index]; unsigned long iflags; unsigned int length; @@ -454,7 +455,6 @@ static int scmi_vio_probe(struct virtio_device *vdev) for (i = 0; i < vq_cnt; i++) { spin_lock_init(&vioch[i]->lock); vioch[i]->vqueue = vqs[i]; - vioch[i]->vqueue->priv = vioch[i]; } vdev->priv = vioch; ^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport @ 2021-04-08 3:45 ` Viresh Kumar 0 siblings, 0 replies; 34+ messages in thread From: Viresh Kumar @ 2021-04-08 3:45 UTC (permalink / raw) To: Peter Hilber Cc: Linux ARM, DTML, virtualization, virtio-dev, Igor Skalkin, Rob Herring, Linux Kernel Mailing List, Sudeep Holla, souvik.chakravarty, alex.bennee, jean-philippe, mikhail.golubev, anton.yakovlev, Michael S. Tsirkin, Jason Wang, Vincent Guittot On Fri, Nov 6, 2020 at 2:59 AM Peter Hilber <peter.hilber@opensynergy.com> wrote: > +static int scmi_vio_probe(struct virtio_device *vdev) > +{ > + struct device *dev = &vdev->dev; > + struct scmi_vio_channel **vioch; > + bool have_vq_rx; > + int vq_cnt; > + int i; > + struct virtqueue *vqs[VIRTIO_SCMI_VQ_MAX_CNT]; > + > + vioch = devm_kcalloc(dev, VIRTIO_SCMI_VQ_MAX_CNT, sizeof(*vioch), > + GFP_KERNEL); > + if (!vioch) > + return -ENOMEM; > + > + have_vq_rx = virtio_has_feature(vdev, VIRTIO_SCMI_F_P2A_CHANNELS); > + vq_cnt = have_vq_rx ? VIRTIO_SCMI_VQ_MAX_CNT : 1; > + > + for (i = 0; i < vq_cnt; i++) { > + vioch[i] = devm_kzalloc(dev, sizeof(**vioch), GFP_KERNEL); > + if (!vioch[i]) > + return -ENOMEM; > + } > + > + if (have_vq_rx) > + vioch[VIRTIO_SCMI_VQ_RX]->is_rx = true; > + > + if (virtio_find_vqs(vdev, vq_cnt, vqs, scmi_vio_complete_callbacks, > + scmi_vio_vqueue_names, NULL)) { > + dev_err(dev, "Failed to get %d virtqueue(s)\n", vq_cnt); > + return -1; > + } > + dev_info(dev, "Found %d virtqueue(s)\n", vq_cnt); > + > + for (i = 0; i < vq_cnt; i++) { > + spin_lock_init(&vioch[i]->lock); > + vioch[i]->vqueue = vqs[i]; > + vioch[i]->vqueue->priv = vioch[i]; The vqueue->priv field is used by core, you can't update it else notifications won't work. > + } > + > + vdev->priv = vioch; > + > + virtio_device_ready(vdev); > + > + return 0; > +} diff --git a/drivers/firmware/arm_scmi/virtio.c b/drivers/firmware/arm_scmi/virtio.c index f70aa72f34f1..b1af77341b30 100644 --- a/drivers/firmware/arm_scmi/virtio.c +++ b/drivers/firmware/arm_scmi/virtio.c @@ -80,7 +80,8 @@ static int scmi_vio_populate_vq_rx(struct scmi_vio_channel *vioch, static void scmi_vio_complete_cb(struct virtqueue *vqueue) { - struct scmi_vio_channel *vioch = vqueue->priv; + struct scmi_vio_channel **_vioch = vqueue->vdev->priv; + struct scmi_vio_channel *vioch = _vioch[vqueue->index]; unsigned long iflags; unsigned int length; @@ -454,7 +455,6 @@ static int scmi_vio_probe(struct virtio_device *vdev) for (i = 0; i < vq_cnt; i++) { spin_lock_init(&vioch[i]->lock); vioch[i]->vqueue = vqs[i]; - vioch[i]->vqueue->priv = vioch[i]; } vdev->priv = vioch; _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 34+ messages in thread
end of thread, other threads:[~2021-04-08 3:48 UTC | newest] Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-05 21:21 [RFC PATCH v2 00/10] firmware: arm_scmi: Add virtio transport Peter Hilber 2020-11-05 21:21 ` [virtio-dev] " Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 01/10] firmware: arm_scmi, smccc, mailbox: Make shmem based transports optional Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 02/10] firmware: arm_scmi: Document that max_msg is a per channel type limit Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 03/10] firmware: arm_scmi: Add op to override max message # Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 04/10] firmware: arm_scmi: Add per message transport data Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 05/10] firmware: arm_scmi: Add xfer_init_buffers transport op Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 06/10] firmware: arm_scmi: Add optional link_supplier() " Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 07/10] firmware: arm_scmi: Add per-device transport private info Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 08/10] firmware: arm_scmi: Add is_scmi_protocol_device() Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 09/10] dt-bindings: arm: Add virtio transport for SCMI Peter Hilber 2020-11-05 21:21 ` [virtio-dev] " Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-05 21:21 ` [RFC PATCH v2 10/10] firmware: arm_scmi: Add virtio transport Peter Hilber 2020-11-05 21:21 ` [virtio-dev] " Peter Hilber 2020-11-05 21:21 ` Peter Hilber 2020-11-10 21:32 ` Cristian Marussi 2020-11-10 21:32 ` Cristian Marussi 2020-11-12 10:57 ` Peter Hilber 2020-11-12 10:57 ` [virtio-dev] " Peter Hilber 2020-11-12 10:57 ` Peter Hilber 2020-11-17 21:20 ` Cristian Marussi 2020-11-17 21:20 ` Cristian Marussi 2021-04-08 3:45 ` Viresh Kumar 2021-04-08 3:45 ` Viresh Kumar
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.