[PATCH v8 0/9] SCMI Notifications Core Support

[PATCH v8 0/9] SCMI Notifications Core Support

[Cristian Marussi]

Hi all,

this series wants to introduce SCMI Notification Support, built on top of
the standard Kernel notification chain subsystem.

At initialization time each SCMI Protocol takes care to register with the
new SCMI notification core the set of its own events which it intends to
support.

Using the API exposed via scmi_handle.notify_ops a Kernel user can register
its own notifier_t callback (via a notifier_block as usual) against any
registered event as identified by the tuple:

		(proto_id, event_id, src_id)

where src_id represents a generic source identifier which is protocol
dependent like domain_id, performance_id, sensor_id and so forth.
(users can anyway do NOT provide any src_id, and subscribe instead to ALL
 the existing (if any) src_id sources for that proto_id/evt_id combination)

Each of the above tuple-specified eventis will be served on its own
dedicated blocking notification chain, dynamically allocated on-demand when
at least one user has shown interest on that event.

Upon a notification delivery all the users' registered notifier_t callbacks
will be in turn invoked and fed with the event_id as @action param and a
generated custom per-event struct _report as @data param.
(as in include/linux/scmi_protocol.h)

The final step of notification delivery via users' callback invocation is
instead delegated to a pool of deferred workers (Kernel cmwq): each
SCMI protocol has its own dedicated worker and dedicated queue to push
events from the rx ISR to the worker.

Based on scmi-next/for-next/scmi 5.7 [1], on top of:

commit f7199cf48902 ("firmware: arm_scmi: Fix return error code in
		     smc_send_message")

This series has been tested on JUNO with an experimental firmware only
supporting Perf Notifications.


Thanks

Cristian

----
v7 --> v8:
- removed unneeded initialized/enabled atomics, added proper barriers
- added a few comments about queueing work item and kfifos
v6 --> v7:
- rebased on top of scmi-next 5.7, dropped the initial 4 patches
  since now already queued on base scmi-next [1]
- fixed some events' proto initialization
- removed some notify_enabled explicit methods exposed in some protocol_ops
  since not supposed to be used directly when using this notification
  framework (and of no other known use)
- exposing SCMI_EVENT_ enums in scmi_protocol.h
- added agent_id field in RESET_ISSUED payload as per reviewed SCMI spec
- removed POWER_STATE_CHANGE_REQUESTED pre-notification definition and
  handling as per reviewedSCMI spec
- fixed report.timestamp field type

v5 --> v6:
- added handle argument to fill_custom_report() helper

v4 --> v5:
- fixed kernel-doc
- added proper barriers around registered protocols and events
  initialization
- reviewed queues allocation using devm_add_action_or_reset
- reviewed REVT_NOTIFY_ENABLE macro

v3 --> v4:
- dropped RFC tag
- avoid one unneeded evt payload memcpy on the ISR RC code path by
  redesigning dispatcher to handle partial queue-reads (in_flight events,
  only header)
- fixed the initialization issue exposed by late SCMI modules loading by
  reviewing the init process to support possible late events registrations
  by protocols and early callbacks registrations by users (pending)
- cleanup/simplification of exit path: SCMI protocols are generally never
  de-initialized after the initial device creation, so do not deinit
  notification core either (we do halt the delivery, stop the wq and empty
  the queues though)
- reduced contention on regustered_events_handler to the minimum during
  delivery by splitting the common registered_events_handlers hashtable
  into a number of per-protocol tables
- converted registered_protocols and registered_events hastable to
  fixed size arrays: simpler and lockless in our usage scenario

v2 --> v3:
- added platform instance awareness to the notification core: a
  notification instance is created for each known handle
- reviewed notification core initialization and shutdown process
- removed generic non-handle-rooted registration API
- added WQ_SYSFS flag to workqueue instance

v1 --> v2:
- dropped anti-tampering patch
- rebased on top of scmi-for-next-5.6, which includes Viresh series that
  make SCMI core independent of transport (5c8a47a5a91d)
- add a few new SCMI transport methods on top of Viresh patch to address
  needs of SCMI Notifications
- reviewed/renamed scmi_handle_xfer_delayed_resp()
- split main SCMI Notification core patch (~1k lines) into three chunks:
  protocol-registration / callbacks-registration / dispatch-and-delivery
- removed awkward usage of IDR maps in favour of pure hashtables
- added enable/disable refcounting in notification core (was broken in v1)
- removed per-protocol candidate API: a single generic API is now proposed
  instead of scmi_register_<proto>_event_notifier(evt_id, *src_id, *nb)
- added handle->notify_ops as an alternative notification API
  for scmi_driver
- moved ALL_SRCIDs enabled handling from protocol code to core code
- reviewed protocol registration/unregistration logic to use devres
- reviewed cleanup phase on shutdown
- fixed  ERROR: reference preceded by free as reported by kbuild test robot

[1] git://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git


Cristian Marussi (9):
  firmware: arm_scmi: Add notification protocol-registration
  firmware: arm_scmi: Add notification callbacks-registration
  firmware: arm_scmi: Add notification dispatch and delivery
  firmware: arm_scmi: Enable notification core
  firmware: arm_scmi: Add Power notifications support
  firmware: arm_scmi: Add Perf notifications support
  firmware: arm_scmi: Add Sensor notifications support
  firmware: arm_scmi: Add Reset notifications support
  firmware: arm_scmi: Add Base notifications support

 drivers/firmware/arm_scmi/Makefile  |    2 +-
 drivers/firmware/arm_scmi/base.c    |  118 ++-
 drivers/firmware/arm_scmi/common.h  |    4 +
 drivers/firmware/arm_scmi/driver.c  |   10 +
 drivers/firmware/arm_scmi/notify.c  | 1480 +++++++++++++++++++++++++++
 drivers/firmware/arm_scmi/notify.h  |   78 ++
 drivers/firmware/arm_scmi/perf.c    |  137 ++-
 drivers/firmware/arm_scmi/power.c   |  101 +-
 drivers/firmware/arm_scmi/reset.c   |  105 +-
 drivers/firmware/arm_scmi/sensors.c |   77 +-
 include/linux/scmi_protocol.h       |  108 +-
 11 files changed, 2190 insertions(+), 30 deletions(-)
 create mode 100644 drivers/firmware/arm_scmi/notify.c
 create mode 100644 drivers/firmware/arm_scmi/notify.h

-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 1/9] firmware: arm_scmi: Add notification protocol-registration

[Cristian Marussi]

Add core SCMI Notifications protocol-registration support: allow protocols
to register their own set of supported events, during their initialization
phase. Notification core can track multiple platform instances by their
handles.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V7 --> V8
- Fixed init/enable procedure, un-needed atomics removed
V4 --> V5
- fixed kernel-doc
- added barriers for registered protocols and events
- using kfifo_alloc and devm_add_action_or_reset
V3 --> V4
- removed scratch ISR buffer, move scratch BH buffer into protocol
  descriptor
- converted registered_protocols and registered_events from hashtables
  into bare fixed-sized arrays
- removed unregister protocols' routines (never called really)
V2 --> V3
- added scmi_notify_instance to track target platform instance
V1 --> V2
- splitted out of V1 patch 04
- moved from IDR maps to real HashTables to store events
- scmi_notifications_initialized is now an atomic_t
- reviewed protocol registration/unregistration to use devres
- fixed:
  drivers/firmware/arm_scmi/notify.c:483:18-23: ERROR:
  	reference preceded by free on line 482

Reported-by: kbuild test robot <lkp@intel.com>
Reported-by: Julia Lawall <julia.lawall@lip6.fr>
---
 drivers/firmware/arm_scmi/Makefile |   2 +-
 drivers/firmware/arm_scmi/common.h |   4 +
 drivers/firmware/arm_scmi/notify.c | 435 +++++++++++++++++++++++++++++
 drivers/firmware/arm_scmi/notify.h |  56 ++++
 include/linux/scmi_protocol.h      |   3 +
 5 files changed, 499 insertions(+), 1 deletion(-)
 create mode 100644 drivers/firmware/arm_scmi/notify.c
 create mode 100644 drivers/firmware/arm_scmi/notify.h

diff --git a/drivers/firmware/arm_scmi/Makefile b/drivers/firmware/arm_scmi/Makefile
index 11b238f81923..d55612362d65 100644
--- a/drivers/firmware/arm_scmi/Makefile
+++ b/drivers/firmware/arm_scmi/Makefile
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-y	= scmi-bus.o scmi-driver.o scmi-protocols.o scmi-transport.o
 scmi-bus-y = bus.o
-scmi-driver-y = driver.o
+scmi-driver-y = driver.o notify.o
 scmi-transport-y = shmem.o
 scmi-transport-$(CONFIG_MAILBOX) += mailbox.o
 scmi-transport-$(CONFIG_HAVE_ARM_SMCCC) += smc.o
diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h
index 31fe5a22a011..c113e578cc6c 100644
--- a/drivers/firmware/arm_scmi/common.h
+++ b/drivers/firmware/arm_scmi/common.h
@@ -6,6 +6,8 @@
  *
  * Copyright (C) 2018 ARM Ltd.
  */
+#ifndef _SCMI_COMMON_H
+#define _SCMI_COMMON_H
 
 #include <linux/bitfield.h>
 #include <linux/completion.h>
@@ -235,3 +237,5 @@ void shmem_fetch_notification(struct scmi_shared_mem __iomem *shmem,
 void shmem_clear_channel(struct scmi_shared_mem __iomem *shmem);
 bool shmem_poll_done(struct scmi_shared_mem __iomem *shmem,
 		     struct scmi_xfer *xfer);
+
+#endif /* _SCMI_COMMON_H */
diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
new file mode 100644
index 000000000000..8b620fc7df50
--- /dev/null
+++ b/drivers/firmware/arm_scmi/notify.c
@@ -0,0 +1,435 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * System Control and Management Interface (SCMI) Notification support
+ *
+ * Copyright (C) 2020 ARM Ltd.
+ */
+/**
+ * DOC: Theory of operation
+ *
+ * SCMI Protocol specification allows the platform to signal events to
+ * interested agents via notification messages: this is an implementation
+ * of the dispatch and delivery of such notifications to the interested users
+ * inside the Linux kernel.
+ *
+ * An SCMI Notification core instance is initialized for each active platform
+ * instance identified by the means of the usual &struct scmi_handle.
+ *
+ * Each SCMI Protocol implementation, during its initialization, registers with
+ * this core its set of supported events using scmi_register_protocol_events():
+ * all the needed descriptors are stored in the &struct registered_protocols and
+ * &struct registered_events arrays.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bug.h>
+#include <linux/compiler.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/kernel.h>
+#include <linux/kfifo.h>
+#include <linux/mutex.h>
+#include <linux/refcount.h>
+#include <linux/scmi_protocol.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "notify.h"
+
+#define	SCMI_MAX_PROTO			256
+#define	SCMI_ALL_SRC_IDS		0xffffUL
+/*
+ * Builds an unsigned 32bit key from the given input tuple to be used
+ * as a key in hashtables.
+ */
+#define MAKE_HASH_KEY(p, e, s)			\
+	((u32)(((p) << 24) | ((e) << 16) | ((s) & SCMI_ALL_SRC_IDS)))
+
+#define MAKE_ALL_SRCS_KEY(p, e)			\
+	MAKE_HASH_KEY((p), (e), SCMI_ALL_SRC_IDS)
+
+struct scmi_registered_protocol_events_desc;
+
+/**
+ * struct scmi_notify_instance  - Represents an instance of the notification
+ * core
+ * @gid: GroupID used for devres
+ * @handle: A reference to the platform instance
+ * @registered_protocols: A statically allocated array containing pointers to
+ *			  all the registered protocol-level specific information
+ *			  related to events' handling
+ *
+ * Each platform instance, represented by a handle, has its own instance of
+ * the notification subsystem represented by this structure.
+ */
+struct scmi_notify_instance {
+	void						*gid;
+	struct scmi_handle				*handle;
+	struct scmi_registered_protocol_events_desc	**registered_protocols;
+};
+
+/**
+ * struct events_queue  - Describes a queue and its associated worker
+ * @sz: Size in bytes of the related kfifo
+ * @kfifo: A dedicated Kernel kfifo descriptor
+ *
+ * Each protocol has its own dedicated events_queue descriptor.
+ */
+struct events_queue {
+	size_t				sz;
+	struct kfifo			kfifo;
+};
+
+/**
+ * struct scmi_event_header  - A utility header
+ * @timestamp: The timestamp, in nanoseconds (boottime), which was associated
+ *	       to this event as soon as it entered the SCMI RX ISR
+ * @evt_id: Event ID (corresponds to the Event MsgID for this Protocol)
+ * @payld_sz: Effective size of the embedded message payload which follows
+ * @payld: A reference to the embedded event payload
+ *
+ * This header is prepended to each received event message payload before
+ * queueing it on the related &struct events_queue.
+ */
+struct scmi_event_header {
+	u64	timestamp;
+	u8	evt_id;
+	size_t	payld_sz;
+	u8	payld[];
+} __packed;
+
+struct scmi_registered_event;
+
+/**
+ * struct scmi_registered_protocol_events_desc  - Protocol Specific information
+ * @id: Protocol ID
+ * @ops: Protocol specific and event-related operations
+ * @equeue: The embedded per-protocol events_queue
+ * @ni: A reference to the initialized instance descriptor
+ * @eh: A reference to pre-allocated buffer to be used as a scratch area by the
+ *	deferred worker when fetching data from the kfifo
+ * @eh_sz: Size of the pre-allocated buffer @eh
+ * @in_flight: A reference to an in flight &struct scmi_registered_event
+ * @num_events: Number of events in @registered_events
+ * @registered_events: A dynamically allocated array holding all the registered
+ *		       events' descriptors, whose fixed-size is determined at
+ *		       compile time.
+ *
+ * All protocols that register at least one event have their protocol-specific
+ * information stored here, together with the embedded allocated events_queue.
+ * These descriptors are stored in the @registered_protocols array at protocol
+ * registration time.
+ *
+ * Once these descriptors are successfully registered, they are NEVER again
+ * removed or modified since protocols do not unregister ever, so that, once
+ * we safely grab a NON-NULL reference from the array we can keep it and use it.
+ */
+struct scmi_registered_protocol_events_desc {
+	u8					id;
+	const struct scmi_protocol_event_ops	*ops;
+	struct events_queue			equeue;
+	struct scmi_notify_instance		*ni;
+	struct scmi_event_header		*eh;
+	size_t					eh_sz;
+	void					*in_flight;
+	int					num_events;
+	struct scmi_registered_event		**registered_events;
+};
+
+/**
+ * struct scmi_registered_event  - Event Specific Information
+ * @proto: A reference to the associated protocol descriptor
+ * @evt: A reference to the associated event descriptor (as provided at
+ *       registration time)
+ * @report: A pre-allocated buffer used by the deferred worker to fill a
+ *	    customized event report
+ * @num_sources: The number of possible sources for this event as stated at
+ *		 events' registration time
+ * @sources: A reference to a dynamically allocated array used to refcount the
+ *	     events' enable requests for all the existing sources
+ * @sources_mtx: A mutex to serialize the access to @sources
+ *
+ * All registered events are represented by one of these structures that are
+ * stored in the @registered_events array at protocol registration time.
+ *
+ * Once these descriptors are successfully registered, they are NEVER again
+ * removed or modified since protocols do not unregister ever, so that once we
+ * safely grab a NON-NULL reference from the table we can keep it and use it.
+ */
+struct scmi_registered_event {
+	struct scmi_registered_protocol_events_desc	*proto;
+	const struct scmi_event				*evt;
+	void						*report;
+	u32						num_sources;
+	refcount_t					*sources;
+	struct mutex					sources_mtx;
+};
+
+/**
+ * scmi_kfifo_free()  - Devres action helper to free the kfifo
+ * @kfifo: The kfifo to free
+ */
+static void scmi_kfifo_free(void *kfifo)
+{
+	kfifo_free((struct kfifo *)kfifo);
+}
+
+/**
+ * scmi_initialize_events_queue()  - Allocate/Initialize a kfifo buffer
+ * @ni: A reference to the notification instance to use
+ * @equeue: The events_queue to initialize
+ * @sz: Size of the kfifo buffer to allocate
+ *
+ * Allocate a buffer for the kfifo and initialize it.
+ *
+ * Return: 0 on Success
+ */
+static int scmi_initialize_events_queue(struct scmi_notify_instance *ni,
+					struct events_queue *equeue, size_t sz)
+{
+	if (kfifo_alloc(&equeue->kfifo, sz, GFP_KERNEL))
+		return -ENOMEM;
+	/* Size could have been roundup to power-of-two */
+	equeue->sz = kfifo_size(&equeue->kfifo);
+
+	return devm_add_action_or_reset(ni->handle->dev, scmi_kfifo_free,
+					&equeue->kfifo);
+}
+
+/**
+ * scmi_allocate_registered_protocol_desc()  - Allocate a registered protocol
+ * events' descriptor
+ * @ni: A reference to the &struct scmi_notify_instance notification instance
+ *	to use
+ * @proto_id: Protocol ID
+ * @queue_sz: Size of the associated queue to allocate
+ * @eh_sz: Size of the event header scratch area to pre-allocate
+ * @num_events: Number of events to support (size of @registered_events)
+ * @ops: Pointer to a struct holding references to protocol specific helpers
+ *	 needed during events handling
+ *
+ * It is supposed to be called only once for each protocol at protocol
+ * initialization time, so it warns if the requested protocol is found already
+ * registered.
+ *
+ * Return: The allocated and registered descriptor on Success
+ */
+static struct scmi_registered_protocol_events_desc *
+scmi_allocate_registered_protocol_desc(struct scmi_notify_instance *ni,
+				       u8 proto_id, size_t queue_sz,
+				       size_t eh_sz, int num_events,
+				const struct scmi_protocol_event_ops *ops)
+{
+	int ret;
+	struct scmi_registered_protocol_events_desc *pd;
+
+	/* Ensure protocols are up to date */
+	smp_rmb();
+	if (ni->registered_protocols[proto_id]) {
+		WARN_ON(1);
+		return ERR_PTR(-EINVAL);
+	}
+
+	pd = devm_kzalloc(ni->handle->dev, sizeof(*pd), GFP_KERNEL);
+	if (!pd)
+		return ERR_PTR(-ENOMEM);
+	pd->id = proto_id;
+	pd->ops = ops;
+	pd->ni = ni;
+
+	ret = scmi_initialize_events_queue(ni, &pd->equeue, queue_sz);
+	if (ret)
+		return ERR_PTR(ret);
+
+	pd->eh = devm_kzalloc(ni->handle->dev, eh_sz, GFP_KERNEL);
+	if (!pd->eh)
+		return ERR_PTR(-ENOMEM);
+	pd->eh_sz = eh_sz;
+
+	pd->registered_events = devm_kcalloc(ni->handle->dev, num_events,
+					     sizeof(char *), GFP_KERNEL);
+	if (!pd->registered_events)
+		return ERR_PTR(-ENOMEM);
+	pd->num_events = num_events;
+
+	return pd;
+}
+
+/**
+ * scmi_register_protocol_events()  - Register Protocol Events with the core
+ * @handle: The handle identifying the platform instance against which the
+ *	    the protocol's events are registered
+ * @proto_id: Protocol ID
+ * @queue_sz: Size in bytes of the associated queue to be allocated
+ * @ops: Protocol specific event-related operations
+ * @evt: Event descriptor array
+ * @num_events: Number of events in @evt array
+ * @num_sources: Number of possible sources for this protocol on this
+ *		 platform.
+ *
+ * Used by SCMI Protocols initialization code to register with the notification
+ * core the list of supported events and their descriptors: takes care to
+ * pre-allocate and store all needed descriptors, scratch buffers and event
+ * queues.
+ *
+ * Return: 0 on Success
+ */
+int scmi_register_protocol_events(const struct scmi_handle *handle,
+				  u8 proto_id, size_t queue_sz,
+				  const struct scmi_protocol_event_ops *ops,
+				  const struct scmi_event *evt, int num_events,
+				  int num_sources)
+{
+	int i;
+	size_t payld_sz = 0;
+	struct scmi_registered_protocol_events_desc *pd;
+	struct scmi_notify_instance *ni;
+
+	if (!ops || !evt || proto_id >= SCMI_MAX_PROTO)
+		return -EINVAL;
+
+	/* Ensure notify_priv is updated */
+	smp_rmb();
+	if (unlikely(!handle->notify_priv))
+		return -ENOMEM;
+	ni = handle->notify_priv;
+
+	/* Attach to the notification main devres group */
+	if (!devres_open_group(ni->handle->dev, ni->gid, GFP_KERNEL))
+		return -ENOMEM;
+
+	for (i = 0; i < num_events; i++)
+		payld_sz = max_t(size_t, payld_sz, evt[i].max_payld_sz);
+	pd = scmi_allocate_registered_protocol_desc(ni, proto_id, queue_sz,
+				    sizeof(struct scmi_event_header) + payld_sz,
+						    num_events, ops);
+	if (IS_ERR(pd))
+		goto err;
+
+	for (i = 0; i < num_events; i++, evt++) {
+		struct scmi_registered_event *r_evt;
+
+		r_evt = devm_kzalloc(ni->handle->dev, sizeof(*r_evt),
+				     GFP_KERNEL);
+		if (!r_evt)
+			goto err;
+		r_evt->proto = pd;
+		r_evt->evt = evt;
+
+		r_evt->sources = devm_kcalloc(ni->handle->dev, num_sources,
+					      sizeof(refcount_t), GFP_KERNEL);
+		if (!r_evt->sources)
+			goto err;
+		r_evt->num_sources = num_sources;
+		mutex_init(&r_evt->sources_mtx);
+
+		r_evt->report = devm_kzalloc(ni->handle->dev,
+					     evt->max_report_sz, GFP_KERNEL);
+		if (!r_evt->report)
+			goto err;
+
+		pd->registered_events[i] = r_evt;
+		/* Ensure events are updated */
+		smp_wmb();
+		pr_info("SCMI Notifications: registered event - %X\n",
+			MAKE_ALL_SRCS_KEY(r_evt->proto->id, r_evt->evt->id));
+	}
+
+	/* Register protocol and events...it will never be removed */
+	ni->registered_protocols[proto_id] = pd;
+	/* Ensure protocols are updated */
+	smp_wmb();
+
+	devres_close_group(ni->handle->dev, ni->gid);
+
+	return 0;
+
+err:
+	pr_warn("SCMI Notifications - Proto:%X - Registration Failed !\n",
+		proto_id);
+	/* A failing protocol registration does not trigger full failure */
+	devres_close_group(ni->handle->dev, ni->gid);
+
+	return -ENOMEM;
+}
+
+/**
+ * scmi_notification_init()  - Initializes Notification Core Support
+ * @handle: The handle identifying the platform instance to initialize
+ *
+ * This function lays out all the basic resources needed by the notification
+ * core instance identified by the provided handle: once done, all of the
+ * SCMI Protocols can register their events with the core during their own
+ * initializations.
+ *
+ * Note that failing to initialize the core notifications support does not
+ * cause the whole SCMI Protocols stack to fail its initialization.
+ *
+ * SCMI Notification Initialization happens in 2 steps:
+ * * initialization: basic common allocations (this function)
+ * * registration: protocols asynchronously come into life and registers their
+ *		   own supported list of events with the core; this causes
+ *		   further per-protocol allocations
+ *
+ * Any user's callback registration attempt, referring a still not registered
+ * event, will be registered as pending and finalized later (if possible)
+ * by scmi_protocols_late_init() work.
+ * This allows for lazy initialization of SCMI Protocols due to late (or
+ * missing) SCMI drivers' modules loading.
+ *
+ * Return: 0 on Success
+ */
+int scmi_notification_init(struct scmi_handle *handle)
+{
+	void *gid;
+	struct scmi_notify_instance *ni;
+
+	gid = devres_open_group(handle->dev, NULL, GFP_KERNEL);
+	if (!gid)
+		return -ENOMEM;
+
+	ni = devm_kzalloc(handle->dev, sizeof(*ni), GFP_KERNEL);
+	if (!ni)
+		goto err;
+
+	ni->gid = gid;
+	ni->handle = handle;
+
+	ni->registered_protocols = devm_kcalloc(handle->dev, SCMI_MAX_PROTO,
+						sizeof(char *), GFP_KERNEL);
+	if (!ni->registered_protocols)
+		goto err;
+
+	handle->notify_priv = ni;
+	/* Ensure handle is up to date */
+	smp_wmb();
+
+	pr_info("SCMI Notifications Core Enabled.\n");
+
+	devres_close_group(handle->dev, ni->gid);
+
+	return 0;
+
+err:
+	pr_warn("SCMI Notifications - Initialization Failed.\n");
+	devres_release_group(handle->dev, NULL);
+	return -ENOMEM;
+}
+
+/**
+ * scmi_notification_exit()  - Shutdown and clean Notification core
+ * @handle: The handle identifying the platform instance to shutdown
+ */
+void scmi_notification_exit(struct scmi_handle *handle)
+{
+	struct scmi_notify_instance *ni;
+
+	/* Ensure notify_priv is updated */
+	smp_rmb();
+	if (unlikely(!handle->notify_priv))
+		return;
+	ni = handle->notify_priv;
+
+	devres_release_group(ni->handle->dev, ni->gid);
+}
diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
new file mode 100644
index 000000000000..54094aaf812a
--- /dev/null
+++ b/drivers/firmware/arm_scmi/notify.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * System Control and Management Interface (SCMI) Message Protocol
+ * notification header file containing some definitions, structures
+ * and function prototypes related to SCMI Notification handling.
+ *
+ * Copyright (C) 2020 ARM Ltd.
+ */
+#ifndef _SCMI_NOTIFY_H
+#define _SCMI_NOTIFY_H
+
+#include <linux/device.h>
+#include <linux/types.h>
+
+/**
+ * struct scmi_event  - Describes an event to be supported
+ * @id: Event ID
+ * @max_payld_sz: Max possible size for the payload of a notif msg of this kind
+ * @max_report_sz: Max possible size for the report of a notif msg of this kind
+ *
+ * Each SCMI protocol, during its initialization phase, can describe the events
+ * it wishes to support in a few struct scmi_event and pass them to the core
+ * using scmi_register_protocol_events().
+ */
+struct scmi_event {
+	u8	id;
+	size_t	max_payld_sz;
+	size_t	max_report_sz;
+};
+
+/**
+ * struct scmi_protocol_event_ops  - Protocol helpers called by the notification
+ *				     core.
+ * @set_notify_enabled: Enable/disable the required evt_id/src_id notifications
+ *			using the proper custom protocol commands.
+ *			Return true if at least one the required src_id
+ *			has been successfully enabled/disabled
+ *
+ * Context: Helpers described in &struct scmi_protocol_event_ops are called
+ *	    only in process context.
+ */
+struct scmi_protocol_event_ops {
+	bool (*set_notify_enabled)(const struct scmi_handle *handle,
+				   u8 evt_id, u32 src_id, bool enabled);
+};
+
+int scmi_notification_init(struct scmi_handle *handle);
+void scmi_notification_exit(struct scmi_handle *handle);
+
+int scmi_register_protocol_events(const struct scmi_handle *handle,
+				  u8 proto_id, size_t queue_sz,
+				  const struct scmi_protocol_event_ops *ops,
+				  const struct scmi_event *evt, int num_events,
+				  int num_sources);
+
+#endif /* _SCMI_NOTIFY_H */
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index ce2f5c28b2df..0679f10ab05e 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -231,6 +231,8 @@ struct scmi_reset_ops {
  *	protocol(for internal use only)
  * @reset_priv: pointer to private data structure specific to reset
  *	protocol(for internal use only)
+ * @notify_priv: pointer to private data structure specific to notifications
+ *	(for internal use only)
  */
 struct scmi_handle {
 	struct device *dev;
@@ -246,6 +248,7 @@ struct scmi_handle {
 	void *power_priv;
 	void *sensor_priv;
 	void *reset_priv;
+	void *notify_priv;
 };
 
 enum scmi_std_protocol {
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 2/9] firmware: arm_scmi: Add notification callbacks-registration

[Cristian Marussi]

Add core SCMI Notifications callbacks-registration support: allow users
to register their own callbacks against the desired events.
Whenever a registration request is issued against a still non existent
event, mark such request as pending for later processing, in order to
account for possible late initializations of SCMI Protocols associated
to loadable drivers.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V7 --> V8
- Fixed init check, un-needed atomics removed
V6 --> V7
- removed un-needed ktime.h include
V4 --> V5
- fix kernel-doc
- reviewed REVT_NOTIFY_ENABLE macro
- added matching barrier for late_init
V3 --> V4
- split registered_handlers hashtable on a per-protocol basis to reduce
  unneeded contention
- introduced pending_handlers table and related late_init worker to finalize
  handlers registration upon effective protocols' registrations
- introduced further safe accessors macros for registered_protocols
  and registered_events arrays
V2 --> V3
- refactored get/put event_handler
- removed generic non-handle-based API
V1 --> V2
- splitted out of V1 patch 04
- moved from IDR maps to real HashTables to store event_handlers
- added proper enable_events refcounting via __scmi_enable_evt()
  [was broken in V1 when using ALL_SRCIDs notification chains]
- reviewed hashtable cleanup strategy in scmi_notifications_exit()
- added scmi_register_event_notifier()/scmi_unregister_event_notifier()
  to include/linux/scmi_protocol.h as a candidate user API
  [no EXPORTs still]
- added notify_ops to handle during initialization as an additional
  internal API for scmi_drivers
---
 drivers/firmware/arm_scmi/notify.c | 695 +++++++++++++++++++++++++++++
 drivers/firmware/arm_scmi/notify.h |  12 +
 include/linux/scmi_protocol.h      |  46 ++
 3 files changed, 753 insertions(+)

diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
index 8b620fc7df50..7cf61dbe2a8e 100644
--- a/drivers/firmware/arm_scmi/notify.c
+++ b/drivers/firmware/arm_scmi/notify.c
@@ -19,17 +19,49 @@
  * this core its set of supported events using scmi_register_protocol_events():
  * all the needed descriptors are stored in the &struct registered_protocols and
  * &struct registered_events arrays.
+ *
+ * Kernel users interested in some specific event can register their callbacks
+ * providing the usual notifier_block descriptor, since this core implements
+ * events' delivery using the standard Kernel notification chains machinery.
+ *
+ * Given the number of possible events defined by SCMI and the extensibility
+ * of the SCMI Protocol itself, the underlying notification chains are created
+ * and destroyed dynamically on demand depending on the number of users
+ * effectively registered for an event, so that no support structures or chains
+ * are allocated until at least one user has registered a notifier_block for
+ * such event. Similarly, events' generation itself is enabled at the platform
+ * level only after at least one user has registered, and it is shutdown after
+ * the last user for that event has gone.
+ *
+ * All users provided callbacks and allocated notification-chains are stored in
+ * the @registered_events_handlers hashtable. Callbacks' registration requests
+ * for still to be registered events are instead kept in the dedicated common
+ * hashtable @pending_events_handlers.
+ *
+ * An event is identified univocally by the tuple (proto_id, evt_id, src_id)
+ * and is served by its own dedicated notification chain; information contained
+ * in such tuples is used, in a few different ways, to generate the needed
+ * hash-keys.
+ *
+ * Here proto_id and evt_id are simply the protocol_id and message_id numbers
+ * as described in the SCMI Protocol specification, while src_id represents an
+ * optional, protocol dependent, source identifier (like domain_id, perf_id
+ * or sensor_id and so forth).
  */
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#include <linux/bitfield.h>
 #include <linux/bug.h>
 #include <linux/compiler.h>
 #include <linux/device.h>
 #include <linux/err.h>
+#include <linux/hashtable.h>
 #include <linux/kernel.h>
 #include <linux/kfifo.h>
+#include <linux/list.h>
 #include <linux/mutex.h>
+#include <linux/notifier.h>
 #include <linux/refcount.h>
 #include <linux/scmi_protocol.h>
 #include <linux/slab.h>
@@ -49,6 +81,74 @@
 #define MAKE_ALL_SRCS_KEY(p, e)			\
 	MAKE_HASH_KEY((p), (e), SCMI_ALL_SRC_IDS)
 
+/*
+ * Assumes that the stored obj includes its own hash-key in a field named 'key':
+ * with this simplification this macro can be equally used for all the objects'
+ * types hashed by this implementation.
+ *
+ * @__ht: The hashtable name
+ * @__obj: A pointer to the object type to be retrieved from the hashtable;
+ *	   it will be used as a cursor while scanning the hastable and it will
+ *	   be possibly left as NULL when @__k is not found
+ * @__k: The key to search for
+ */
+#define KEY_FIND(__ht, __obj, __k)				\
+({								\
+	hash_for_each_possible((__ht), (__obj), hash, (__k))	\
+		if (likely((__obj)->key == (__k)))		\
+			break;					\
+	__obj;							\
+})
+
+#define PROTO_ID_MASK			GENMASK(31, 24)
+#define EVT_ID_MASK			GENMASK(23, 16)
+#define SRC_ID_MASK			GENMASK(15, 0)
+#define KEY_XTRACT_PROTO_ID(key)	FIELD_GET(PROTO_ID_MASK, (key))
+#define KEY_XTRACT_EVT_ID(key)		FIELD_GET(EVT_ID_MASK, (key))
+#define KEY_XTRACT_SRC_ID(key)		FIELD_GET(SRC_ID_MASK, (key))
+
+/*
+ * A set of macros used to access safely @registered_protocols and
+ * @registered_events arrays; these are fixed in size and each entry is possibly
+ * populated at protocols' registration time and then only read but NEVER
+ * modified or removed.
+ */
+#define SCMI_GET_PROTO(__ni, __pid)					\
+({									\
+	struct scmi_registered_protocol_events_desc *__pd = NULL;	\
+									\
+	if ((__ni) && (__pid) < SCMI_MAX_PROTO)				\
+		__pd = READ_ONCE((__ni)->registered_protocols[(__pid)]);\
+	__pd;								\
+})
+
+#define SCMI_GET_REVT_FROM_PD(__pd, __eid)				\
+({									\
+	struct scmi_registered_event *__revt = NULL;			\
+									\
+	if ((__pd) && (__eid) < (__pd)->num_events)			\
+		__revt = READ_ONCE((__pd)->registered_events[(__eid)]);	\
+	__revt;								\
+})
+
+#define SCMI_GET_REVT(__ni, __pid, __eid)				\
+({									\
+	struct scmi_registered_event *__revt = NULL;			\
+	struct scmi_registered_protocol_events_desc *__pd = NULL;	\
+									\
+	__pd = SCMI_GET_PROTO((__ni), (__pid));				\
+	__revt = SCMI_GET_REVT_FROM_PD(__pd, (__eid));			\
+	__revt;								\
+})
+
+/* A couple of utility macros to limit cruft when calling protocols' helpers */
+#define REVT_NOTIFY_ENABLE(revt, eid, sid)				       \
+	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
+						(eid), (sid), true))
+#define REVT_NOTIFY_DISABLE(revt, eid, sid)				       \
+	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
+						(eid), (sid), false))
+
 struct scmi_registered_protocol_events_desc;
 
 /**
@@ -56,9 +156,13 @@ struct scmi_registered_protocol_events_desc;
  * core
  * @gid: GroupID used for devres
  * @handle: A reference to the platform instance
+ * @init_work: A work item to perform final initializations of pending handlers
+ * @pending_mtx: A mutex to protect @pending_events_handlers
  * @registered_protocols: A statically allocated array containing pointers to
  *			  all the registered protocol-level specific information
  *			  related to events' handling
+ * @pending_events_handlers: An hashtable containing all pending events'
+ *			     handlers descriptors
  *
  * Each platform instance, represented by a handle, has its own instance of
  * the notification subsystem represented by this structure.
@@ -66,7 +170,12 @@ struct scmi_registered_protocol_events_desc;
 struct scmi_notify_instance {
 	void						*gid;
 	struct scmi_handle				*handle;
+
+	struct work_struct				init_work;
+
+	struct mutex					pending_mtx;
 	struct scmi_registered_protocol_events_desc	**registered_protocols;
+	DECLARE_HASHTABLE(pending_events_handlers, 8);
 };
 
 /**
@@ -115,6 +224,9 @@ struct scmi_registered_event;
  * @registered_events: A dynamically allocated array holding all the registered
  *		       events' descriptors, whose fixed-size is determined at
  *		       compile time.
+ * @registered_mtx: A mutex to protect @registered_events_handlers
+ * @registered_events_handlers: An hashtable containing all events' handlers
+ *				descriptors registered for this protocol
  *
  * All protocols that register at least one event have their protocol-specific
  * information stored here, together with the embedded allocated events_queue.
@@ -135,6 +247,8 @@ struct scmi_registered_protocol_events_desc {
 	void					*in_flight;
 	int					num_events;
 	struct scmi_registered_event		**registered_events;
+	struct mutex				registered_mtx;
+	DECLARE_HASHTABLE(registered_events_handlers, 8);
 };
 
 /**
@@ -166,6 +280,38 @@ struct scmi_registered_event {
 	struct mutex					sources_mtx;
 };
 
+/**
+ * struct scmi_event_handler  - Event handler information
+ * @key: The used hashkey
+ * @users: A reference count for number of active users for this handler
+ * @r_evt: A reference to the associated registered event; when this is NULL
+ *	   this handler is pending, which means that identifies a set of
+ *	   callbacks intended to be attached to an event which is still not
+ *	   known nor registered by any protocol at that point in time
+ * @chain: The notification chain dedicated to this specific event tuple
+ * @hash: The hlist_node used for collision handling
+ * @enabled: A boolean which records if event's generation has been already
+ *	     enabled for this handler as a whole
+ *
+ * This structure collects all the information needed to process a received
+ * event identified by the tuple (proto_id, evt_id, src_id).
+ * These descriptors are stored in a per-protocol @registered_events_handlers
+ * table using as a key a value derived from that tuple.
+ */
+struct scmi_event_handler {
+	u32				key;
+	refcount_t			users;
+	struct scmi_registered_event	*r_evt;
+	struct blocking_notifier_head	chain;
+	struct hlist_node		hash;
+	bool				enabled;
+};
+
+#define IS_HNDL_PENDING(hndl)	((hndl)->r_evt == NULL)
+
+static void scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
+				      struct scmi_event_handler *hndl);
+
 /**
  * scmi_kfifo_free()  - Devres action helper to free the kfifo
  * @kfifo: The kfifo to free
@@ -253,6 +399,10 @@ scmi_allocate_registered_protocol_desc(struct scmi_notify_instance *ni,
 		return ERR_PTR(-ENOMEM);
 	pd->num_events = num_events;
 
+	/* Initialize per protocol handlers table */
+	mutex_init(&pd->registered_mtx);
+	hash_init(pd->registered_events_handlers);
+
 	return pd;
 }
 
@@ -343,6 +493,12 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
 
 	devres_close_group(ni->handle->dev, ni->gid);
 
+	/*
+	 * Finalize any pending events' handler which could have been waiting
+	 * for this protocol's events registration.
+	 */
+	schedule_work(&ni->init_work);
+
 	return 0;
 
 err:
@@ -354,6 +510,539 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
 	return -ENOMEM;
 }
 
+/**
+ * scmi_allocate_event_handler()  - Allocate Event handler
+ * @ni: A reference to the notification instance to use
+ * @evt_key: 32bit key uniquely bind to the event identified by the tuple
+ *	     (proto_id, evt_id, src_id)
+ *
+ * Allocate an event handler and related notification chain associated with
+ * the provided event handler key.
+ * Note that, at this point, a related registered_event is still to be
+ * associated to this handler descriptor (hndl->r_evt == NULL), so the handler
+ * is initialized as pending.
+ *
+ * Context: Assumes to be called with @pending_mtx already acquired.
+ * Return: the freshly allocated structure on Success
+ */
+static struct scmi_event_handler *
+scmi_allocate_event_handler(struct scmi_notify_instance *ni, u32 evt_key)
+{
+	struct scmi_event_handler *hndl;
+
+	hndl = kzalloc(sizeof(*hndl), GFP_KERNEL);
+	if (!hndl)
+		return ERR_PTR(-ENOMEM);
+	hndl->key = evt_key;
+	BLOCKING_INIT_NOTIFIER_HEAD(&hndl->chain);
+	refcount_set(&hndl->users, 1);
+	/* New handlers are created pending */
+	hash_add(ni->pending_events_handlers, &hndl->hash, hndl->key);
+
+	return hndl;
+}
+
+/**
+ * scmi_free_event_handler()  - Free the provided Event handler
+ * @hndl: The event handler structure to free
+ *
+ * Context: Assumes to be called with proper locking acquired depending
+ *	    on the situation.
+ */
+static void scmi_free_event_handler(struct scmi_event_handler *hndl)
+{
+	hash_del(&hndl->hash);
+	kfree(hndl);
+}
+
+/**
+ * scmi_bind_event_handler()  - Helper to attempt binding an handler to an event
+ * @ni: A reference to the notification instance to use
+ * @hndl: The event handler to bind
+ *
+ * If an associated registered event is found, move the handler from the pending
+ * into the registered table.
+ *
+ * Context: Assumes to be called with @pending_mtx already acquired.
+ * Return: True if bind was successful, False otherwise
+ */
+static inline bool scmi_bind_event_handler(struct scmi_notify_instance *ni,
+					   struct scmi_event_handler *hndl)
+{
+	struct scmi_registered_event *r_evt;
+
+
+	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(hndl->key),
+			      KEY_XTRACT_EVT_ID(hndl->key));
+	if (unlikely(!r_evt))
+		return false;
+
+	/* Remove from pending and insert into registered */
+	hash_del(&hndl->hash);
+	hndl->r_evt = r_evt;
+	mutex_lock(&r_evt->proto->registered_mtx);
+	hash_add(r_evt->proto->registered_events_handlers,
+		 &hndl->hash, hndl->key);
+	mutex_unlock(&r_evt->proto->registered_mtx);
+
+	return true;
+}
+
+/**
+ * scmi_valid_pending_handler()  - Helper to check pending status of handlers
+ * @ni: A reference to the notification instance to use
+ * @hndl: The event handler to check
+ *
+ * An handler is considered pending when its r_evt == NULL, because the related
+ * event was still unknown at handler's registration time; anyway, since all
+ * protocols register their supported events once for all at protocols'
+ * initialization time, a pending handler cannot be considered valid anymore if
+ * the underlying event (which it is waiting for), belongs to an already
+ * initialized and registered protocol.
+ *
+ * Return: True if pending registration is still valid, False otherwise.
+ */
+static inline bool scmi_valid_pending_handler(struct scmi_notify_instance *ni,
+					      struct scmi_event_handler *hndl)
+{
+	struct scmi_registered_protocol_events_desc *pd;
+
+	if (unlikely(!IS_HNDL_PENDING(hndl)))
+		return false;
+
+	pd = SCMI_GET_PROTO(ni, KEY_XTRACT_PROTO_ID(hndl->key));
+	if (pd)
+		return false;
+
+	return true;
+}
+
+/**
+ * scmi_register_event_handler()  - Register whenever possible an Event handler
+ * @ni: A reference to the notification instance to use
+ * @hndl: The event handler to register
+ *
+ * At first try to bind an event handler to its associated event, then check if
+ * it was at least a valid pending handler: if it was not bound nor valid return
+ * false.
+ *
+ * Valid pending incomplete bindings will be periodically retried by a dedicated
+ * worker which is kicked each time a new protocol completes its own
+ * registration phase.
+ *
+ * Context: Assumes to be called with @pending_mtx acquired.
+ * Return: True if a normal or a valid pending registration has been completed,
+ *	   False otherwise
+ */
+static bool scmi_register_event_handler(struct scmi_notify_instance *ni,
+					struct scmi_event_handler *hndl)
+{
+	bool ret;
+
+	ret = scmi_bind_event_handler(ni, hndl);
+	if (ret) {
+		pr_info("SCMI Notifications: registered NEW handler - key:%X\n",
+			hndl->key);
+	} else {
+		ret = scmi_valid_pending_handler(ni, hndl);
+		if (ret)
+			pr_info("SCMI Notifications: registered PENDING handler - key:%X\n",
+				hndl->key);
+	}
+
+	return ret;
+}
+
+/**
+ * __scmi_event_handler_get_ops()  - Utility to get or create an event handler
+ * @ni: A reference to the notification instance to use
+ * @evt_key: The event key to use
+ * @create: A boolean flag to specify if a handler must be created when
+ *	    not already existent
+ *
+ * Search for the desired handler matching the key in both the per-protocol
+ * registered table and the common pending table:
+ * * if found adjust users refcount
+ * * if not found and @create is true, create and register the new handler:
+ *   handler could end up being registered as pending if no matching event
+ *   could be found.
+ *
+ * An handler is guaranteed to reside in one and only one of the tables at
+ * any one time; to ensure this the whole search and create is performed
+ * holding the @pending_mtx lock, with @registered_mtx additionally acquired
+ * if needed.
+ *
+ * Note that when a nested acquisition of these mutexes is needed the locking
+ * order is always (same as in @init_work):
+ * 1. pending_mtx
+ * 2. registered_mtx
+ *
+ * Events generation is NOT enabled right after creation within this routine
+ * since at creation time we usually want to have all setup and ready before
+ * events really start flowing.
+ *
+ * Return: A properly refcounted handler on Success, NULL on Failure
+ */
+static inline struct scmi_event_handler *
+__scmi_event_handler_get_ops(struct scmi_notify_instance *ni,
+			     u32 evt_key, bool create)
+{
+	struct scmi_registered_event *r_evt;
+	struct scmi_event_handler *hndl = NULL;
+
+	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(evt_key),
+			      KEY_XTRACT_EVT_ID(evt_key));
+
+	mutex_lock(&ni->pending_mtx);
+	/* Search registered events at first ... if possible at all */
+	if (likely(r_evt)) {
+		mutex_lock(&r_evt->proto->registered_mtx);
+		hndl = KEY_FIND(r_evt->proto->registered_events_handlers,
+				hndl, evt_key);
+		if (likely(hndl))
+			refcount_inc(&hndl->users);
+		mutex_unlock(&r_evt->proto->registered_mtx);
+	}
+
+	/* ...then amongst pending. */
+	if (unlikely(!hndl)) {
+		hndl = KEY_FIND(ni->pending_events_handlers, hndl, evt_key);
+		if (likely(hndl))
+			refcount_inc(&hndl->users);
+	}
+
+	/* Create if still not found and required */
+	if (!hndl && create) {
+		hndl = scmi_allocate_event_handler(ni, evt_key);
+		if (!IS_ERR_OR_NULL(hndl)) {
+			if (!scmi_register_event_handler(ni, hndl)) {
+				pr_info("SCMI Notifications: purging UNKNOWN handler - key:%X\n",
+					hndl->key);
+				/* this hndl can be only a pending one */
+				scmi_put_handler_unlocked(ni, hndl);
+				hndl = NULL;
+			}
+		}
+	}
+	mutex_unlock(&ni->pending_mtx);
+
+	return hndl;
+}
+
+static struct scmi_event_handler *
+scmi_get_handler(struct scmi_notify_instance *ni, u32 evt_key)
+{
+	return __scmi_event_handler_get_ops(ni, evt_key, false);
+}
+
+static struct scmi_event_handler *
+scmi_get_or_create_handler(struct scmi_notify_instance *ni, u32 evt_key)
+{
+	return __scmi_event_handler_get_ops(ni, evt_key, true);
+}
+
+/**
+ * __scmi_enable_evt()  - Enable/disable events generation
+ * @r_evt: The registered event to act upon
+ * @src_id: The src_id to act upon
+ * @enable: The action to perform: true->Enable, false->Disable
+ *
+ * Takes care of proper refcounting while performing enable/disable: handles
+ * the special case of ALL sources requests by itself.
+ *
+ * Return: True when the required action has been successfully executed
+ */
+static inline bool __scmi_enable_evt(struct scmi_registered_event *r_evt,
+				     u32 src_id, bool enable)
+{
+	int ret = 0;
+	u32 num_sources;
+	refcount_t *sid;
+
+	if (src_id == SCMI_ALL_SRC_IDS) {
+		src_id = 0;
+		num_sources = r_evt->num_sources;
+	} else if (src_id < r_evt->num_sources) {
+		num_sources = 1;
+	} else {
+		return ret;
+	}
+
+	mutex_lock(&r_evt->sources_mtx);
+	if (enable) {
+		for (; num_sources; src_id++, num_sources--) {
+			bool r;
+
+			sid = &r_evt->sources[src_id];
+			if (refcount_read(sid) == 0) {
+				r = REVT_NOTIFY_ENABLE(r_evt,
+						       r_evt->evt->id, src_id);
+				if (r)
+					refcount_set(sid, 1);
+			} else {
+				refcount_inc(sid);
+				r = true;
+			}
+			ret += r;
+		}
+	} else {
+		for (; num_sources; src_id++, num_sources--) {
+			sid = &r_evt->sources[src_id];
+			if (refcount_dec_and_test(sid))
+				REVT_NOTIFY_DISABLE(r_evt,
+						    r_evt->evt->id, src_id);
+		}
+		ret = 1;
+	}
+	mutex_unlock(&r_evt->sources_mtx);
+
+	return ret;
+}
+
+static bool scmi_enable_events(struct scmi_event_handler *hndl)
+{
+	if (!hndl->enabled)
+		hndl->enabled = __scmi_enable_evt(hndl->r_evt,
+						  KEY_XTRACT_SRC_ID(hndl->key),
+						  true);
+	return hndl->enabled;
+}
+
+static bool scmi_disable_events(struct scmi_event_handler *hndl)
+{
+	if (hndl->enabled)
+		hndl->enabled = !__scmi_enable_evt(hndl->r_evt,
+						   KEY_XTRACT_SRC_ID(hndl->key),
+						   false);
+	return !hndl->enabled;
+}
+
+/**
+ * scmi_put_handler_unlocked()  - Put an event handler
+ * @ni: A reference to the notification instance to use
+ * @hndl: The event handler to act upon
+ *
+ * After having got exclusive access to the registered handlers hashtable,
+ * update the refcount and if @hndl is no more in use by anyone:
+ * * ask for events' generation disabling
+ * * unregister and free the handler itself
+ *
+ * Context: Assumes all the proper locking has been managed by the caller.
+ */
+static void
+scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
+				struct scmi_event_handler *hndl)
+{
+	if (refcount_dec_and_test(&hndl->users)) {
+		if (likely(!IS_HNDL_PENDING(hndl)))
+			scmi_disable_events(hndl);
+		scmi_free_event_handler(hndl);
+	}
+}
+
+static void scmi_put_handler(struct scmi_notify_instance *ni,
+			     struct scmi_event_handler *hndl)
+{
+	struct scmi_registered_event *r_evt = hndl->r_evt;
+
+	mutex_lock(&ni->pending_mtx);
+	if (r_evt)
+		mutex_lock(&r_evt->proto->registered_mtx);
+
+	scmi_put_handler_unlocked(ni, hndl);
+
+	if (r_evt)
+		mutex_unlock(&r_evt->proto->registered_mtx);
+	mutex_unlock(&ni->pending_mtx);
+}
+
+/**
+ * scmi_event_handler_enable_events()  - Enable events associated to an handler
+ * @hndl: The Event handler to act upon
+ *
+ * Return: True on success
+ */
+static bool scmi_event_handler_enable_events(struct scmi_event_handler *hndl)
+{
+	if (!scmi_enable_events(hndl)) {
+		pr_err("SCMI Notifications: Failed to ENABLE events for key:%X !\n",
+		       hndl->key);
+		return false;
+	}
+
+	return true;
+}
+
+/**
+ * scmi_register_notifier()  - Register a notifier_block for an event
+ * @handle: The handle identifying the platform instance against which the
+ *	    callback is registered
+ * @proto_id: Protocol ID
+ * @evt_id: Event ID
+ * @src_id: Source ID, when NULL register for events coming form ALL possible
+ *	    sources
+ * @nb: A standard notifier block to register for the specified event
+ *
+ * Generic helper to register a notifier_block against a protocol event.
+ *
+ * A notifier_block @nb will be registered for each distinct event identified
+ * by the tuple (proto_id, evt_id, src_id) on a dedicated notification chain
+ * so that:
+ *
+ *	(proto_X, evt_Y, src_Z) --> chain_X_Y_Z
+ *
+ * @src_id meaning is protocol specific and identifies the origin of the event
+ * (like domain_id, sensor_id and so forth).
+ *
+ * @src_id can be NULL to signify that the caller is interested in receiving
+ * notifications from ALL the available sources for that protocol OR simply that
+ * the protocol does not support distinct sources.
+ *
+ * As soon as one user for the specified tuple appears, an handler is created,
+ * and that specific event's generation is enabled at the platform level, unless
+ * an associated registered event is found missing, meaning that the needed
+ * protocol is still to be initialized and the handler has just been registered
+ * as still pending.
+ *
+ * Return: Return 0 on Success
+ */
+static int scmi_register_notifier(const struct scmi_handle *handle,
+				  u8 proto_id, u8 evt_id, u32 *src_id,
+				  struct notifier_block *nb)
+{
+	int ret = 0;
+	u32 evt_key;
+	struct scmi_event_handler *hndl;
+	struct scmi_notify_instance *ni;
+
+	/* Ensure notify_priv is updated */
+	smp_rmb();
+	if (unlikely(!handle->notify_priv))
+		return -ENODEV;
+	ni = handle->notify_priv;
+
+	evt_key = MAKE_HASH_KEY(proto_id, evt_id,
+				src_id ? *src_id : SCMI_ALL_SRC_IDS);
+	hndl = scmi_get_or_create_handler(ni, evt_key);
+	if (IS_ERR_OR_NULL(hndl))
+		return PTR_ERR(hndl);
+
+	blocking_notifier_chain_register(&hndl->chain, nb);
+
+	/* Enable events for not pending handlers */
+	if (likely(!IS_HNDL_PENDING(hndl))) {
+		if (!scmi_event_handler_enable_events(hndl)) {
+			scmi_put_handler(ni, hndl);
+			ret = -EINVAL;
+		}
+	}
+
+	return ret;
+}
+
+/**
+ * scmi_unregister_notifier()  - Unregister a notifier_block for an event
+ * @handle: The handle identifying the platform instance against which the
+ *	    callback is unregistered
+ * @proto_id: Protocol ID
+ * @evt_id: Event ID
+ * @src_id: Source ID
+ * @nb: The notifier_block to unregister
+ *
+ * Takes care to unregister the provided @nb from the notification chain
+ * associated to the specified event and, if there are no more users for the
+ * event handler, frees also the associated event handler structures.
+ * (this could possibly cause disabling of event's generation at platform level)
+ *
+ * Return: 0 on Success
+ */
+static int scmi_unregister_notifier(const struct scmi_handle *handle,
+				    u8 proto_id, u8 evt_id, u32 *src_id,
+				    struct notifier_block *nb)
+{
+	u32 evt_key;
+	struct scmi_event_handler *hndl;
+	struct scmi_notify_instance *ni;
+
+	/* Ensure notify_priv is updated */
+	smp_rmb();
+	if (unlikely(!handle->notify_priv))
+		return -ENODEV;
+	ni = handle->notify_priv;
+
+	evt_key = MAKE_HASH_KEY(proto_id, evt_id,
+				src_id ? *src_id : SCMI_ALL_SRC_IDS);
+	hndl = scmi_get_handler(ni, evt_key);
+	if (IS_ERR_OR_NULL(hndl))
+		return -EINVAL;
+
+	blocking_notifier_chain_unregister(&hndl->chain, nb);
+	scmi_put_handler(ni, hndl);
+
+	/*
+	 * Free the handler (and stop events) if this happens to be the last
+	 * known user callback for this handler; a possible concurrently ongoing
+	 * run of @scmi_lookup_and_call_event_chain will cause this to happen
+	 * in that context safely instead.
+	 */
+	scmi_put_handler(ni, hndl);
+
+	return 0;
+}
+
+/**
+ * scmi_protocols_late_init()  - Worker for late initialization
+ * @work: The work item to use associated to the proper SCMI instance
+ *
+ * This kicks in whenever a new protocol has completed its own registration via
+ * scmi_register_protocol_events(): it is in charge of scanning the table of
+ * pending handlers (registered by users while the related protocol was still
+ * not initialized) and finalizing their initialization whenever possible;
+ * invalid pending handlers are purged at this point in time.
+ */
+static void scmi_protocols_late_init(struct work_struct *work)
+{
+	int bkt;
+	struct scmi_event_handler *hndl;
+	struct scmi_notify_instance *ni;
+	struct hlist_node *tmp;
+
+	ni = container_of(work, struct scmi_notify_instance, init_work);
+
+	/* Ensure protocols and events are up to date */
+	smp_rmb();
+
+	mutex_lock(&ni->pending_mtx);
+	hash_for_each_safe(ni->pending_events_handlers, bkt, tmp, hndl, hash) {
+		bool ret;
+
+		ret = scmi_bind_event_handler(ni, hndl);
+		if (ret) {
+			pr_info("SCMI Notifications: finalized PENDING handler - key:%X\n",
+				hndl->key);
+			ret = scmi_event_handler_enable_events(hndl);
+		} else {
+			ret = scmi_valid_pending_handler(ni, hndl);
+		}
+		if (!ret) {
+			pr_info("SCMI Notifications: purging PENDING handler - key:%X\n",
+				hndl->key);
+			/* this hndl can be only a pending one */
+			scmi_put_handler_unlocked(ni, hndl);
+		}
+	}
+	mutex_unlock(&ni->pending_mtx);
+}
+
+/*
+ * notify_ops are attached to the handle so that can be accessed
+ * directly from an scmi_driver to register its own notifiers.
+ */
+static struct scmi_notify_ops notify_ops = {
+	.register_event_notifier = scmi_register_notifier,
+	.unregister_event_notifier = scmi_unregister_notifier,
+};
+
 /**
  * scmi_notification_init()  - Initializes Notification Core Support
  * @handle: The handle identifying the platform instance to initialize
@@ -401,6 +1090,12 @@ int scmi_notification_init(struct scmi_handle *handle)
 	if (!ni->registered_protocols)
 		goto err;
 
+	mutex_init(&ni->pending_mtx);
+	hash_init(ni->pending_events_handlers);
+
+	INIT_WORK(&ni->init_work, scmi_protocols_late_init);
+
+	handle->notify_ops = &notify_ops;
 	handle->notify_priv = ni;
 	/* Ensure handle is up to date */
 	smp_wmb();
diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
index 54094aaf812a..f0561fb30970 100644
--- a/drivers/firmware/arm_scmi/notify.h
+++ b/drivers/firmware/arm_scmi/notify.h
@@ -9,9 +9,21 @@
 #ifndef _SCMI_NOTIFY_H
 #define _SCMI_NOTIFY_H
 
+#include <linux/bug.h>
 #include <linux/device.h>
 #include <linux/types.h>
 
+#define MAP_EVT_TO_ENABLE_CMD(id, map)			\
+({							\
+	int ret = -1;					\
+							\
+	if (likely((id) < ARRAY_SIZE((map))))		\
+		ret = (map)[(id)];			\
+	else						\
+		WARN(1, "UN-KNOWN evt_id:%d\n", (id));	\
+	ret;						\
+})
+
 /**
  * struct scmi_event  - Describes an event to be supported
  * @id: Event ID
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index 0679f10ab05e..0fb97a589b30 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -9,6 +9,7 @@
 #define _LINUX_SCMI_PROTOCOL_H
 
 #include <linux/device.h>
+#include <linux/notifier.h>
 #include <linux/types.h>
 
 #define SCMI_MAX_STR_SIZE	16
@@ -211,6 +212,49 @@ struct scmi_reset_ops {
 	int (*deassert)(const struct scmi_handle *handle, u32 domain);
 };
 
+/**
+ * struct scmi_notify_ops  - represents notifications' operations provided by
+ * SCMI core
+ * @register_event_notifier: Register a notifier_block for the requested event
+ * @unregister_event_notifier: Unregister a notifier_block for the requested
+ *			       event
+ *
+ * A user can register/unregister its own notifier_block against the wanted
+ * platform instance regarding the desired event identified by the
+ * tuple: (proto_id, evt_id, src_id) using the provided register/unregister
+ * interface where:
+ *
+ * @handle: The handle identifying the platform instance to use
+ * @proto_id: The protocol ID as in SCMI Specification
+ * @evt_id: The message ID of the desired event as in SCMI Specification
+ * @src_id: A pointer to the desired source ID if different sources are
+ *	    possible for the protocol (like domain_id, sensor_id...etc)
+ *
+ * @src_id can be provided as NULL if it simply does NOT make sense for
+ * the protocol at hand, OR if the user is explicitly interested in
+ * receiving notifications from ANY existent source associated to the
+ * specified proto_id / evt_id.
+ *
+ * Received notifications are finally delivered to the registered users,
+ * invoking the callback provided with the notifier_block *nb as follows:
+ *
+ *	int user_cb(nb, evt_id, report)
+ *
+ * with:
+ *
+ * @nb: The notifier block provided by the user
+ * @evt_id: The message ID of the delivered event
+ * @report: A custom struct describing the specific event delivered
+ */
+struct scmi_notify_ops {
+	int (*register_event_notifier)(const struct scmi_handle *handle,
+				       u8 proto_id, u8 evt_id, u32 *src_id,
+				       struct notifier_block *nb);
+	int (*unregister_event_notifier)(const struct scmi_handle *handle,
+					 u8 proto_id, u8 evt_id, u32 *src_id,
+					 struct notifier_block *nb);
+};
+
 /**
  * struct scmi_handle - Handle returned to ARM SCMI clients for usage.
  *
@@ -221,6 +265,7 @@ struct scmi_reset_ops {
  * @clk_ops: pointer to set of clock protocol operations
  * @sensor_ops: pointer to set of sensor protocol operations
  * @reset_ops: pointer to set of reset protocol operations
+ * @notify_ops: pointer to set of notifications related operations
  * @perf_priv: pointer to private data structure specific to performance
  *	protocol(for internal use only)
  * @clk_priv: pointer to private data structure specific to clock
@@ -242,6 +287,7 @@ struct scmi_handle {
 	struct scmi_power_ops *power_ops;
 	struct scmi_sensor_ops *sensor_ops;
 	struct scmi_reset_ops *reset_ops;
+	struct scmi_notify_ops *notify_ops;
 	/* for protocol internal use */
 	void *perf_priv;
 	void *clk_priv;
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 3/9] firmware: arm_scmi: Add notification dispatch and delivery

[Cristian Marussi]

Add core SCMI Notifications dispatch and delivery support logic which is
able, at first, to dispatch well-known received events from the RX ISR to
the dedicated deferred worker, and then, from there, to final deliver the
events to the registered users' callbacks.

Dispatch and delivery is just added here, still not enabled.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V7 --> V8
- Fixed enabled check in scmi_notify() not to use atomics
- Added a few comments about queueing works
V5 --> V6
- added handle argument to fill_custom_report()
V4 --> V5
- fixed kernel-doc
- fixed unneded var initialization
V3 --> V4
- dispatcher now handles dequeuing of events in chunks (header+payload):
  handling of these in_flight events let us remove one unneeded memcpy
  on RX interrupt path (scmi_notify)
- deferred dispatcher now access their own per-protocol handlers' table
  reducing locking contention on the RX path
V2 --> V3
- exposing wq in sysfs via WQ_SYSFS
V1 --> V2
- splitted out of V1 patch 04
- moved from IDR maps to real HashTables to store event_handlers
- simplified delivery logic
---
 drivers/firmware/arm_scmi/notify.c | 354 ++++++++++++++++++++++++++++-
 drivers/firmware/arm_scmi/notify.h |  10 +
 2 files changed, 362 insertions(+), 2 deletions(-)

diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
index 7cf61dbe2a8e..d582f71fde5b 100644
--- a/drivers/firmware/arm_scmi/notify.c
+++ b/drivers/firmware/arm_scmi/notify.c
@@ -47,6 +47,27 @@
  * as described in the SCMI Protocol specification, while src_id represents an
  * optional, protocol dependent, source identifier (like domain_id, perf_id
  * or sensor_id and so forth).
+ *
+ * Upon reception of a notification message from the platform the SCMI RX ISR
+ * passes the received message payload and some ancillary information (including
+ * an arrival timestamp in nanoseconds) to the core via @scmi_notify() which
+ * pushes the event-data itself on a protocol-dedicated kfifo queue for further
+ * deferred processing as specified in @scmi_events_dispatcher().
+ *
+ * Each protocol has it own dedicated work_struct and worker which, once kicked
+ * by the ISR, takes care to empty its own dedicated queue, deliverying the
+ * queued items into the proper notification-chain: notifications processing can
+ * proceed concurrently on distinct workers only between events belonging to
+ * different protocols while delivery of events within the same protocol is
+ * still strictly sequentially ordered by time of arrival.
+ *
+ * Events' information is then extracted from the SCMI Notification messages and
+ * conveyed, converted into a custom per-event report struct, as the void *data
+ * param to the user callback provided by the registered notifier_block, so that
+ * from the user perspective his callback will look invoked like:
+ *
+ * int user_cb(struct notifier_block *nb, unsigned long event_id, void *report)
+ *
  */
 
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
@@ -66,6 +87,7 @@
 #include <linux/scmi_protocol.h>
 #include <linux/slab.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 
 #include "notify.h"
 
@@ -148,6 +170,9 @@
 #define REVT_NOTIFY_DISABLE(revt, eid, sid)				       \
 	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
 						(eid), (sid), false))
+#define REVT_FILL_REPORT(revt, ...)					       \
+	((revt)->proto->ops->fill_custom_report((revt)->proto->ni->handle,     \
+						__VA_ARGS__))
 
 struct scmi_registered_protocol_events_desc;
 
@@ -157,6 +182,7 @@ struct scmi_registered_protocol_events_desc;
  * @gid: GroupID used for devres
  * @handle: A reference to the platform instance
  * @init_work: A work item to perform final initializations of pending handlers
+ * @notify_wq: A reference to the allocated Kernel cmwq
  * @pending_mtx: A mutex to protect @pending_events_handlers
  * @registered_protocols: A statically allocated array containing pointers to
  *			  all the registered protocol-level specific information
@@ -173,6 +199,8 @@ struct scmi_notify_instance {
 
 	struct work_struct				init_work;
 
+	struct workqueue_struct				*notify_wq;
+
 	struct mutex					pending_mtx;
 	struct scmi_registered_protocol_events_desc	**registered_protocols;
 	DECLARE_HASHTABLE(pending_events_handlers, 8);
@@ -182,12 +210,16 @@ struct scmi_notify_instance {
  * struct events_queue  - Describes a queue and its associated worker
  * @sz: Size in bytes of the related kfifo
  * @kfifo: A dedicated Kernel kfifo descriptor
+ * @notify_work: A custom work item bound to this queue
+ * @wq: A reference to the associated workqueue
  *
  * Each protocol has its own dedicated events_queue descriptor.
  */
 struct events_queue {
 	size_t				sz;
 	struct kfifo			kfifo;
+	struct work_struct		notify_work;
+	struct workqueue_struct		*wq;
 };
 
 /**
@@ -309,9 +341,264 @@ struct scmi_event_handler {
 
 #define IS_HNDL_PENDING(hndl)	((hndl)->r_evt == NULL)
 
+static struct scmi_event_handler *
+scmi_get_active_handler(struct scmi_notify_instance *ni, u32 evt_key);
+static void scmi_put_active_handler(struct scmi_notify_instance *ni,
+				    struct scmi_event_handler *hndl);
 static void scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
 				      struct scmi_event_handler *hndl);
 
+/**
+ * scmi_lookup_and_call_event_chain()  - Lookup the proper chain and call it
+ * @ni: A reference to the notification instance to use
+ * @evt_key: The key to use to lookup the related notification chain
+ * @report: The customized event-specific report to pass down to the callbacks
+ *	    as their *data parameter.
+ */
+static inline void
+scmi_lookup_and_call_event_chain(struct scmi_notify_instance *ni,
+				 u32 evt_key, void *report)
+{
+	int ret;
+	struct scmi_event_handler *hndl;
+
+	/* Here ensure the event handler cannot vanish while using it */
+	hndl = scmi_get_active_handler(ni, evt_key);
+	if (IS_ERR_OR_NULL(hndl))
+		return;
+
+	ret = blocking_notifier_call_chain(&hndl->chain,
+					   KEY_XTRACT_EVT_ID(evt_key),
+					   report);
+	/* Notifiers are NOT supposed to cut the chain ... */
+	WARN_ON_ONCE(ret & NOTIFY_STOP_MASK);
+
+	scmi_put_active_handler(ni, hndl);
+}
+
+/**
+ * scmi_process_event_header()  - Dequeue and process an event header
+ * @eq: The queue to use
+ * @pd: The protocol descriptor to use
+ *
+ * Read an event header from the protocol queue into the dedicated scratch
+ * buffer and looks for a matching registered event; in case an anomalously
+ * sized read is detected just flush the queue.
+ *
+ * Return:
+ * * a reference to the matching registered event when found
+ * * ERR_PTR(-EINVAL) when NO registered event could be found
+ * * NULL when the queue is empty
+ */
+static inline struct scmi_registered_event *
+scmi_process_event_header(struct events_queue *eq,
+			  struct scmi_registered_protocol_events_desc *pd)
+{
+	unsigned int outs;
+	struct scmi_registered_event *r_evt;
+
+	outs = kfifo_out(&eq->kfifo, pd->eh,
+			 sizeof(struct scmi_event_header));
+	if (!outs)
+		return NULL;
+	if (outs != sizeof(struct scmi_event_header)) {
+		pr_err("SCMI Notifications: corrupted EVT header. Flush.\n");
+		kfifo_reset_out(&eq->kfifo);
+		return NULL;
+	}
+
+	r_evt = SCMI_GET_REVT_FROM_PD(pd, pd->eh->evt_id);
+	if (!r_evt)
+		r_evt = ERR_PTR(-EINVAL);
+
+	return r_evt;
+}
+
+/**
+ * scmi_process_event_payload()  - Dequeue and process an event payload
+ * @eq: The queue to use
+ * @pd: The protocol descriptor to use
+ * @r_evt: The registered event descriptor to use
+ *
+ * Read an event payload from the protocol queue into the dedicated scratch
+ * buffer, fills a custom report and then look for matching event handlers and
+ * call them; skip any unknown event (as marked by scmi_process_event_header())
+ * and in case an anomalously sized read is detected just flush the queue.
+ *
+ * Return: False when the queue is empty
+ */
+static inline bool
+scmi_process_event_payload(struct events_queue *eq,
+			   struct scmi_registered_protocol_events_desc *pd,
+			   struct scmi_registered_event *r_evt)
+{
+	u32 src_id, key;
+	unsigned int outs;
+	void *report = NULL;
+
+	outs = kfifo_out(&eq->kfifo, pd->eh->payld, pd->eh->payld_sz);
+	if (unlikely(!outs))
+		return false;
+
+	/* Any in-flight event has now been officially processed */
+	pd->in_flight = NULL;
+
+	if (unlikely(outs != pd->eh->payld_sz)) {
+		pr_err("SCMI Notifications: corrupted EVT Payload. Flush.\n");
+		kfifo_reset_out(&eq->kfifo);
+		return false;
+	}
+
+	if (IS_ERR(r_evt)) {
+		pr_warn("SCMI Notifications: SKIP UNKNOWN EVT - proto:%X  evt:%d\n",
+			pd->id, pd->eh->evt_id);
+		return true;
+	}
+
+	report = REVT_FILL_REPORT(r_evt, pd->eh->evt_id, pd->eh->timestamp,
+				  pd->eh->payld, pd->eh->payld_sz,
+				  r_evt->report, &src_id);
+	if (!report) {
+		pr_err("SCMI Notifications: Report not available - proto:%X  evt:%d\n",
+		       pd->id, pd->eh->evt_id);
+		return true;
+	}
+
+	/* At first search for a generic ALL src_ids handler... */
+	key = MAKE_ALL_SRCS_KEY(pd->id, pd->eh->evt_id);
+	scmi_lookup_and_call_event_chain(pd->ni, key, report);
+
+	/* ...then search for any specific src_id */
+	key = MAKE_HASH_KEY(pd->id, pd->eh->evt_id, src_id);
+	scmi_lookup_and_call_event_chain(pd->ni, key, report);
+
+	return true;
+}
+
+/**
+ * scmi_events_dispatcher()  - Common worker logic for all work items.
+ * @work: The work item to use, which is associated to a dedicated events_queue
+ *
+ * Logic:
+ *  1. dequeue one pending RX notification (queued in SCMI RX ISR context)
+ *  2. generate a custom event report from the received event message
+ *  3. lookup for any registered ALL_SRC_IDs handler:
+ *    - > call the related notification chain passing in the report
+ *  4. lookup for any registered specific SRC_ID handler:
+ *    - > call the related notification chain passing in the report
+ *
+ * Note that:
+ * * a dedicated per-protocol kfifo queue is used: in this way an anomalous
+ *   flood of events cannot saturate other protocols' queues.
+ * * each per-protocol queue is associated to a distinct work_item, which
+ *   means, in turn, that:
+ *   + all protocols can process their dedicated queues concurrently
+ *     (since notify_wq:max_active != 1)
+ *   + anyway at most one worker instance is allowed to run on the same queue
+ *     concurrently: this ensures that we can have only one concurrent
+ *     reader/writer on the associated kfifo, so that we can use it lock-less
+ *
+ * Context: Process context.
+ */
+static void scmi_events_dispatcher(struct work_struct *work)
+{
+	struct events_queue *eq;
+	struct scmi_registered_protocol_events_desc *pd;
+	struct scmi_registered_event *r_evt;
+
+	eq = container_of(work, struct events_queue, notify_work);
+	pd = container_of(eq, struct scmi_registered_protocol_events_desc,
+			  equeue);
+	/*
+	 * In order to keep the queue lock-less and the number of memcopies
+	 * to the bare minimum needed, the dispatcher accounts for the
+	 * possibility of per-protocol in-flight events: i.e. an event whose
+	 * reception could end up being split across two subsequent runs of this
+	 * worker, first the header, then the payload.
+	 */
+	do {
+		if (likely(!pd->in_flight)) {
+			r_evt = scmi_process_event_header(eq, pd);
+			if (!r_evt)
+				break;
+			pd->in_flight = r_evt;
+		} else {
+			r_evt = pd->in_flight;
+		}
+	} while (scmi_process_event_payload(eq, pd, r_evt));
+}
+
+/**
+ * scmi_notify()  - Queues a notification for further deferred processing
+ * @handle: The handle identifying the platform instance from which the
+ *	    dispatched event is generated
+ * @proto_id: Protocol ID
+ * @evt_id: Event ID (msgID)
+ * @buf: Event Message Payload (without the header)
+ * @len: Event Message Payload size
+ * @ts: RX Timestamp in nanoseconds (boottime)
+ *
+ * Context: Called in interrupt context to queue a received event for
+ * deferred processing.
+ *
+ * Return: 0 on Success
+ */
+int scmi_notify(const struct scmi_handle *handle, u8 proto_id, u8 evt_id,
+		const void *buf, size_t len, u64 ts)
+{
+	struct scmi_registered_event *r_evt;
+	struct scmi_event_header eh;
+	struct scmi_notify_instance *ni;
+
+	/* Ensure notify_priv is updated */
+	smp_rmb();
+	if (unlikely(!handle->notify_priv))
+		return 0;
+	ni = handle->notify_priv;
+
+	r_evt = SCMI_GET_REVT(ni, proto_id, evt_id);
+	if (unlikely(!r_evt))
+		return -EINVAL;
+
+	if (unlikely(len > r_evt->evt->max_payld_sz)) {
+		pr_err("SCMI Notifications: discard badly sized message\n");
+		return -EINVAL;
+	}
+	if (unlikely(kfifo_avail(&r_evt->proto->equeue.kfifo) <
+		     sizeof(eh) + len)) {
+		pr_warn("SCMI Notifications: queue full dropping proto_id:%d  evt_id:%d  ts:%lld\n",
+			proto_id, evt_id, ts);
+		return -ENOMEM;
+	}
+
+	eh.timestamp = ts;
+	eh.evt_id = evt_id;
+	eh.payld_sz = len;
+	/*
+	 * Header and payload are enqueued with two distinct kfifo_in() (so non
+	 * atomic), but this situation is handled properly on the consumer side
+	 * with in-flight events tracking.
+	 */
+	kfifo_in(&r_evt->proto->equeue.kfifo, &eh, sizeof(eh));
+	kfifo_in(&r_evt->proto->equeue.kfifo, buf, len);
+	/*
+	 * Don't care about return value here since we just want to ensure that
+	 * a work is queued all the times whenever some items have been pushed
+	 * on the kfifo:
+	 * - if work was already queued it will simply fail to queue a new one
+	 *   since it is not needed
+	 * - if work was not queued already it will be now, even in case work
+	 *   was in fact already running: this behavior avoids any possible race
+	 *   when this function pushes new items onto the kfifos after the
+	 *   related executing worker had already determined the kfifo to be
+	 *   empty and it was terminating.
+	 */
+	queue_work(r_evt->proto->equeue.wq,
+		   &r_evt->proto->equeue.notify_work);
+
+	return 0;
+}
+
 /**
  * scmi_kfifo_free()  - Devres action helper to free the kfifo
  * @kfifo: The kfifo to free
@@ -334,13 +621,22 @@ static void scmi_kfifo_free(void *kfifo)
 static int scmi_initialize_events_queue(struct scmi_notify_instance *ni,
 					struct events_queue *equeue, size_t sz)
 {
+	int ret;
+
 	if (kfifo_alloc(&equeue->kfifo, sz, GFP_KERNEL))
 		return -ENOMEM;
 	/* Size could have been roundup to power-of-two */
 	equeue->sz = kfifo_size(&equeue->kfifo);
 
-	return devm_add_action_or_reset(ni->handle->dev, scmi_kfifo_free,
-					&equeue->kfifo);
+	ret = devm_add_action_or_reset(ni->handle->dev, scmi_kfifo_free,
+				       &equeue->kfifo);
+	if (ret)
+		return ret;
+
+	INIT_WORK(&equeue->notify_work, scmi_events_dispatcher);
+	equeue->wq = ni->notify_wq;
+
+	return ret;
 }
 
 /**
@@ -741,6 +1037,37 @@ scmi_get_or_create_handler(struct scmi_notify_instance *ni, u32 evt_key)
 	return __scmi_event_handler_get_ops(ni, evt_key, true);
 }
 
+/**
+ * scmi_get_active_handler()  - Helper to get active handlers only
+ * @ni: A reference to the notification instance to use
+ * @evt_key: The event key to use
+ *
+ * Search for the desired handler matching the key only in the per-protocol
+ * table of registered handlers: this is called only from the dispatching path
+ * so want to be as quick as possible and do not care about pending.
+ *
+ * Return: A properly refcounted active handler
+ */
+static struct scmi_event_handler *
+scmi_get_active_handler(struct scmi_notify_instance *ni, u32 evt_key)
+{
+	struct scmi_registered_event *r_evt;
+	struct scmi_event_handler *hndl = NULL;
+
+	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(evt_key),
+			      KEY_XTRACT_EVT_ID(evt_key));
+	if (likely(r_evt)) {
+		mutex_lock(&r_evt->proto->registered_mtx);
+		hndl = KEY_FIND(r_evt->proto->registered_events_handlers,
+				hndl, evt_key);
+		if (likely(hndl))
+			refcount_inc(&hndl->users);
+		mutex_unlock(&r_evt->proto->registered_mtx);
+	}
+
+	return hndl;
+}
+
 /**
  * __scmi_enable_evt()  - Enable/disable events generation
  * @r_evt: The registered event to act upon
@@ -856,6 +1183,16 @@ static void scmi_put_handler(struct scmi_notify_instance *ni,
 	mutex_unlock(&ni->pending_mtx);
 }
 
+static void scmi_put_active_handler(struct scmi_notify_instance *ni,
+					  struct scmi_event_handler *hndl)
+{
+	struct scmi_registered_event *r_evt = hndl->r_evt;
+
+	mutex_lock(&r_evt->proto->registered_mtx);
+	scmi_put_handler_unlocked(ni, hndl);
+	mutex_unlock(&r_evt->proto->registered_mtx);
+}
+
 /**
  * scmi_event_handler_enable_events()  - Enable events associated to an handler
  * @hndl: The Event handler to act upon
@@ -1085,6 +1422,12 @@ int scmi_notification_init(struct scmi_handle *handle)
 	ni->gid = gid;
 	ni->handle = handle;
 
+	ni->notify_wq = alloc_workqueue("scmi_notify",
+					WQ_UNBOUND | WQ_FREEZABLE | WQ_SYSFS,
+					0);
+	if (!ni->notify_wq)
+		goto err;
+
 	ni->registered_protocols = devm_kcalloc(handle->dev, SCMI_MAX_PROTO,
 						sizeof(char *), GFP_KERNEL);
 	if (!ni->registered_protocols)
@@ -1126,5 +1469,12 @@ void scmi_notification_exit(struct scmi_handle *handle)
 		return;
 	ni = handle->notify_priv;
 
+	handle->notify_priv = NULL;
+	/* Ensure handle is up to date */
+	smp_wmb();
+
+	/* Destroy while letting pending work complete */
+	destroy_workqueue(ni->notify_wq);
+
 	devres_release_group(ni->handle->dev, ni->gid);
 }
diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
index f0561fb30970..a55c041180bf 100644
--- a/drivers/firmware/arm_scmi/notify.h
+++ b/drivers/firmware/arm_scmi/notify.h
@@ -47,6 +47,11 @@ struct scmi_event {
  *			using the proper custom protocol commands.
  *			Return true if at least one the required src_id
  *			has been successfully enabled/disabled
+ * @fill_custom_report: fills a custom event report from the provided
+ *			event message payld identifying the event
+ *			specific src_id.
+ *			Return NULL on failure otherwise @report now fully
+ *			populated
  *
  * Context: Helpers described in &struct scmi_protocol_event_ops are called
  *	    only in process context.
@@ -54,6 +59,9 @@ struct scmi_event {
 struct scmi_protocol_event_ops {
 	bool (*set_notify_enabled)(const struct scmi_handle *handle,
 				   u8 evt_id, u32 src_id, bool enabled);
+	void *(*fill_custom_report)(const struct scmi_handle *handle,
+				    u8 evt_id, u64 timestamp, const void *payld,
+				    size_t payld_sz, void *report, u32 *src_id);
 };
 
 int scmi_notification_init(struct scmi_handle *handle);
@@ -64,5 +72,7 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
 				  const struct scmi_protocol_event_ops *ops,
 				  const struct scmi_event *evt, int num_events,
 				  int num_sources);
+int scmi_notify(const struct scmi_handle *handle, u8 proto_id, u8 evt_id,
+		const void *buf, size_t len, u64 ts);
 
 #endif /* _SCMI_NOTIFY_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 4/9] firmware: arm_scmi: Enable notification core

[Cristian Marussi]

Initialize and enable SCMI Notifications core support during bus/driver
probe phase, so that protocols can start registering their supported
events during their initialization.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V3 --> V4
- simplified core initialization: protocols events' registrations is now
  disjoint from users' callback registrations, so that events' generation
  can be enabled earlier for registered events and delayed for pending
  ones in order to support deferred (or missing) protocol initialization
V2 --> V3
- reviewed core initialization: all implemented protocols must complete
  their protocol-events registration phases before notifications can be
  enabled as a whole; in the meantime any user's callback registration
  requests possibly issued while the notifications were not enabled
  remain pending: a dedicated worker completes the handlers registration
  once all protocols have been initialized.
  NOTE THAT this can lead to ISSUES with late inserted or missing SCMI
  modules (i.e. for protocols defined in the DT and implemented by the
  platform but lazily loaded or not loaded at all.), since in these
  scenarios notifications dispatching will be enabled later or never.
- reviewed core exit: protocol users (devices) are accounted on probe/
  remove, and protocols' events are unregisteredonce last user go
  (can happen only at shutdown)
V1 --> V2
- added timestamping
- moved notification init/exit and using devres
---
 drivers/firmware/arm_scmi/driver.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c
index fec308e47b9d..688268ee99cb 100644
--- a/drivers/firmware/arm_scmi/driver.c
+++ b/drivers/firmware/arm_scmi/driver.c
@@ -26,6 +26,7 @@
 #include <linux/slab.h>
 
 #include "common.h"
+#include "notify.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/scmi.h>
@@ -204,11 +205,13 @@ __scmi_xfer_put(struct scmi_xfers_info *minfo, struct scmi_xfer *xfer)
 
 static void scmi_handle_notification(struct scmi_chan_info *cinfo, u32 msg_hdr)
 {
+	u64 ts;
 	struct scmi_xfer *xfer;
 	struct device *dev = cinfo->dev;
 	struct scmi_info *info = handle_to_scmi_info(cinfo->handle);
 	struct scmi_xfers_info *minfo = &info->rx_minfo;
 
+	ts = ktime_get_boottime_ns();
 	xfer = scmi_xfer_get(cinfo->handle, minfo);
 	if (IS_ERR(xfer)) {
 		dev_err(dev, "failed to get free message slot (%ld)\n",
@@ -221,6 +224,8 @@ static void scmi_handle_notification(struct scmi_chan_info *cinfo, u32 msg_hdr)
 	scmi_dump_header_dbg(dev, &xfer->hdr);
 	info->desc->ops->fetch_notification(cinfo, info->desc->max_msg_size,
 					    xfer);
+	scmi_notify(cinfo->handle, xfer->hdr.protocol_id,
+		    xfer->hdr.id, xfer->rx.buf, xfer->rx.len, ts);
 
 	trace_scmi_rx_done(xfer->transfer_id, xfer->hdr.id,
 			   xfer->hdr.protocol_id, xfer->hdr.seq,
@@ -789,6 +794,9 @@ static int scmi_probe(struct platform_device *pdev)
 	if (ret)
 		return ret;
 
+	if (scmi_notification_init(handle))
+		dev_err(dev, "SCMI Notifications NOT available.\n");
+
 	ret = scmi_base_protocol_init(handle);
 	if (ret) {
 		dev_err(dev, "unable to communicate with SCMI(%d)\n", ret);
@@ -831,6 +839,8 @@ static int scmi_remove(struct platform_device *pdev)
 	struct scmi_info *info = platform_get_drvdata(pdev);
 	struct idr *idr = &info->tx_idr;
 
+	scmi_notification_exit(&info->handle);
+
 	mutex_lock(&scmi_list_mutex);
 	if (info->users)
 		ret = -EBUSY;
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 5/9] firmware: arm_scmi: Add Power notifications support

[Cristian Marussi]

Make SCMI Power protocol register with the notification core.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V6 --> V7
- fixed report.timestamp type
- removed POWER_STATE_CHANGE_REQUESTED motification handling (deprecated)
- fixed max_payld_sz initialization
- expose SCMI_EVENT_ in linux/scmi_protocol.h
V5 --> V6
- added handle argument to fill_custom_report()
V4 --> V5
- fixed unsual return construct
V3 --> V4
- scmi_event field renamed
V2 --> V3
- added handle awareness
V1 --> V2
- simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
  logic out of protocol. ALL_SRCIDs logic is now in charge of the
  notification core, together with proper reference counting of enables
- switched to devres protocol-registration
---
 drivers/firmware/arm_scmi/power.c | 101 ++++++++++++++++++++++++++++--
 include/linux/scmi_protocol.h     |  12 ++++
 2 files changed, 107 insertions(+), 6 deletions(-)

diff --git a/drivers/firmware/arm_scmi/power.c b/drivers/firmware/arm_scmi/power.c
index cf7f0312381b..101229e9fd18 100644
--- a/drivers/firmware/arm_scmi/power.c
+++ b/drivers/firmware/arm_scmi/power.c
@@ -5,19 +5,16 @@
  * Copyright (C) 2018 ARM Ltd.
  */
 
+#include <linux/scmi_protocol.h>
+
 #include "common.h"
+#include "notify.h"
 
 enum scmi_power_protocol_cmd {
 	POWER_DOMAIN_ATTRIBUTES = 0x3,
 	POWER_STATE_SET = 0x4,
 	POWER_STATE_GET = 0x5,
 	POWER_STATE_NOTIFY = 0x6,
-	POWER_STATE_CHANGE_REQUESTED_NOTIFY = 0x7,
-};
-
-enum scmi_power_protocol_notify {
-	POWER_STATE_CHANGED = 0x0,
-	POWER_STATE_CHANGE_REQUESTED = 0x1,
 };
 
 struct scmi_msg_resp_power_attributes {
@@ -48,6 +45,12 @@ struct scmi_power_state_notify {
 	__le32 notify_enable;
 };
 
+struct scmi_power_state_notify_payld {
+	__le32 agent_id;
+	__le32 domain_id;
+	__le32 power_state;
+};
+
 struct power_dom_info {
 	bool state_set_sync;
 	bool state_set_async;
@@ -186,6 +189,86 @@ static struct scmi_power_ops power_ops = {
 	.state_get = scmi_power_state_get,
 };
 
+static int scmi_power_request_notify(const struct scmi_handle *handle,
+				     u32 domain, bool enable)
+{
+	int ret;
+	struct scmi_xfer *t;
+	struct scmi_power_state_notify *notify;
+
+	ret = scmi_xfer_get_init(handle, POWER_STATE_NOTIFY,
+				 SCMI_PROTOCOL_POWER, sizeof(*notify), 0, &t);
+	if (ret)
+		return ret;
+
+	notify = t->tx.buf;
+	notify->domain = cpu_to_le32(domain);
+	notify->notify_enable = enable ? cpu_to_le32(BIT(0)) : 0;
+
+	ret = scmi_do_xfer(handle, t);
+
+	scmi_xfer_put(handle, t);
+	return ret;
+}
+
+static bool scmi_power_set_notify_enabled(const struct scmi_handle *handle,
+					  u8 evt_id, u32 src_id, bool enable)
+{
+	int ret;
+
+	ret = scmi_power_request_notify(handle, src_id, enable);
+	if (ret)
+		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLE - evt[%X] dom[%d] - ret:%d\n",
+				SCMI_PROTOCOL_POWER, evt_id, src_id, ret);
+
+	return !ret;
+}
+
+static void *scmi_power_fill_custom_report(const struct scmi_handle *handle,
+					   u8 evt_id, u64 timestamp,
+					   const void *payld, size_t payld_sz,
+					   void *report, u32 *src_id)
+{
+	void *rep = NULL;
+
+	switch (evt_id) {
+	case SCMI_EVENT_POWER_STATE_CHANGED:
+	{
+		const struct scmi_power_state_notify_payld *p = payld;
+		struct scmi_power_state_changed_report *r = report;
+
+		if (sizeof(*p) != payld_sz)
+			break;
+
+		r->timestamp = timestamp;
+		r->agent_id = le32_to_cpu(p->agent_id);
+		r->domain_id = le32_to_cpu(p->domain_id);
+		r->power_state = le32_to_cpu(p->power_state);
+		*src_id = r->domain_id;
+		rep = r;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return rep;
+}
+
+static const struct scmi_event power_events[] = {
+	{
+		.id = SCMI_EVENT_POWER_STATE_CHANGED,
+		.max_payld_sz = sizeof(struct scmi_power_state_notify_payld),
+		.max_report_sz =
+			sizeof(struct scmi_power_state_changed_report),
+	},
+};
+
+static const struct scmi_protocol_event_ops power_event_ops = {
+	.set_notify_enabled = scmi_power_set_notify_enabled,
+	.fill_custom_report = scmi_power_fill_custom_report,
+};
+
 static int scmi_power_protocol_init(struct scmi_handle *handle)
 {
 	int domain;
@@ -214,6 +297,12 @@ static int scmi_power_protocol_init(struct scmi_handle *handle)
 		scmi_power_domain_attributes_get(handle, domain, dom);
 	}
 
+	scmi_register_protocol_events(handle,
+				      SCMI_PROTOCOL_POWER, PAGE_SIZE,
+				      &power_event_ops, power_events,
+				      ARRAY_SIZE(power_events),
+				      pinfo->num_domains);
+
 	pinfo->version = version;
 	handle->power_ops = &power_ops;
 	handle->power_priv = pinfo;
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index 0fb97a589b30..b2fe3bb9871d 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -373,4 +373,16 @@ typedef int (*scmi_prot_init_fn_t)(struct scmi_handle *);
 int scmi_protocol_register(int protocol_id, scmi_prot_init_fn_t fn);
 void scmi_protocol_unregister(int protocol_id);
 
+/* SCMI Notification API - Custom Event Reports */
+enum scmi_notification_events {
+	SCMI_EVENT_POWER_STATE_CHANGED = 0x0,
+};
+
+struct scmi_power_state_changed_report {
+	u64 timestamp;
+	u32 agent_id;
+	u32 domain_id;
+	u32 power_state;
+};
+
 #endif /* _LINUX_SCMI_PROTOCOL_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 6/9] firmware: arm_scmi: Add Perf notifications support

[Cristian Marussi]

Make SCMI Perf protocol register with the notification core.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V6 --> V7
- fixed report.timestamp type
- fixed max_payld_sz initialization
- expose SCMI_EVENT_ in linux/scmi_protocol.h
V5 --> V6
- added handle argument to fill_custom_report()
V4 --> V5
- fixed unsual return construct
V3 --> V4
- scmi_event field renamed
V2 --> V3
- added handle awareness
V1 --> V2
- simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
  logic out of protocol. ALL_SRCIDs logic is now in charge of the
  notification core, together with proper reference counting of enables
- switched to devres protocol-registration
---
 drivers/firmware/arm_scmi/perf.c | 137 +++++++++++++++++++++++++++++--
 include/linux/scmi_protocol.h    |  17 ++++
 2 files changed, 149 insertions(+), 5 deletions(-)

diff --git a/drivers/firmware/arm_scmi/perf.c b/drivers/firmware/arm_scmi/perf.c
index eadc171e254b..72b0fcb5e40b 100644
--- a/drivers/firmware/arm_scmi/perf.c
+++ b/drivers/firmware/arm_scmi/perf.c
@@ -11,9 +11,11 @@
 #include <linux/io-64-nonatomic-hi-lo.h>
 #include <linux/platform_device.h>
 #include <linux/pm_opp.h>
+#include <linux/scmi_protocol.h>
 #include <linux/sort.h>
 
 #include "common.h"
+#include "notify.h"
 
 enum scmi_performance_protocol_cmd {
 	PERF_DOMAIN_ATTRIBUTES = 0x3,
@@ -27,11 +29,6 @@ enum scmi_performance_protocol_cmd {
 	PERF_DESCRIBE_FASTCHANNEL = 0xb,
 };
 
-enum scmi_performance_protocol_notify {
-	PERFORMANCE_LIMITS_CHANGED = 0x0,
-	PERFORMANCE_LEVEL_CHANGED = 0x1,
-};
-
 struct scmi_opp {
 	u32 perf;
 	u32 power;
@@ -86,6 +83,19 @@ struct scmi_perf_notify_level_or_limits {
 	__le32 notify_enable;
 };
 
+struct scmi_perf_limits_notify_payld {
+	__le32 agent_id;
+	__le32 domain_id;
+	__le32 range_max;
+	__le32 range_min;
+};
+
+struct scmi_perf_level_notify_payld {
+	__le32 agent_id;
+	__le32 domain_id;
+	__le32 performance_level;
+};
+
 struct scmi_msg_resp_perf_describe_levels {
 	__le16 num_returned;
 	__le16 num_remaining;
@@ -158,6 +168,11 @@ struct scmi_perf_info {
 	struct perf_dom_info *dom_info;
 };
 
+static enum scmi_performance_protocol_cmd evt_2_cmd[] = {
+	PERF_NOTIFY_LIMITS,
+	PERF_NOTIFY_LEVEL,
+};
+
 static int scmi_perf_attributes_get(const struct scmi_handle *handle,
 				    struct scmi_perf_info *pi)
 {
@@ -488,6 +503,29 @@ static int scmi_perf_level_get(const struct scmi_handle *handle, u32 domain,
 	return scmi_perf_mb_level_get(handle, domain, level, poll);
 }
 
+static int scmi_perf_level_limits_notify(const struct scmi_handle *handle,
+					 u32 domain, int message_id,
+					 bool enable)
+{
+	int ret;
+	struct scmi_xfer *t;
+	struct scmi_perf_notify_level_or_limits *notify;
+
+	ret = scmi_xfer_get_init(handle, message_id, SCMI_PROTOCOL_PERF,
+				 sizeof(*notify), 0, &t);
+	if (ret)
+		return ret;
+
+	notify = t->tx.buf;
+	notify->domain = cpu_to_le32(domain);
+	notify->notify_enable = enable ? cpu_to_le32(BIT(0)) : 0;
+
+	ret = scmi_do_xfer(handle, t);
+
+	scmi_xfer_put(handle, t);
+	return ret;
+}
+
 static bool scmi_perf_fc_size_is_valid(u32 msg, u32 size)
 {
 	if ((msg == PERF_LEVEL_GET || msg == PERF_LEVEL_SET) && size == 4)
@@ -710,6 +748,89 @@ static struct scmi_perf_ops perf_ops = {
 	.est_power_get = scmi_dvfs_est_power_get,
 };
 
+static bool scmi_perf_set_notify_enabled(const struct scmi_handle *handle,
+					 u8 evt_id, u32 src_id, bool enable)
+{
+	int ret, cmd_id;
+
+	cmd_id = MAP_EVT_TO_ENABLE_CMD(evt_id, evt_2_cmd);
+	if (cmd_id < 0)
+		return false;
+
+	ret = scmi_perf_level_limits_notify(handle, src_id, cmd_id, enable);
+	if (ret)
+		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLED - evt[%X] dom[%d] - ret:%d\n",
+				SCMI_PROTOCOL_PERF, evt_id, src_id, ret);
+
+	return !ret;
+}
+
+static void *scmi_perf_fill_custom_report(const struct scmi_handle *handle,
+					  u8 evt_id, u64 timestamp,
+					  const void *payld, size_t payld_sz,
+					  void *report, u32 *src_id)
+{
+	void *rep = NULL;
+
+	switch (evt_id) {
+	case SCMI_EVENT_PERFORMANCE_LIMITS_CHANGED:
+	{
+		const struct scmi_perf_limits_notify_payld *p = payld;
+		struct scmi_perf_limits_report *r = report;
+
+		if (sizeof(*p) != payld_sz)
+			break;
+
+		r->timestamp = timestamp;
+		r->agent_id = le32_to_cpu(p->agent_id);
+		r->domain_id = le32_to_cpu(p->domain_id);
+		r->range_max = le32_to_cpu(p->range_max);
+		r->range_min = le32_to_cpu(p->range_min);
+		*src_id = r->domain_id;
+		rep = r;
+		break;
+	}
+	case SCMI_EVENT_PERFORMANCE_LEVEL_CHANGED:
+	{
+		const struct scmi_perf_level_notify_payld *p = payld;
+		struct scmi_perf_level_report *r = report;
+
+		if (sizeof(*p) != payld_sz)
+			break;
+
+		r->timestamp = timestamp;
+		r->agent_id = le32_to_cpu(p->agent_id);
+		r->domain_id = le32_to_cpu(p->domain_id);
+		r->performance_level = le32_to_cpu(p->performance_level);
+		*src_id = r->domain_id;
+		rep = r;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return rep;
+}
+
+static const struct scmi_event perf_events[] = {
+	{
+		.id = SCMI_EVENT_PERFORMANCE_LIMITS_CHANGED,
+		.max_payld_sz = sizeof(struct scmi_perf_limits_notify_payld),
+		.max_report_sz = sizeof(struct scmi_perf_limits_report),
+	},
+	{
+		.id = SCMI_EVENT_PERFORMANCE_LEVEL_CHANGED,
+		.max_payld_sz = sizeof(struct scmi_perf_level_notify_payld),
+		.max_report_sz = sizeof(struct scmi_perf_level_report),
+	},
+};
+
+static const struct scmi_protocol_event_ops perf_event_ops = {
+	.set_notify_enabled = scmi_perf_set_notify_enabled,
+	.fill_custom_report = scmi_perf_fill_custom_report,
+};
+
 static int scmi_perf_protocol_init(struct scmi_handle *handle)
 {
 	int domain;
@@ -742,6 +863,12 @@ static int scmi_perf_protocol_init(struct scmi_handle *handle)
 			scmi_perf_domain_init_fc(handle, domain, &dom->fc_info);
 	}
 
+	scmi_register_protocol_events(handle,
+				      SCMI_PROTOCOL_PERF, PAGE_SIZE,
+				      &perf_event_ops, perf_events,
+				      ARRAY_SIZE(perf_events),
+				      pinfo->num_domains);
+
 	pinfo->version = version;
 	handle->perf_ops = &perf_ops;
 	handle->perf_priv = pinfo;
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index b2fe3bb9871d..832b03ef37c7 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -376,6 +376,8 @@ void scmi_protocol_unregister(int protocol_id);
 /* SCMI Notification API - Custom Event Reports */
 enum scmi_notification_events {
 	SCMI_EVENT_POWER_STATE_CHANGED = 0x0,
+	SCMI_EVENT_PERFORMANCE_LIMITS_CHANGED = 0x0,
+	SCMI_EVENT_PERFORMANCE_LEVEL_CHANGED = 0x1,
 };
 
 struct scmi_power_state_changed_report {
@@ -385,4 +387,19 @@ struct scmi_power_state_changed_report {
 	u32 power_state;
 };
 
+struct scmi_perf_limits_report {
+	u64 timestamp;
+	u32 agent_id;
+	u32 domain_id;
+	u32 range_max;
+	u32 range_min;
+};
+
+struct scmi_perf_level_report {
+	u64 timestamp;
+	u32 agent_id;
+	u32 domain_id;
+	u32 performance_level;
+};
+
 #endif /* _LINUX_SCMI_PROTOCOL_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 7/9] firmware: arm_scmi: Add Sensor notifications support

[Cristian Marussi]

Make SCMI Sensor protocol register with the notification core.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V6 --> V7
- fixed report.timestamp type
- removed trip_point_notify from .sensor_ops
- fixed max_payld_sz initialization
- expose SCMI_EVENT_ in linux/scmi_protocol.h
V5 --> V6
- added handle argument to fill_custom_report()
V4 --> V5
- fixed unsual return construct
V3 --> V4
- scmi_event field renamed
V2 --> V3
- added handle awareness
V1 --> V2
- simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
  logic out of protocol. ALL_SRCIDs logic is now in charge of the
  notification core, together with proper reference counting of enables
- switched to devres protocol-registration
---
 drivers/firmware/arm_scmi/sensors.c | 77 +++++++++++++++++++++++++++--
 include/linux/scmi_protocol.h       | 13 +++--
 2 files changed, 80 insertions(+), 10 deletions(-)

diff --git a/drivers/firmware/arm_scmi/sensors.c b/drivers/firmware/arm_scmi/sensors.c
index db1b1ab303da..88fdd887f5b5 100644
--- a/drivers/firmware/arm_scmi/sensors.c
+++ b/drivers/firmware/arm_scmi/sensors.c
@@ -5,7 +5,10 @@
  * Copyright (C) 2018 ARM Ltd.
  */
 
+#include <linux/scmi_protocol.h>
+
 #include "common.h"
+#include "notify.h"
 
 enum scmi_sensor_protocol_cmd {
 	SENSOR_DESCRIPTION_GET = 0x3,
@@ -14,10 +17,6 @@ enum scmi_sensor_protocol_cmd {
 	SENSOR_READING_GET = 0x6,
 };
 
-enum scmi_sensor_protocol_notify {
-	SENSOR_TRIP_POINT_EVENT = 0x0,
-};
-
 struct scmi_msg_resp_sensor_attributes {
 	__le16 num_sensors;
 	u8 max_requests;
@@ -71,6 +70,12 @@ struct scmi_msg_sensor_reading_get {
 #define SENSOR_READ_ASYNC	BIT(0)
 };
 
+struct scmi_sensor_trip_notify_payld {
+	__le32 agent_id;
+	__le32 sensor_id;
+	__le32 trip_point_desc;
+};
+
 struct sensors_info {
 	u32 version;
 	int num_sensors;
@@ -271,11 +276,67 @@ static int scmi_sensor_count_get(const struct scmi_handle *handle)
 static struct scmi_sensor_ops sensor_ops = {
 	.count_get = scmi_sensor_count_get,
 	.info_get = scmi_sensor_info_get,
-	.trip_point_notify = scmi_sensor_trip_point_notify,
 	.trip_point_config = scmi_sensor_trip_point_config,
 	.reading_get = scmi_sensor_reading_get,
 };
 
+static bool scmi_sensor_set_notify_enabled(const struct scmi_handle *handle,
+					   u8 evt_id, u32 src_id, bool enable)
+{
+	int ret;
+
+	ret = scmi_sensor_trip_point_notify(handle, src_id, enable);
+	if (ret)
+		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLED - evt[%X] dom[%d] - ret:%d\n",
+			SCMI_PROTOCOL_SENSOR, evt_id, src_id, ret);
+
+	return !ret;
+}
+
+static void *scmi_sensor_fill_custom_report(const struct scmi_handle *handle,
+					    u8 evt_id, u64 timestamp,
+					    const void *payld, size_t payld_sz,
+					    void *report, u32 *src_id)
+{
+	void *rep = NULL;
+
+	switch (evt_id) {
+	case SCMI_EVENT_SENSOR_TRIP_POINT_EVENT:
+	{
+		const struct scmi_sensor_trip_notify_payld *p = payld;
+		struct scmi_sensor_trip_point_report *r = report;
+
+		if (sizeof(*p) != payld_sz)
+			break;
+
+		r->timestamp = timestamp;
+		r->agent_id = le32_to_cpu(p->agent_id);
+		r->sensor_id = le32_to_cpu(p->sensor_id);
+		r->trip_point_desc = le32_to_cpu(p->trip_point_desc);
+		*src_id = r->sensor_id;
+		rep = r;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return rep;
+}
+
+static const struct scmi_event sensor_events[] = {
+	{
+		.id = SCMI_EVENT_SENSOR_TRIP_POINT_EVENT,
+		.max_payld_sz = sizeof(struct scmi_sensor_trip_notify_payld),
+		.max_report_sz = sizeof(struct scmi_sensor_trip_point_report),
+	},
+};
+
+static const struct scmi_protocol_event_ops sensor_event_ops = {
+	.set_notify_enabled = scmi_sensor_set_notify_enabled,
+	.fill_custom_report = scmi_sensor_fill_custom_report,
+};
+
 static int scmi_sensors_protocol_init(struct scmi_handle *handle)
 {
 	u32 version;
@@ -299,6 +360,12 @@ static int scmi_sensors_protocol_init(struct scmi_handle *handle)
 
 	scmi_sensor_description_get(handle, sinfo);
 
+	scmi_register_protocol_events(handle,
+				      SCMI_PROTOCOL_SENSOR, PAGE_SIZE,
+				      &sensor_event_ops, sensor_events,
+				      ARRAY_SIZE(sensor_events),
+				      sinfo->num_sensors);
+
 	sinfo->version = version;
 	handle->sensor_ops = &sensor_ops;
 	handle->sensor_priv = sinfo;
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index 832b03ef37c7..7d1e8d24c880 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -174,18 +174,13 @@ enum scmi_sensor_class {
  *
  * @count_get: get the count of sensors provided by SCMI
  * @info_get: get the information of the specified sensor
- * @trip_point_notify: control notifications on cross-over events for
- *	the trip-points
  * @trip_point_config: selects and configures a trip-point of interest
  * @reading_get: gets the current value of the sensor
  */
 struct scmi_sensor_ops {
 	int (*count_get)(const struct scmi_handle *handle);
-
 	const struct scmi_sensor_info *(*info_get)
 		(const struct scmi_handle *handle, u32 sensor_id);
-	int (*trip_point_notify)(const struct scmi_handle *handle,
-				 u32 sensor_id, bool enable);
 	int (*trip_point_config)(const struct scmi_handle *handle,
 				 u32 sensor_id, u8 trip_id, u64 trip_value);
 	int (*reading_get)(const struct scmi_handle *handle, u32 sensor_id,
@@ -378,6 +373,7 @@ enum scmi_notification_events {
 	SCMI_EVENT_POWER_STATE_CHANGED = 0x0,
 	SCMI_EVENT_PERFORMANCE_LIMITS_CHANGED = 0x0,
 	SCMI_EVENT_PERFORMANCE_LEVEL_CHANGED = 0x1,
+	SCMI_EVENT_SENSOR_TRIP_POINT_EVENT = 0x0,
 };
 
 struct scmi_power_state_changed_report {
@@ -402,4 +398,11 @@ struct scmi_perf_level_report {
 	u32 performance_level;
 };
 
+struct scmi_sensor_trip_point_report {
+	u64 timestamp;
+	u32 agent_id;
+	u32 sensor_id;
+	u32 trip_point_desc;
+};
+
 #endif /* _LINUX_SCMI_PROTOCOL_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 8/9] firmware: arm_scmi: Add Reset notifications support

[Cristian Marussi]

Make SCMI Reset protocol register with the notification core.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V6 --> V7
- fixed report.timestamp type
- added agent_id notification field
- fixed .max_payld_sz initialization
- expose SCMI_EVENT_ in linux/scmi_protocol.h
V5 --> V6
- added handle argument to fill_custom_report()
V4 --> V5
- fixed unsual return construct
V3 --> V4
- scmi_event field renamed
V2 --> V3
- added handle awareness
V1 --> V2
- simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
  logic out of protocol. ALL_SRCIDs logic is now in charge of the
  notification core, together with proper reference counting of enables
- switched to devres protocol-registration
---
 drivers/firmware/arm_scmi/reset.c | 105 ++++++++++++++++++++++++++++--
 include/linux/scmi_protocol.h     |   8 +++
 2 files changed, 109 insertions(+), 4 deletions(-)

diff --git a/drivers/firmware/arm_scmi/reset.c b/drivers/firmware/arm_scmi/reset.c
index de73054554f3..e80b30526b91 100644
--- a/drivers/firmware/arm_scmi/reset.c
+++ b/drivers/firmware/arm_scmi/reset.c
@@ -5,7 +5,10 @@
  * Copyright (C) 2019 ARM Ltd.
  */
 
+#include <linux/scmi_protocol.h>
+
 #include "common.h"
+#include "notify.h"
 
 enum scmi_reset_protocol_cmd {
 	RESET_DOMAIN_ATTRIBUTES = 0x3,
@@ -13,10 +16,6 @@ enum scmi_reset_protocol_cmd {
 	RESET_NOTIFY = 0x5,
 };
 
-enum scmi_reset_protocol_notify {
-	RESET_ISSUED = 0x0,
-};
-
 #define NUM_RESET_DOMAIN_MASK	0xffff
 #define RESET_NOTIFY_ENABLE	BIT(0)
 
@@ -40,6 +39,18 @@ struct scmi_msg_reset_domain_reset {
 #define ARCH_COLD_RESET		(ARCH_RESET_TYPE | COLD_RESET_STATE)
 };
 
+struct scmi_msg_reset_notify {
+	__le32 id;
+	__le32 event_control;
+#define RESET_TP_NOTIFY_ALL	BIT(0)
+};
+
+struct scmi_reset_issued_notify_payld {
+	__le32 agent_id;
+	__le32 domain_id;
+	__le32 reset_state;
+};
+
 struct reset_dom_info {
 	bool async_reset;
 	bool reset_notify;
@@ -190,6 +201,86 @@ static struct scmi_reset_ops reset_ops = {
 	.deassert = scmi_reset_domain_deassert,
 };
 
+static int scmi_reset_notify(const struct scmi_handle *handle, u32 domain_id,
+			     bool enable)
+{
+	int ret;
+	u32 evt_cntl = enable ? RESET_TP_NOTIFY_ALL : 0;
+	struct scmi_xfer *t;
+	struct scmi_msg_reset_notify *cfg;
+
+	ret = scmi_xfer_get_init(handle, RESET_NOTIFY,
+				 SCMI_PROTOCOL_RESET, sizeof(*cfg), 0, &t);
+	if (ret)
+		return ret;
+
+	cfg = t->tx.buf;
+	cfg->id = cpu_to_le32(domain_id);
+	cfg->event_control = cpu_to_le32(evt_cntl);
+
+	ret = scmi_do_xfer(handle, t);
+
+	scmi_xfer_put(handle, t);
+	return ret;
+}
+
+static bool scmi_reset_set_notify_enabled(const struct scmi_handle *handle,
+					  u8 evt_id, u32 src_id, bool enable)
+{
+	int ret;
+
+	ret = scmi_reset_notify(handle, src_id, enable);
+	if (ret)
+		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLED - evt[%X] dom[%d] - ret:%d\n",
+			SCMI_PROTOCOL_RESET, evt_id, src_id, ret);
+
+	return !ret;
+}
+
+static void *scmi_reset_fill_custom_report(const struct scmi_handle *handle,
+					   u8 evt_id, u64 timestamp,
+					   const void *payld, size_t payld_sz,
+					   void *report, u32 *src_id)
+{
+	void *rep = NULL;
+
+	switch (evt_id) {
+	case SCMI_EVENT_RESET_ISSUED:
+	{
+		const struct scmi_reset_issued_notify_payld *p = payld;
+		struct scmi_reset_issued_report *r = report;
+
+		if (sizeof(*p) != payld_sz)
+			break;
+
+		r->timestamp = timestamp;
+		r->agent_id = le32_to_cpu(p->agent_id);
+		r->domain_id = le32_to_cpu(p->domain_id);
+		r->reset_state = le32_to_cpu(p->reset_state);
+		*src_id = r->domain_id;
+		rep = r;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return rep;
+}
+
+static const struct scmi_event reset_events[] = {
+	{
+		.id = SCMI_EVENT_RESET_ISSUED,
+		.max_payld_sz = sizeof(struct scmi_reset_issued_notify_payld),
+		.max_report_sz = sizeof(struct scmi_reset_issued_report),
+	},
+};
+
+static const struct scmi_protocol_event_ops reset_event_ops = {
+	.set_notify_enabled = scmi_reset_set_notify_enabled,
+	.fill_custom_report = scmi_reset_fill_custom_report,
+};
+
 static int scmi_reset_protocol_init(struct scmi_handle *handle)
 {
 	int domain;
@@ -218,6 +309,12 @@ static int scmi_reset_protocol_init(struct scmi_handle *handle)
 		scmi_reset_domain_attributes_get(handle, domain, dom);
 	}
 
+	scmi_register_protocol_events(handle,
+				      SCMI_PROTOCOL_RESET, PAGE_SIZE,
+				      &reset_event_ops, reset_events,
+				      ARRAY_SIZE(reset_events),
+				      pinfo->num_domains);
+
 	pinfo->version = version;
 	handle->reset_ops = &reset_ops;
 	handle->reset_priv = pinfo;
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index 7d1e8d24c880..091263aa5733 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -374,6 +374,7 @@ enum scmi_notification_events {
 	SCMI_EVENT_PERFORMANCE_LIMITS_CHANGED = 0x0,
 	SCMI_EVENT_PERFORMANCE_LEVEL_CHANGED = 0x1,
 	SCMI_EVENT_SENSOR_TRIP_POINT_EVENT = 0x0,
+	SCMI_EVENT_RESET_ISSUED = 0x0,
 };
 
 struct scmi_power_state_changed_report {
@@ -405,4 +406,11 @@ struct scmi_sensor_trip_point_report {
 	u32 trip_point_desc;
 };
 
+struct scmi_reset_issued_report {
+	u64 timestamp;
+	u32 agent_id;
+	u32 domain_id;
+	u32 reset_state;
+};
+
 #endif /* _LINUX_SCMI_PROTOCOL_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 9/9] firmware: arm_scmi: Add Base notifications support

[Cristian Marussi]

Make SCMI Base protocol register with the notification core.

Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
---
V6 --> V7
- fixed report.timestamp type
- fix max_payld_sz initialization
- fix report layout and initialization
- expose SCMI_EVENT_ in linux/scmi_protocol.h
V5 --> V6
- added handle argument to fill_custom_report()
V4 --> V5
- fixed unsual return construct
V3 --> V4
- scmi_event field renamed
V2 --> V3
- added handle awareness
V1 --> V2
- simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
  logic out of protocol. ALL_SRCIDs logic is now in charge of the
  notification core, together with proper reference counting of enables
- switched to devres protocol-registration
---
 drivers/firmware/arm_scmi/base.c | 118 +++++++++++++++++++++++++++++--
 include/linux/scmi_protocol.h    |   9 +++
 2 files changed, 123 insertions(+), 4 deletions(-)

diff --git a/drivers/firmware/arm_scmi/base.c b/drivers/firmware/arm_scmi/base.c
index ce7d9203e41b..dcb098d8d823 100644
--- a/drivers/firmware/arm_scmi/base.c
+++ b/drivers/firmware/arm_scmi/base.c
@@ -5,7 +5,13 @@
  * Copyright (C) 2018 ARM Ltd.
  */
 
+#include <linux/scmi_protocol.h>
+
 #include "common.h"
+#include "notify.h"
+
+#define SCMI_BASE_NUM_SOURCES		1
+#define SCMI_BASE_MAX_CMD_ERR_COUNT	1024
 
 enum scmi_base_protocol_cmd {
 	BASE_DISCOVER_VENDOR = 0x3,
@@ -19,16 +25,25 @@ enum scmi_base_protocol_cmd {
 	BASE_RESET_AGENT_CONFIGURATION = 0xb,
 };
 
-enum scmi_base_protocol_notify {
-	BASE_ERROR_EVENT = 0x0,
-};
-
 struct scmi_msg_resp_base_attributes {
 	u8 num_protocols;
 	u8 num_agents;
 	__le16 reserved;
 };
 
+struct scmi_msg_base_error_notify {
+	__le32 event_control;
+#define BASE_TP_NOTIFY_ALL	BIT(0)
+};
+
+struct scmi_base_error_notify_payld {
+	__le32 agent_id;
+	__le32 error_status;
+#define IS_FATAL_ERROR(x)	((x) & BIT(31))
+#define ERROR_CMD_COUNT(x)	FIELD_GET(GENMASK(9, 0), (x))
+	__le64 msg_reports[SCMI_BASE_MAX_CMD_ERR_COUNT];
+};
+
 /**
  * scmi_base_attributes_get() - gets the implementation details
  *	that are associated with the base protocol.
@@ -222,6 +237,95 @@ static int scmi_base_discover_agent_get(const struct scmi_handle *handle,
 	return ret;
 }
 
+static int scmi_base_error_notify(const struct scmi_handle *handle, bool enable)
+{
+	int ret;
+	u32 evt_cntl = enable ? BASE_TP_NOTIFY_ALL : 0;
+	struct scmi_xfer *t;
+	struct scmi_msg_base_error_notify *cfg;
+
+	ret = scmi_xfer_get_init(handle, BASE_NOTIFY_ERRORS,
+				 SCMI_PROTOCOL_BASE, sizeof(*cfg), 0, &t);
+	if (ret)
+		return ret;
+
+	cfg = t->tx.buf;
+	cfg->event_control = cpu_to_le32(evt_cntl);
+
+	ret = scmi_do_xfer(handle, t);
+
+	scmi_xfer_put(handle, t);
+	return ret;
+}
+
+static bool scmi_base_set_notify_enabled(const struct scmi_handle *handle,
+					 u8 evt_id, u32 src_id, bool enable)
+{
+	int ret;
+
+	ret = scmi_base_error_notify(handle, enable);
+	if (ret)
+		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLED - evt[%X] ret:%d\n",
+			SCMI_PROTOCOL_BASE, evt_id, ret);
+
+	return !ret;
+}
+
+static void *scmi_base_fill_custom_report(const struct scmi_handle *handle,
+					  u8 evt_id, u64 timestamp,
+					  const void *payld, size_t payld_sz,
+					  void *report, u32 *src_id)
+{
+	void *rep = NULL;
+
+	switch (evt_id) {
+	case SCMI_EVENT_BASE_ERROR_EVENT:
+	{
+		int i;
+		const struct scmi_base_error_notify_payld *p = payld;
+		struct scmi_base_error_report *r = report;
+
+		/*
+		 * BaseError notification payload is variable in size but
+		 * up to a maximum length determined by the struct ponted by p.
+		 * Instead payld_sz is the effective length of this notification
+		 * payload so cannot be greater of the maximum allowed size as
+		 * pointed by p.
+		 */
+		if (sizeof(*p) < payld_sz)
+			break;
+
+		r->timestamp = timestamp;
+		r->agent_id = le32_to_cpu(p->agent_id);
+		r->fatal = IS_FATAL_ERROR(le32_to_cpu(p->error_status));
+		r->cmd_count = ERROR_CMD_COUNT(le32_to_cpu(p->error_status));
+		for (i = 0; i < r->cmd_count; i++)
+			r->reports[i] = le64_to_cpu(p->msg_reports[i]);
+		*src_id = 0;
+		rep = r;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return rep;
+}
+
+static const struct scmi_event base_events[] = {
+	{
+		.id = SCMI_EVENT_BASE_ERROR_EVENT,
+		.max_payld_sz = sizeof(struct scmi_base_error_notify_payld),
+		.max_report_sz = sizeof(struct scmi_base_error_report) +
+				  SCMI_BASE_MAX_CMD_ERR_COUNT * sizeof(u64),
+	},
+};
+
+static const struct scmi_protocol_event_ops base_event_ops = {
+	.set_notify_enabled = scmi_base_set_notify_enabled,
+	.fill_custom_report = scmi_base_fill_custom_report,
+};
+
 int scmi_base_protocol_init(struct scmi_handle *h)
 {
 	int id, ret;
@@ -256,6 +360,12 @@ int scmi_base_protocol_init(struct scmi_handle *h)
 	dev_dbg(dev, "Found %d protocol(s) %d agent(s)\n", rev->num_protocols,
 		rev->num_agents);
 
+	scmi_register_protocol_events(handle,
+				      SCMI_PROTOCOL_BASE, (4 * PAGE_SIZE),
+				      &base_event_ops, base_events,
+				      ARRAY_SIZE(base_events),
+				      SCMI_BASE_NUM_SOURCES);
+
 	for (id = 0; id < rev->num_agents; id++) {
 		scmi_base_discover_agent_get(handle, id, name);
 		dev_dbg(dev, "Agent %d: %s\n", id, name);
diff --git a/include/linux/scmi_protocol.h b/include/linux/scmi_protocol.h
index 091263aa5733..bb858d329dae 100644
--- a/include/linux/scmi_protocol.h
+++ b/include/linux/scmi_protocol.h
@@ -375,6 +375,7 @@ enum scmi_notification_events {
 	SCMI_EVENT_PERFORMANCE_LEVEL_CHANGED = 0x1,
 	SCMI_EVENT_SENSOR_TRIP_POINT_EVENT = 0x0,
 	SCMI_EVENT_RESET_ISSUED = 0x0,
+	SCMI_EVENT_BASE_ERROR_EVENT = 0x0,
 };
 
 struct scmi_power_state_changed_report {
@@ -413,4 +414,12 @@ struct scmi_reset_issued_report {
 	u32 reset_state;
 };
 
+struct scmi_base_error_report {
+	u64 timestamp;
+	u32 agent_id;
+	bool fatal;
+	u16 cmd_count;
+	u64 reports[0];
+};
+
 #endif /* _LINUX_SCMI_PROTOCOL_H */
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 9/9] firmware: arm_scmi: Add Base notifications support

[Sudeep Holla]

On Wed, May 20, 2020 at 09:11:18AM +0100, Cristian Marussi wrote:
> Make SCMI Base protocol register with the notification core.
>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> ---
> V6 --> V7
> - fixed report.timestamp type
> - fix max_payld_sz initialization
> - fix report layout and initialization
> - expose SCMI_EVENT_ in linux/scmi_protocol.h
> V5 --> V6
> - added handle argument to fill_custom_report()
> V4 --> V5
> - fixed unsual return construct
> V3 --> V4
> - scmi_event field renamed
> V2 --> V3
> - added handle awareness
> V1 --> V2
> - simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
>   logic out of protocol. ALL_SRCIDs logic is now in charge of the
>   notification core, together with proper reference counting of enables
> - switched to devres protocol-registration
> ---
>  drivers/firmware/arm_scmi/base.c | 118 +++++++++++++++++++++++++++++--
>  include/linux/scmi_protocol.h    |   9 +++
>  2 files changed, 123 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/firmware/arm_scmi/base.c b/drivers/firmware/arm_scmi/base.c
> index ce7d9203e41b..dcb098d8d823 100644
> --- a/drivers/firmware/arm_scmi/base.c
> +++ b/drivers/firmware/arm_scmi/base.c
> @@ -5,7 +5,13 @@
>   * Copyright (C) 2018 ARM Ltd.
>   */
>
> +#include <linux/scmi_protocol.h>
> +
>  #include "common.h"
> +#include "notify.h"
> +
> +#define SCMI_BASE_NUM_SOURCES		1
> +#define SCMI_BASE_MAX_CMD_ERR_COUNT	1024
>

I am not sure of this, see below.

>  enum scmi_base_protocol_cmd {
>  	BASE_DISCOVER_VENDOR = 0x3,
> @@ -19,16 +25,25 @@ enum scmi_base_protocol_cmd {
>  	BASE_RESET_AGENT_CONFIGURATION = 0xb,
>  };
>
> -enum scmi_base_protocol_notify {
> -	BASE_ERROR_EVENT = 0x0,
> -};
> -
>  struct scmi_msg_resp_base_attributes {
>  	u8 num_protocols;
>  	u8 num_agents;
>  	__le16 reserved;
>  };
>
> +struct scmi_msg_base_error_notify {
> +	__le32 event_control;
> +#define BASE_TP_NOTIFY_ALL	BIT(0)
> +};
> +
> +struct scmi_base_error_notify_payld {
> +	__le32 agent_id;
> +	__le32 error_status;
> +#define IS_FATAL_ERROR(x)	((x) & BIT(31))
> +#define ERROR_CMD_COUNT(x)	FIELD_GET(GENMASK(9, 0), (x))
> +	__le64 msg_reports[SCMI_BASE_MAX_CMD_ERR_COUNT];

This entire payload needs to fit in shmem and having huge shmem just
for this sounds not so good to me. If this can be large, it needs to
be iterated multiple times to get the full log.

> +};
> +
>  /**
>   * scmi_base_attributes_get() - gets the implementation details
>   *	that are associated with the base protocol.
> @@ -222,6 +237,95 @@ static int scmi_base_discover_agent_get(const struct scmi_handle *handle,
>  	return ret;
>  }
>
> +static int scmi_base_error_notify(const struct scmi_handle *handle, bool enable)
> +{
> +	int ret;
> +	u32 evt_cntl = enable ? BASE_TP_NOTIFY_ALL : 0;
> +	struct scmi_xfer *t;
> +	struct scmi_msg_base_error_notify *cfg;
> +
> +	ret = scmi_xfer_get_init(handle, BASE_NOTIFY_ERRORS,
> +				 SCMI_PROTOCOL_BASE, sizeof(*cfg), 0, &t);
> +	if (ret)
> +		return ret;
> +
> +	cfg = t->tx.buf;
> +	cfg->event_control = cpu_to_le32(evt_cntl);
> +
> +	ret = scmi_do_xfer(handle, t);
> +
> +	scmi_xfer_put(handle, t);
> +	return ret;
> +}
> +
> +static bool scmi_base_set_notify_enabled(const struct scmi_handle *handle,
> +					 u8 evt_id, u32 src_id, bool enable)
> +{
> +	int ret;
> +
> +	ret = scmi_base_error_notify(handle, enable);
> +	if (ret)
> +		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLED - evt[%X] ret:%d\n",
> +			SCMI_PROTOCOL_BASE, evt_id, ret);
> +

I would make all these debug as they are not fatal. The users can decide
if they are fatal and log it appropriately.

> +	return !ret;
> +}
> +
> +static void *scmi_base_fill_custom_report(const struct scmi_handle *handle,
> +					  u8 evt_id, u64 timestamp,
> +					  const void *payld, size_t payld_sz,
> +					  void *report, u32 *src_id)
> +{
> +	void *rep = NULL;
> +
> +	switch (evt_id) {
> +	case SCMI_EVENT_BASE_ERROR_EVENT:

Drop switch for now, just check for evt_id to be SCMI_EVENT_BASE_ERROR_EVENT.

> +	{
> +		int i;
> +		const struct scmi_base_error_notify_payld *p = payld;
> +		struct scmi_base_error_report *r = report;
> +
> +		/*
> +		 * BaseError notification payload is variable in size but
> +		 * up to a maximum length determined by the struct ponted by p.
> +		 * Instead payld_sz is the effective length of this notification
> +		 * payload so cannot be greater of the maximum allowed size as
> +		 * pointed by p.
> +		 */
> +		if (sizeof(*p) < payld_sz)
> +			break;
> +
> +		r->timestamp = timestamp;
> +		r->agent_id = le32_to_cpu(p->agent_id);
> +		r->fatal = IS_FATAL_ERROR(le32_to_cpu(p->error_status));
> +		r->cmd_count = ERROR_CMD_COUNT(le32_to_cpu(p->error_status));
> +		for (i = 0; i < r->cmd_count; i++)
> +			r->reports[i] = le64_to_cpu(p->msg_reports[i]);
> +		*src_id = 0;
> +		rep = r;
> +		break;
> +	}
> +	default:
> +		break;
> +	}
> +
> +	return rep;
> +}
> +
> +static const struct scmi_event base_events[] = {
> +	{
> +		.id = SCMI_EVENT_BASE_ERROR_EVENT,
> +		.max_payld_sz = sizeof(struct scmi_base_error_notify_payld),
> +		.max_report_sz = sizeof(struct scmi_base_error_report) +
> +				  SCMI_BASE_MAX_CMD_ERR_COUNT * sizeof(u64),
> +	},
> +};
> +
> +static const struct scmi_protocol_event_ops base_event_ops = {
> +	.set_notify_enabled = scmi_base_set_notify_enabled,
> +	.fill_custom_report = scmi_base_fill_custom_report,
> +};
> +
>  int scmi_base_protocol_init(struct scmi_handle *h)
>  {
>  	int id, ret;
> @@ -256,6 +360,12 @@ int scmi_base_protocol_init(struct scmi_handle *h)
>  	dev_dbg(dev, "Found %d protocol(s) %d agent(s)\n", rev->num_protocols,
>  		rev->num_agents);
>
> +	scmi_register_protocol_events(handle,
> +				      SCMI_PROTOCOL_BASE, (4 * PAGE_SIZE),

The size 4 * PAGE_SZ is not clear. For me this can't be more that
max_msg_size.

The comments in this patch applies to last 5 patches(all protocols basically)

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/9] firmware: arm_scmi: Add notification protocol-registration

[Sudeep Holla]

On Wed, May 20, 2020 at 09:11:10AM +0100, Cristian Marussi wrote:
> Add core SCMI Notifications protocol-registration support: allow protocols
> to register their own set of supported events, during their initialization
> phase. Notification core can track multiple platform instances by their
> handles.
>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> ---
>  drivers/firmware/arm_scmi/Makefile |   2 +-
>  drivers/firmware/arm_scmi/common.h |   4 +
>  drivers/firmware/arm_scmi/notify.c | 435 +++++++++++++++++++++++++++++
>  drivers/firmware/arm_scmi/notify.h |  56 ++++
>  include/linux/scmi_protocol.h      |   3 +
>  5 files changed, 499 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/firmware/arm_scmi/notify.c
>  create mode 100644 drivers/firmware/arm_scmi/notify.h
>
> diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
> new file mode 100644
> index 000000000000..8b620fc7df50
> --- /dev/null
> +++ b/drivers/firmware/arm_scmi/notify.c
> @@ -0,0 +1,435 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * System Control and Management Interface (SCMI) Notification support
> + *
> + * Copyright (C) 2020 ARM Ltd.
> + */
> +/**
> + * DOC: Theory of operation
> + *
> + * SCMI Protocol specification allows the platform to signal events to
> + * interested agents via notification messages: this is an implementation
> + * of the dispatch and delivery of such notifications to the interested users
> + * inside the Linux kernel.
> + *
> + * An SCMI Notification core instance is initialized for each active platform
> + * instance identified by the means of the usual &struct scmi_handle.
> + *
> + * Each SCMI Protocol implementation, during its initialization, registers with
> + * this core its set of supported events using scmi_register_protocol_events():
> + * all the needed descriptors are stored in the &struct registered_protocols and
> + * &struct registered_events arrays.
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/bug.h>
> +#include <linux/compiler.h>
> +#include <linux/device.h>
> +#include <linux/err.h>
> +#include <linux/kernel.h>
> +#include <linux/kfifo.h>
> +#include <linux/mutex.h>
> +#include <linux/refcount.h>
> +#include <linux/scmi_protocol.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +
> +#include "notify.h"
> +
> +#define	SCMI_MAX_PROTO			256

[nit] Space after define instead of tab.

> +#define	SCMI_ALL_SRC_IDS		0xffffUL

Once you move {PROTO,EVT,SRC}_ID_MASK here, you can just {re-}use SRC_ID_MASK

> +/*
> + * Builds an unsigned 32bit key from the given input tuple to be used
> + * as a key in hashtables.
> + */
> +#define MAKE_HASH_KEY(p, e, s)			\
> +	((u32)(((p) << 24) | ((e) << 16) | ((s) & SCMI_ALL_SRC_IDS)))
> +

You can use {PROTO,EVT,SRC}_ID_MASK here and FIELD_PREP in the above
macro.

> +#define MAKE_ALL_SRCS_KEY(p, e)			\
> +	MAKE_HASH_KEY((p), (e), SCMI_ALL_SRC_IDS)
> +
> +struct scmi_registered_protocol_events_desc;
> +

Just a thought here: can we rename all protocol_event* as just event*.
The term protocol is kind of implicit and makes the names too long.
I feel some are really too long :). Similar if you think protocol
is implicit elsewhere, drop it.

> +/**
> + * struct scmi_notify_instance  - Represents an instance of the notification
> + * core
> + * @gid: GroupID used for devres
> + * @handle: A reference to the platform instance
> + * @registered_protocols: A statically allocated array containing pointers to
> + *			  all the registered protocol-level specific information
> + *			  related to events' handling
> + *
> + * Each platform instance, represented by a handle, has its own instance of
> + * the notification subsystem represented by this structure.
> + */
> +struct scmi_notify_instance {
> +	void						*gid;
> +	struct scmi_handle				*handle;
> +	struct scmi_registered_protocol_events_desc	**registered_protocols;
> +};
> +
> +/**
> + * struct events_queue  - Describes a queue and its associated worker
> + * @sz: Size in bytes of the related kfifo
> + * @kfifo: A dedicated Kernel kfifo descriptor
> + *
> + * Each protocol has its own dedicated events_queue descriptor.
> + */
> +struct events_queue {
> +	size_t				sz;
> +	struct kfifo			kfifo;
> +};
> +

[nit] Add tabs only for alignment, just to keep it uniform throughout.

> +/**
> + * struct scmi_event_header  - A utility header
> + * @timestamp: The timestamp, in nanoseconds (boottime), which was associated
> + *	       to this event as soon as it entered the SCMI RX ISR
> + * @evt_id: Event ID (corresponds to the Event MsgID for this Protocol)
> + * @payld_sz: Effective size of the embedded message payload which follows
> + * @payld: A reference to the embedded event payload
> + *
> + * This header is prepended to each received event message payload before
> + * queueing it on the related &struct events_queue.
> + */
> +struct scmi_event_header {
> +	u64	timestamp;
> +	u8	evt_id;
> +	size_t	payld_sz;
> +	u8	payld[];
> +} __packed;
> +

Is this directly read from the shmem ? Trying to understand need for
__packed.

> +struct scmi_registered_event;
> +
> +/**
> + * struct scmi_registered_protocol_events_desc  - Protocol Specific information
> + * @id: Protocol ID
> + * @ops: Protocol specific and event-related operations
> + * @equeue: The embedded per-protocol events_queue
> + * @ni: A reference to the initialized instance descriptor
> + * @eh: A reference to pre-allocated buffer to be used as a scratch area by the
> + *	deferred worker when fetching data from the kfifo
> + * @eh_sz: Size of the pre-allocated buffer @eh
> + * @in_flight: A reference to an in flight &struct scmi_registered_event
> + * @num_events: Number of events in @registered_events
> + * @registered_events: A dynamically allocated array holding all the registered
> + *		       events' descriptors, whose fixed-size is determined at
> + *		       compile time.
> + *
> + * All protocols that register at least one event have their protocol-specific
> + * information stored here, together with the embedded allocated events_queue.
> + * These descriptors are stored in the @registered_protocols array at protocol
> + * registration time.
> + *
> + * Once these descriptors are successfully registered, they are NEVER again
> + * removed or modified since protocols do not unregister ever, so that, once
> + * we safely grab a NON-NULL reference from the array we can keep it and use it.
> + */
> +struct scmi_registered_protocol_events_desc {
> +	u8					id;
> +	const struct scmi_protocol_event_ops	*ops;
> +	struct events_queue			equeue;
> +	struct scmi_notify_instance		*ni;
> +	struct scmi_event_header		*eh;
> +	size_t					eh_sz;
> +	void					*in_flight;
> +	int					num_events;
> +	struct scmi_registered_event		**registered_events;
> +};
> +
> +/**
> + * struct scmi_registered_event  - Event Specific Information
> + * @proto: A reference to the associated protocol descriptor
> + * @evt: A reference to the associated event descriptor (as provided at
> + *       registration time)
> + * @report: A pre-allocated buffer used by the deferred worker to fill a
> + *	    customized event report
> + * @num_sources: The number of possible sources for this event as stated at
> + *		 events' registration time
> + * @sources: A reference to a dynamically allocated array used to refcount the
> + *	     events' enable requests for all the existing sources
> + * @sources_mtx: A mutex to serialize the access to @sources
> + *
> + * All registered events are represented by one of these structures that are
> + * stored in the @registered_events array at protocol registration time.
> + *
> + * Once these descriptors are successfully registered, they are NEVER again
> + * removed or modified since protocols do not unregister ever, so that once we
> + * safely grab a NON-NULL reference from the table we can keep it and use it.
> + */
> +struct scmi_registered_event {
> +	struct scmi_registered_protocol_events_desc	*proto;
> +	const struct scmi_event				*evt;
> +	void						*report;
> +	u32						num_sources;
> +	refcount_t					*sources;
> +	struct mutex					sources_mtx;
> +};
> +
> +/**
> + * scmi_kfifo_free()  - Devres action helper to free the kfifo
> + * @kfifo: The kfifo to free
> + */
> +static void scmi_kfifo_free(void *kfifo)
> +{
> +	kfifo_free((struct kfifo *)kfifo);
> +}
> +
> +/**
> + * scmi_initialize_events_queue()  - Allocate/Initialize a kfifo buffer
> + * @ni: A reference to the notification instance to use
> + * @equeue: The events_queue to initialize
> + * @sz: Size of the kfifo buffer to allocate
> + *
> + * Allocate a buffer for the kfifo and initialize it.
> + *
> + * Return: 0 on Success
> + */
> +static int scmi_initialize_events_queue(struct scmi_notify_instance *ni,
> +					struct events_queue *equeue, size_t sz)
> +{
> +	if (kfifo_alloc(&equeue->kfifo, sz, GFP_KERNEL))
> +		return -ENOMEM;
> +	/* Size could have been roundup to power-of-two */
> +	equeue->sz = kfifo_size(&equeue->kfifo);
> +
> +	return devm_add_action_or_reset(ni->handle->dev, scmi_kfifo_free,
> +					&equeue->kfifo);
> +}
> +
> +/**
> + * scmi_allocate_registered_protocol_desc()  - Allocate a registered protocol
> + * events' descriptor
> + * @ni: A reference to the &struct scmi_notify_instance notification instance
> + *	to use
> + * @proto_id: Protocol ID
> + * @queue_sz: Size of the associated queue to allocate
> + * @eh_sz: Size of the event header scratch area to pre-allocate
> + * @num_events: Number of events to support (size of @registered_events)
> + * @ops: Pointer to a struct holding references to protocol specific helpers
> + *	 needed during events handling
> + *
> + * It is supposed to be called only once for each protocol at protocol
> + * initialization time, so it warns if the requested protocol is found already
> + * registered.
> + *
> + * Return: The allocated and registered descriptor on Success
> + */
> +static struct scmi_registered_protocol_events_desc *
> +scmi_allocate_registered_protocol_desc(struct scmi_notify_instance *ni,
> +				       u8 proto_id, size_t queue_sz,
> +				       size_t eh_sz, int num_events,
> +				const struct scmi_protocol_event_ops *ops)
> +{
> +	int ret;
> +	struct scmi_registered_protocol_events_desc *pd;
> +
> +	/* Ensure protocols are up to date */
> +	smp_rmb();
> +	if (ni->registered_protocols[proto_id]) {
> +		WARN_ON(1);

Can't this be if (WARN_ON(ni->registered_protocols[proto_id])) ?

> +		return ERR_PTR(-EINVAL);
> +	}
> +
> +	pd = devm_kzalloc(ni->handle->dev, sizeof(*pd), GFP_KERNEL);
> +	if (!pd)
> +		return ERR_PTR(-ENOMEM);
> +	pd->id = proto_id;
> +	pd->ops = ops;
> +	pd->ni = ni;
> +
> +	ret = scmi_initialize_events_queue(ni, &pd->equeue, queue_sz);
> +	if (ret)
> +		return ERR_PTR(ret);
> +
> +	pd->eh = devm_kzalloc(ni->handle->dev, eh_sz, GFP_KERNEL);
> +	if (!pd->eh)
> +		return ERR_PTR(-ENOMEM);
> +	pd->eh_sz = eh_sz;
> +
> +	pd->registered_events = devm_kcalloc(ni->handle->dev, num_events,
> +					     sizeof(char *), GFP_KERNEL);
> +	if (!pd->registered_events)
> +		return ERR_PTR(-ENOMEM);
> +	pd->num_events = num_events;
> +
> +	return pd;
> +}
> +
> +/**
> + * scmi_register_protocol_events()  - Register Protocol Events with the core
> + * @handle: The handle identifying the platform instance against which the
> + *	    the protocol's events are registered
> + * @proto_id: Protocol ID
> + * @queue_sz: Size in bytes of the associated queue to be allocated
> + * @ops: Protocol specific event-related operations
> + * @evt: Event descriptor array
> + * @num_events: Number of events in @evt array
> + * @num_sources: Number of possible sources for this protocol on this
> + *		 platform.
> + *
> + * Used by SCMI Protocols initialization code to register with the notification
> + * core the list of supported events and their descriptors: takes care to
> + * pre-allocate and store all needed descriptors, scratch buffers and event
> + * queues.
> + *
> + * Return: 0 on Success
> + */
> +int scmi_register_protocol_events(const struct scmi_handle *handle,
> +				  u8 proto_id, size_t queue_sz,
> +				  const struct scmi_protocol_event_ops *ops,
> +				  const struct scmi_event *evt, int num_events,
> +				  int num_sources)
> +{
> +	int i;
> +	size_t payld_sz = 0;
> +	struct scmi_registered_protocol_events_desc *pd;
> +	struct scmi_notify_instance *ni;
> +
> +	if (!ops || !evt || proto_id >= SCMI_MAX_PROTO)
> +		return -EINVAL;
> +
> +	/* Ensure notify_priv is updated */
> +	smp_rmb();
> +	if (unlikely(!handle->notify_priv))
> +		return -ENOMEM;
> +	ni = handle->notify_priv;
> +
> +	/* Attach to the notification main devres group */
> +	if (!devres_open_group(ni->handle->dev, ni->gid, GFP_KERNEL))
> +		return -ENOMEM;
> +
> +	for (i = 0; i < num_events; i++)
> +		payld_sz = max_t(size_t, payld_sz, evt[i].max_payld_sz);
> +	pd = scmi_allocate_registered_protocol_desc(ni, proto_id, queue_sz,
> +				    sizeof(struct scmi_event_header) + payld_sz,
> +						    num_events, ops);
> +	if (IS_ERR(pd))
> +		goto err;
> +
> +	for (i = 0; i < num_events; i++, evt++) {
> +		struct scmi_registered_event *r_evt;
> +
> +		r_evt = devm_kzalloc(ni->handle->dev, sizeof(*r_evt),
> +				     GFP_KERNEL);
> +		if (!r_evt)
> +			goto err;
> +		r_evt->proto = pd;
> +		r_evt->evt = evt;
> +
> +		r_evt->sources = devm_kcalloc(ni->handle->dev, num_sources,
> +					      sizeof(refcount_t), GFP_KERNEL);
> +		if (!r_evt->sources)
> +			goto err;
> +		r_evt->num_sources = num_sources;
> +		mutex_init(&r_evt->sources_mtx);
> +
> +		r_evt->report = devm_kzalloc(ni->handle->dev,
> +					     evt->max_report_sz, GFP_KERNEL);
> +		if (!r_evt->report)
> +			goto err;
> +
> +		pd->registered_events[i] = r_evt;
> +		/* Ensure events are updated */
> +		smp_wmb();
> +		pr_info("SCMI Notifications: registered event - %X\n",
> +			MAKE_ALL_SRCS_KEY(r_evt->proto->id, r_evt->evt->id));
> +	}
> +
> +	/* Register protocol and events...it will never be removed */
> +	ni->registered_protocols[proto_id] = pd;
> +	/* Ensure protocols are updated */
> +	smp_wmb();
> +
> +	devres_close_group(ni->handle->dev, ni->gid);
> +
> +	return 0;
> +
> +err:
> +	pr_warn("SCMI Notifications - Proto:%X - Registration Failed !\n",
> +		proto_id);
> +	/* A failing protocol registration does not trigger full failure */
> +	devres_close_group(ni->handle->dev, ni->gid);
> +
> +	return -ENOMEM;
> +}
> +
> +/**
> + * scmi_notification_init()  - Initializes Notification Core Support
> + * @handle: The handle identifying the platform instance to initialize
> + *
> + * This function lays out all the basic resources needed by the notification
> + * core instance identified by the provided handle: once done, all of the
> + * SCMI Protocols can register their events with the core during their own
> + * initializations.
> + *
> + * Note that failing to initialize the core notifications support does not
> + * cause the whole SCMI Protocols stack to fail its initialization.
> + *
> + * SCMI Notification Initialization happens in 2 steps:
> + * * initialization: basic common allocations (this function)
> + * * registration: protocols asynchronously come into life and registers their
> + *		   own supported list of events with the core; this causes
> + *		   further per-protocol allocations
> + *
> + * Any user's callback registration attempt, referring a still not registered
> + * event, will be registered as pending and finalized later (if possible)
> + * by scmi_protocols_late_init() work.
> + * This allows for lazy initialization of SCMI Protocols due to late (or
> + * missing) SCMI drivers' modules loading.
> + *
> + * Return: 0 on Success
> + */
> +int scmi_notification_init(struct scmi_handle *handle)
> +{
> +	void *gid;
> +	struct scmi_notify_instance *ni;
> +
> +	gid = devres_open_group(handle->dev, NULL, GFP_KERNEL);
> +	if (!gid)
> +		return -ENOMEM;
> +
> +	ni = devm_kzalloc(handle->dev, sizeof(*ni), GFP_KERNEL);
> +	if (!ni)
> +		goto err;
> +
> +	ni->gid = gid;
> +	ni->handle = handle;
> +
> +	ni->registered_protocols = devm_kcalloc(handle->dev, SCMI_MAX_PROTO,
> +						sizeof(char *), GFP_KERNEL);

May not be too expensive, do we have to allocate for all 256 possible
protocols ? Will it help if we share list of implemented protocols.
I know this may get complex once we add support for registering protocols
later, but do we need it now ?

> +	if (!ni->registered_protocols)
> +		goto err;
> +
> +	handle->notify_priv = ni;
> +	/* Ensure handle is up to date */
> +	smp_wmb();
> +
> +	pr_info("SCMI Notifications Core Enabled.\n");
> +
> +	devres_close_group(handle->dev, ni->gid);
> +
> +	return 0;
> +
> +err:
> +	pr_warn("SCMI Notifications - Initialization Failed.\n");

You have defined pr_fmt, do you still need "SCMI Notifications" for
all the logging(err/warn/info) everywhere in the file. Making use of
it will help you trim the messages without loosing it in the log. Update
pr_fmt if needed.

> +	devres_release_group(handle->dev, NULL);
> +	return -ENOMEM;
> +}
> +
> +/**
> + * scmi_notification_exit()  - Shutdown and clean Notification core
> + * @handle: The handle identifying the platform instance to shutdown
> + */
> +void scmi_notification_exit(struct scmi_handle *handle)
> +{
> +	struct scmi_notify_instance *ni;
> +
> +	/* Ensure notify_priv is updated */
> +	smp_rmb();
> +	if (unlikely(!handle->notify_priv))
> +		return;
> +	ni = handle->notify_priv;
> +
> +	devres_release_group(ni->handle->dev, ni->gid);
> +}
> diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
> new file mode 100644
> index 000000000000..54094aaf812a
> --- /dev/null
> +++ b/drivers/firmware/arm_scmi/notify.h
> @@ -0,0 +1,56 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * System Control and Management Interface (SCMI) Message Protocol
> + * notification header file containing some definitions, structures
> + * and function prototypes related to SCMI Notification handling.
> + *
> + * Copyright (C) 2020 ARM Ltd.
> + */
> +#ifndef _SCMI_NOTIFY_H
> +#define _SCMI_NOTIFY_H
> +
> +#include <linux/device.h>
> +#include <linux/types.h>
> +
> +/**
> + * struct scmi_event  - Describes an event to be supported
> + * @id: Event ID
> + * @max_payld_sz: Max possible size for the payload of a notif msg of this kind
> + * @max_report_sz: Max possible size for the report of a notif msg of this kind

:s/notif msg of this kind/notification message/

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 2/9] firmware: arm_scmi: Add notification callbacks-registration

[Sudeep Holla]

On Wed, May 20, 2020 at 09:11:11AM +0100, Cristian Marussi wrote:
> Add core SCMI Notifications callbacks-registration support: allow users
> to register their own callbacks against the desired events.
> Whenever a registration request is issued against a still non existent
> event, mark such request as pending for later processing, in order to
> account for possible late initializations of SCMI Protocols associated
> to loadable drivers.
> 
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> ---
> V7 --> V8
> - Fixed init check, un-needed atomics removed
> V6 --> V7
> - removed un-needed ktime.h include
> V4 --> V5
> - fix kernel-doc
> - reviewed REVT_NOTIFY_ENABLE macro
> - added matching barrier for late_init
> V3 --> V4
> - split registered_handlers hashtable on a per-protocol basis to reduce
>   unneeded contention
> - introduced pending_handlers table and related late_init worker to finalize
>   handlers registration upon effective protocols' registrations
> - introduced further safe accessors macros for registered_protocols
>   and registered_events arrays
> V2 --> V3
> - refactored get/put event_handler
> - removed generic non-handle-based API
> V1 --> V2
> - splitted out of V1 patch 04
> - moved from IDR maps to real HashTables to store event_handlers
> - added proper enable_events refcounting via __scmi_enable_evt()
>   [was broken in V1 when using ALL_SRCIDs notification chains]
> - reviewed hashtable cleanup strategy in scmi_notifications_exit()
> - added scmi_register_event_notifier()/scmi_unregister_event_notifier()
>   to include/linux/scmi_protocol.h as a candidate user API
>   [no EXPORTs still]
> - added notify_ops to handle during initialization as an additional
>   internal API for scmi_drivers
> ---
>  drivers/firmware/arm_scmi/notify.c | 695 +++++++++++++++++++++++++++++
>  drivers/firmware/arm_scmi/notify.h |  12 +
>  include/linux/scmi_protocol.h      |  46 ++
>  3 files changed, 753 insertions(+)
> 
> diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
> index 8b620fc7df50..7cf61dbe2a8e 100644
> --- a/drivers/firmware/arm_scmi/notify.c
> +++ b/drivers/firmware/arm_scmi/notify.c
> @@ -19,17 +19,49 @@
>   * this core its set of supported events using scmi_register_protocol_events():
>   * all the needed descriptors are stored in the &struct registered_protocols and
>   * &struct registered_events arrays.
> + *
> + * Kernel users interested in some specific event can register their callbacks
> + * providing the usual notifier_block descriptor, since this core implements
> + * events' delivery using the standard Kernel notification chains machinery.
> + *
> + * Given the number of possible events defined by SCMI and the extensibility
> + * of the SCMI Protocol itself, the underlying notification chains are created
> + * and destroyed dynamically on demand depending on the number of users
> + * effectively registered for an event, so that no support structures or chains
> + * are allocated until at least one user has registered a notifier_block for
> + * such event. Similarly, events' generation itself is enabled at the platform
> + * level only after at least one user has registered, and it is shutdown after
> + * the last user for that event has gone.
> + *
> + * All users provided callbacks and allocated notification-chains are stored in
> + * the @registered_events_handlers hashtable. Callbacks' registration requests
> + * for still to be registered events are instead kept in the dedicated common
> + * hashtable @pending_events_handlers.
> + *
> + * An event is identified univocally by the tuple (proto_id, evt_id, src_id)
> + * and is served by its own dedicated notification chain; information contained
> + * in such tuples is used, in a few different ways, to generate the needed
> + * hash-keys.
> + *
> + * Here proto_id and evt_id are simply the protocol_id and message_id numbers
> + * as described in the SCMI Protocol specification, while src_id represents an
> + * optional, protocol dependent, source identifier (like domain_id, perf_id
> + * or sensor_id and so forth).
>   */
>  
>  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
>  
> +#include <linux/bitfield.h>
>  #include <linux/bug.h>
>  #include <linux/compiler.h>
>  #include <linux/device.h>
>  #include <linux/err.h>
> +#include <linux/hashtable.h>
>  #include <linux/kernel.h>
>  #include <linux/kfifo.h>
> +#include <linux/list.h>
>  #include <linux/mutex.h>
> +#include <linux/notifier.h>
>  #include <linux/refcount.h>
>  #include <linux/scmi_protocol.h>
>  #include <linux/slab.h>
> @@ -49,6 +81,74 @@
>  #define MAKE_ALL_SRCS_KEY(p, e)			\
>  	MAKE_HASH_KEY((p), (e), SCMI_ALL_SRC_IDS)
>  
> +/*
> + * Assumes that the stored obj includes its own hash-key in a field named 'key':
> + * with this simplification this macro can be equally used for all the objects'
> + * types hashed by this implementation.
> + *
> + * @__ht: The hashtable name
> + * @__obj: A pointer to the object type to be retrieved from the hashtable;
> + *	   it will be used as a cursor while scanning the hastable and it will
> + *	   be possibly left as NULL when @__k is not found
> + * @__k: The key to search for
> + */
> +#define KEY_FIND(__ht, __obj, __k)				\
> +({								\
> +	hash_for_each_possible((__ht), (__obj), hash, (__k))	\
> +		if (likely((__obj)->key == (__k)))		\
> +			break;					\
> +	__obj;							\
> +})
> +
> +#define PROTO_ID_MASK			GENMASK(31, 24)
> +#define EVT_ID_MASK			GENMASK(23, 16)
> +#define SRC_ID_MASK			GENMASK(15, 0)
> +#define KEY_XTRACT_PROTO_ID(key)	FIELD_GET(PROTO_ID_MASK, (key))
> +#define KEY_XTRACT_EVT_ID(key)		FIELD_GET(EVT_ID_MASK, (key))
> +#define KEY_XTRACT_SRC_ID(key)		FIELD_GET(SRC_ID_MASK, (key))
> +

You could move this to patch 1 and improve macros as explained there.

> +/*
> + * A set of macros used to access safely @registered_protocols and
> + * @registered_events arrays; these are fixed in size and each entry is possibly
> + * populated at protocols' registration time and then only read but NEVER
> + * modified or removed.
> + */
> +#define SCMI_GET_PROTO(__ni, __pid)					\
> +({									\
> +	struct scmi_registered_protocol_events_desc *__pd = NULL;	\
> +									\
> +	if ((__ni) && (__pid) < SCMI_MAX_PROTO)				\
> +		__pd = READ_ONCE((__ni)->registered_protocols[(__pid)]);\
> +	__pd;								\
> +})
> +
> +#define SCMI_GET_REVT_FROM_PD(__pd, __eid)				\
> +({									\
> +	struct scmi_registered_event *__revt = NULL;			\
> +									\
> +	if ((__pd) && (__eid) < (__pd)->num_events)			\
> +		__revt = READ_ONCE((__pd)->registered_events[(__eid)]);	\
> +	__revt;								\
> +})
> +
> +#define SCMI_GET_REVT(__ni, __pid, __eid)				\
> +({									\
> +	struct scmi_registered_event *__revt = NULL;			\
> +	struct scmi_registered_protocol_events_desc *__pd = NULL;	\

Do we need NULL assignment above ? The 2 macros deal with that already.

> +									\
> +	__pd = SCMI_GET_PROTO((__ni), (__pid));				\
> +	__revt = SCMI_GET_REVT_FROM_PD(__pd, (__eid));			\
> +	__revt;								\
> +})
> +
> +/* A couple of utility macros to limit cruft when calling protocols' helpers */
> +#define REVT_NOTIFY_ENABLE(revt, eid, sid)				       \
> +	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
> +						(eid), (sid), true))
> +#define REVT_NOTIFY_DISABLE(revt, eid, sid)				       \
> +	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
> +						(eid), (sid), false))
> +
>  struct scmi_registered_protocol_events_desc;
>  
>  /**
> @@ -56,9 +156,13 @@ struct scmi_registered_protocol_events_desc;
>   * core
>   * @gid: GroupID used for devres
>   * @handle: A reference to the platform instance
> + * @init_work: A work item to perform final initializations of pending handlers
> + * @pending_mtx: A mutex to protect @pending_events_handlers
>   * @registered_protocols: A statically allocated array containing pointers to
>   *			  all the registered protocol-level specific information
>   *			  related to events' handling
> + * @pending_events_handlers: An hashtable containing all pending events'
> + *			     handlers descriptors
>   *
>   * Each platform instance, represented by a handle, has its own instance of
>   * the notification subsystem represented by this structure.
> @@ -66,7 +170,12 @@ struct scmi_registered_protocol_events_desc;
>  struct scmi_notify_instance {
>  	void						*gid;
>  	struct scmi_handle				*handle;
> +
> +	struct work_struct				init_work;
> +
> +	struct mutex					pending_mtx;
>  	struct scmi_registered_protocol_events_desc	**registered_protocols;
> +	DECLARE_HASHTABLE(pending_events_handlers, 8);

What's the magic 8 here ? May be worth adding comment or reuse some macro
if already defined ?

>  };
>  
>  /**
> @@ -115,6 +224,9 @@ struct scmi_registered_event;
>   * @registered_events: A dynamically allocated array holding all the registered
>   *		       events' descriptors, whose fixed-size is determined at
>   *		       compile time.
> + * @registered_mtx: A mutex to protect @registered_events_handlers
> + * @registered_events_handlers: An hashtable containing all events' handlers
> + *				descriptors registered for this protocol
>   *
>   * All protocols that register at least one event have their protocol-specific
>   * information stored here, together with the embedded allocated events_queue.
> @@ -135,6 +247,8 @@ struct scmi_registered_protocol_events_desc {
>  	void					*in_flight;
>  	int					num_events;
>  	struct scmi_registered_event		**registered_events;
> +	struct mutex				registered_mtx;
> +	DECLARE_HASHTABLE(registered_events_handlers, 8);
>  };
>

Ditto

>  /**
> @@ -166,6 +280,38 @@ struct scmi_registered_event {
>  	struct mutex					sources_mtx;
>  };
>  
> +/**
> + * struct scmi_event_handler  - Event handler information
> + * @key: The used hashkey
> + * @users: A reference count for number of active users for this handler
> + * @r_evt: A reference to the associated registered event; when this is NULL
> + *	   this handler is pending, which means that identifies a set of
> + *	   callbacks intended to be attached to an event which is still not
> + *	   known nor registered by any protocol at that point in time
> + * @chain: The notification chain dedicated to this specific event tuple
> + * @hash: The hlist_node used for collision handling
> + * @enabled: A boolean which records if event's generation has been already
> + *	     enabled for this handler as a whole
> + *
> + * This structure collects all the information needed to process a received
> + * event identified by the tuple (proto_id, evt_id, src_id).
> + * These descriptors are stored in a per-protocol @registered_events_handlers
> + * table using as a key a value derived from that tuple.
> + */
> +struct scmi_event_handler {
> +	u32				key;
> +	refcount_t			users;
> +	struct scmi_registered_event	*r_evt;
> +	struct blocking_notifier_head	chain;
> +	struct hlist_node		hash;
> +	bool				enabled;
> +};
> +
> +#define IS_HNDL_PENDING(hndl)	((hndl)->r_evt == NULL)
> +
> +static void scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
> +				      struct scmi_event_handler *hndl);
> +
>  /**
>   * scmi_kfifo_free()  - Devres action helper to free the kfifo
>   * @kfifo: The kfifo to free
> @@ -253,6 +399,10 @@ scmi_allocate_registered_protocol_desc(struct scmi_notify_instance *ni,
>  		return ERR_PTR(-ENOMEM);
>  	pd->num_events = num_events;
>  
> +	/* Initialize per protocol handlers table */
> +	mutex_init(&pd->registered_mtx);
> +	hash_init(pd->registered_events_handlers);
> +
>  	return pd;
>  }
>  
> @@ -343,6 +493,12 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
>  
>  	devres_close_group(ni->handle->dev, ni->gid);
>  
> +	/*
> +	 * Finalize any pending events' handler which could have been waiting
> +	 * for this protocol's events registration.
> +	 */
> +	schedule_work(&ni->init_work);
> +
>  	return 0;
>  
>  err:
> @@ -354,6 +510,539 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
>  	return -ENOMEM;
>  }
>  
> +/**
> + * scmi_allocate_event_handler()  - Allocate Event handler
> + * @ni: A reference to the notification instance to use
> + * @evt_key: 32bit key uniquely bind to the event identified by the tuple
> + *	     (proto_id, evt_id, src_id)
> + *
> + * Allocate an event handler and related notification chain associated with
> + * the provided event handler key.
> + * Note that, at this point, a related registered_event is still to be
> + * associated to this handler descriptor (hndl->r_evt == NULL), so the handler
> + * is initialized as pending.
> + *
> + * Context: Assumes to be called with @pending_mtx already acquired.
> + * Return: the freshly allocated structure on Success
> + */
> +static struct scmi_event_handler *
> +scmi_allocate_event_handler(struct scmi_notify_instance *ni, u32 evt_key)
> +{
> +	struct scmi_event_handler *hndl;
> +
> +	hndl = kzalloc(sizeof(*hndl), GFP_KERNEL);
> +	if (!hndl)
> +		return ERR_PTR(-ENOMEM);
> +	hndl->key = evt_key;
> +	BLOCKING_INIT_NOTIFIER_HEAD(&hndl->chain);
> +	refcount_set(&hndl->users, 1);
> +	/* New handlers are created pending */
> +	hash_add(ni->pending_events_handlers, &hndl->hash, hndl->key);
> +
> +	return hndl;
> +}
> +
> +/**
> + * scmi_free_event_handler()  - Free the provided Event handler
> + * @hndl: The event handler structure to free
> + *
> + * Context: Assumes to be called with proper locking acquired depending
> + *	    on the situation.
> + */
> +static void scmi_free_event_handler(struct scmi_event_handler *hndl)
> +{
> +	hash_del(&hndl->hash);
> +	kfree(hndl);
> +}
> +
> +/**
> + * scmi_bind_event_handler()  - Helper to attempt binding an handler to an event
> + * @ni: A reference to the notification instance to use
> + * @hndl: The event handler to bind
> + *
> + * If an associated registered event is found, move the handler from the pending
> + * into the registered table.
> + *
> + * Context: Assumes to be called with @pending_mtx already acquired.
> + * Return: True if bind was successful, False otherwise
> + */
> +static inline bool scmi_bind_event_handler(struct scmi_notify_instance *ni,
> +					   struct scmi_event_handler *hndl)
> +{
> +	struct scmi_registered_event *r_evt;
> +
> +
> +	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(hndl->key),
> +			      KEY_XTRACT_EVT_ID(hndl->key));
> +	if (unlikely(!r_evt))
> +		return false;
> +
> +	/* Remove from pending and insert into registered */
> +	hash_del(&hndl->hash);
> +	hndl->r_evt = r_evt;
> +	mutex_lock(&r_evt->proto->registered_mtx);
> +	hash_add(r_evt->proto->registered_events_handlers,
> +		 &hndl->hash, hndl->key);
> +	mutex_unlock(&r_evt->proto->registered_mtx);
> +
> +	return true;
> +}
> +
> +/**
> + * scmi_valid_pending_handler()  - Helper to check pending status of handlers
> + * @ni: A reference to the notification instance to use
> + * @hndl: The event handler to check
> + *
> + * An handler is considered pending when its r_evt == NULL, because the related
> + * event was still unknown at handler's registration time; anyway, since all
> + * protocols register their supported events once for all at protocols'
> + * initialization time, a pending handler cannot be considered valid anymore if
> + * the underlying event (which it is waiting for), belongs to an already
> + * initialized and registered protocol.
> + *
> + * Return: True if pending registration is still valid, False otherwise.
> + */
> +static inline bool scmi_valid_pending_handler(struct scmi_notify_instance *ni,
> +					      struct scmi_event_handler *hndl)
> +{
> +	struct scmi_registered_protocol_events_desc *pd;
> +
> +	if (unlikely(!IS_HNDL_PENDING(hndl)))
> +		return false;
> +
> +	pd = SCMI_GET_PROTO(ni, KEY_XTRACT_PROTO_ID(hndl->key));
> +	if (pd)
> +		return false;
> +
> +	return true;
> +}
> +
> +/**
> + * scmi_register_event_handler()  - Register whenever possible an Event handler
> + * @ni: A reference to the notification instance to use
> + * @hndl: The event handler to register
> + *
> + * At first try to bind an event handler to its associated event, then check if
> + * it was at least a valid pending handler: if it was not bound nor valid return
> + * false.
> + *
> + * Valid pending incomplete bindings will be periodically retried by a dedicated
> + * worker which is kicked each time a new protocol completes its own
> + * registration phase.
> + *
> + * Context: Assumes to be called with @pending_mtx acquired.
> + * Return: True if a normal or a valid pending registration has been completed,
> + *	   False otherwise
> + */
> +static bool scmi_register_event_handler(struct scmi_notify_instance *ni,
> +					struct scmi_event_handler *hndl)
> +{
> +	bool ret;
> +
> +	ret = scmi_bind_event_handler(ni, hndl);
> +	if (ret) {
> +		pr_info("SCMI Notifications: registered NEW handler - key:%X\n",
> +			hndl->key);
> +	} else {
> +		ret = scmi_valid_pending_handler(ni, hndl);
> +		if (ret)
> +			pr_info("SCMI Notifications: registered PENDING handler - key:%X\n",
> +				hndl->key);
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * __scmi_event_handler_get_ops()  - Utility to get or create an event handler
> + * @ni: A reference to the notification instance to use
> + * @evt_key: The event key to use
> + * @create: A boolean flag to specify if a handler must be created when
> + *	    not already existent
> + *
> + * Search for the desired handler matching the key in both the per-protocol
> + * registered table and the common pending table:
> + * * if found adjust users refcount
> + * * if not found and @create is true, create and register the new handler:
> + *   handler could end up being registered as pending if no matching event
> + *   could be found.
> + *
> + * An handler is guaranteed to reside in one and only one of the tables at
> + * any one time; to ensure this the whole search and create is performed
> + * holding the @pending_mtx lock, with @registered_mtx additionally acquired
> + * if needed.
> + *
> + * Note that when a nested acquisition of these mutexes is needed the locking
> + * order is always (same as in @init_work):
> + * 1. pending_mtx
> + * 2. registered_mtx
> + *
> + * Events generation is NOT enabled right after creation within this routine
> + * since at creation time we usually want to have all setup and ready before
> + * events really start flowing.
> + *
> + * Return: A properly refcounted handler on Success, NULL on Failure
> + */
> +static inline struct scmi_event_handler *
> +__scmi_event_handler_get_ops(struct scmi_notify_instance *ni,
> +			     u32 evt_key, bool create)
> +{
> +	struct scmi_registered_event *r_evt;
> +	struct scmi_event_handler *hndl = NULL;
> +
> +	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(evt_key),
> +			      KEY_XTRACT_EVT_ID(evt_key));
> +
> +	mutex_lock(&ni->pending_mtx);
> +	/* Search registered events at first ... if possible at all */
> +	if (likely(r_evt)) {
> +		mutex_lock(&r_evt->proto->registered_mtx);
> +		hndl = KEY_FIND(r_evt->proto->registered_events_handlers,
> +				hndl, evt_key);
> +		if (likely(hndl))
> +			refcount_inc(&hndl->users);
> +		mutex_unlock(&r_evt->proto->registered_mtx);
> +	}
> +
> +	/* ...then amongst pending. */
> +	if (unlikely(!hndl)) {
> +		hndl = KEY_FIND(ni->pending_events_handlers, hndl, evt_key);
> +		if (likely(hndl))
> +			refcount_inc(&hndl->users);
> +	}
> +
> +	/* Create if still not found and required */
> +	if (!hndl && create) {
> +		hndl = scmi_allocate_event_handler(ni, evt_key);
> +		if (!IS_ERR_OR_NULL(hndl)) {
> +			if (!scmi_register_event_handler(ni, hndl)) {
> +				pr_info("SCMI Notifications: purging UNKNOWN handler - key:%X\n",
> +					hndl->key);
> +				/* this hndl can be only a pending one */
> +				scmi_put_handler_unlocked(ni, hndl);
> +				hndl = NULL;
> +			}
> +		}
> +	}
> +	mutex_unlock(&ni->pending_mtx);
> +
> +	return hndl;
> +}
> +
> +static struct scmi_event_handler *
> +scmi_get_handler(struct scmi_notify_instance *ni, u32 evt_key)
> +{
> +	return __scmi_event_handler_get_ops(ni, evt_key, false);
> +}
> +
> +static struct scmi_event_handler *
> +scmi_get_or_create_handler(struct scmi_notify_instance *ni, u32 evt_key)
> +{
> +	return __scmi_event_handler_get_ops(ni, evt_key, true);
> +}
> +
> +/**
> + * __scmi_enable_evt()  - Enable/disable events generation
> + * @r_evt: The registered event to act upon
> + * @src_id: The src_id to act upon
> + * @enable: The action to perform: true->Enable, false->Disable
> + *
> + * Takes care of proper refcounting while performing enable/disable: handles
> + * the special case of ALL sources requests by itself.
> + *
> + * Return: True when the required action has been successfully executed
> + */
> +static inline bool __scmi_enable_evt(struct scmi_registered_event *r_evt,
> +				     u32 src_id, bool enable)
> +{
> +	int ret = 0;
> +	u32 num_sources;
> +	refcount_t *sid;
> +
> +	if (src_id == SCMI_ALL_SRC_IDS) {
> +		src_id = 0;
> +		num_sources = r_evt->num_sources;
> +	} else if (src_id < r_evt->num_sources) {
> +		num_sources = 1;
> +	} else {
> +		return ret;
> +	}
> +
> +	mutex_lock(&r_evt->sources_mtx);
> +	if (enable) {
> +		for (; num_sources; src_id++, num_sources--) {
> +			bool r;
> +
> +			sid = &r_evt->sources[src_id];
> +			if (refcount_read(sid) == 0) {
> +				r = REVT_NOTIFY_ENABLE(r_evt,
> +						       r_evt->evt->id, src_id);
> +				if (r)
> +					refcount_set(sid, 1);
> +			} else {
> +				refcount_inc(sid);
> +				r = true;
> +			}
> +			ret += r;
> +		}
> +	} else {
> +		for (; num_sources; src_id++, num_sources--) {
> +			sid = &r_evt->sources[src_id];
> +			if (refcount_dec_and_test(sid))
> +				REVT_NOTIFY_DISABLE(r_evt,
> +						    r_evt->evt->id, src_id);
> +		}
> +		ret = 1;
> +	}
> +	mutex_unlock(&r_evt->sources_mtx);
> +
> +	return ret;
> +}
> +
> +static bool scmi_enable_events(struct scmi_event_handler *hndl)
> +{
> +	if (!hndl->enabled)
> +		hndl->enabled = __scmi_enable_evt(hndl->r_evt,
> +						  KEY_XTRACT_SRC_ID(hndl->key),
> +						  true);
> +	return hndl->enabled;
> +}
> +
> +static bool scmi_disable_events(struct scmi_event_handler *hndl)
> +{
> +	if (hndl->enabled)
> +		hndl->enabled = !__scmi_enable_evt(hndl->r_evt,
> +						   KEY_XTRACT_SRC_ID(hndl->key),
> +						   false);
> +	return !hndl->enabled;
> +}
> +
> +/**
> + * scmi_put_handler_unlocked()  - Put an event handler
> + * @ni: A reference to the notification instance to use
> + * @hndl: The event handler to act upon
> + *
> + * After having got exclusive access to the registered handlers hashtable,
> + * update the refcount and if @hndl is no more in use by anyone:
> + * * ask for events' generation disabling
> + * * unregister and free the handler itself
> + *
> + * Context: Assumes all the proper locking has been managed by the caller.
> + */
> +static void
> +scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
> +				struct scmi_event_handler *hndl)
> +{
> +	if (refcount_dec_and_test(&hndl->users)) {
> +		if (likely(!IS_HNDL_PENDING(hndl)))
> +			scmi_disable_events(hndl);
> +		scmi_free_event_handler(hndl);
> +	}
> +}
> +
> +static void scmi_put_handler(struct scmi_notify_instance *ni,
> +			     struct scmi_event_handler *hndl)
> +{
> +	struct scmi_registered_event *r_evt = hndl->r_evt;
> +
> +	mutex_lock(&ni->pending_mtx);
> +	if (r_evt)
> +		mutex_lock(&r_evt->proto->registered_mtx);
> +
> +	scmi_put_handler_unlocked(ni, hndl);
> +
> +	if (r_evt)
> +		mutex_unlock(&r_evt->proto->registered_mtx);
> +	mutex_unlock(&ni->pending_mtx);
> +}
> +
> +/**
> + * scmi_event_handler_enable_events()  - Enable events associated to an handler
> + * @hndl: The Event handler to act upon
> + *
> + * Return: True on success
> + */
> +static bool scmi_event_handler_enable_events(struct scmi_event_handler *hndl)
> +{
> +	if (!scmi_enable_events(hndl)) {
> +		pr_err("SCMI Notifications: Failed to ENABLE events for key:%X !\n",
> +		       hndl->key);
> +		return false;
> +	}
> +
> +	return true;
> +}
> +
> +/**
> + * scmi_register_notifier()  - Register a notifier_block for an event
> + * @handle: The handle identifying the platform instance against which the
> + *	    callback is registered
> + * @proto_id: Protocol ID
> + * @evt_id: Event ID
> + * @src_id: Source ID, when NULL register for events coming form ALL possible
> + *	    sources
> + * @nb: A standard notifier block to register for the specified event
> + *
> + * Generic helper to register a notifier_block against a protocol event.
> + *
> + * A notifier_block @nb will be registered for each distinct event identified
> + * by the tuple (proto_id, evt_id, src_id) on a dedicated notification chain
> + * so that:
> + *
> + *	(proto_X, evt_Y, src_Z) --> chain_X_Y_Z
> + *
> + * @src_id meaning is protocol specific and identifies the origin of the event
> + * (like domain_id, sensor_id and so forth).
> + *
> + * @src_id can be NULL to signify that the caller is interested in receiving
> + * notifications from ALL the available sources for that protocol OR simply that
> + * the protocol does not support distinct sources.
> + *
> + * As soon as one user for the specified tuple appears, an handler is created,
> + * and that specific event's generation is enabled at the platform level, unless
> + * an associated registered event is found missing, meaning that the needed
> + * protocol is still to be initialized and the handler has just been registered
> + * as still pending.
> + *
> + * Return: Return 0 on Success
> + */
> +static int scmi_register_notifier(const struct scmi_handle *handle,
> +				  u8 proto_id, u8 evt_id, u32 *src_id,
> +				  struct notifier_block *nb)
> +{
> +	int ret = 0;
> +	u32 evt_key;
> +	struct scmi_event_handler *hndl;
> +	struct scmi_notify_instance *ni;
> +
> +	/* Ensure notify_priv is updated */
> +	smp_rmb();
> +	if (unlikely(!handle->notify_priv))
> +		return -ENODEV;
> +	ni = handle->notify_priv;
> +
> +	evt_key = MAKE_HASH_KEY(proto_id, evt_id,
> +				src_id ? *src_id : SCMI_ALL_SRC_IDS);
> +	hndl = scmi_get_or_create_handler(ni, evt_key);
> +	if (IS_ERR_OR_NULL(hndl))
> +		return PTR_ERR(hndl);
> +
> +	blocking_notifier_chain_register(&hndl->chain, nb);
> +
> +	/* Enable events for not pending handlers */
> +	if (likely(!IS_HNDL_PENDING(hndl))) {
> +		if (!scmi_event_handler_enable_events(hndl)) {
> +			scmi_put_handler(ni, hndl);
> +			ret = -EINVAL;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * scmi_unregister_notifier()  - Unregister a notifier_block for an event
> + * @handle: The handle identifying the platform instance against which the
> + *	    callback is unregistered
> + * @proto_id: Protocol ID
> + * @evt_id: Event ID
> + * @src_id: Source ID
> + * @nb: The notifier_block to unregister
> + *
> + * Takes care to unregister the provided @nb from the notification chain
> + * associated to the specified event and, if there are no more users for the
> + * event handler, frees also the associated event handler structures.
> + * (this could possibly cause disabling of event's generation at platform level)
> + *
> + * Return: 0 on Success
> + */
> +static int scmi_unregister_notifier(const struct scmi_handle *handle,
> +				    u8 proto_id, u8 evt_id, u32 *src_id,
> +				    struct notifier_block *nb)
> +{
> +	u32 evt_key;
> +	struct scmi_event_handler *hndl;
> +	struct scmi_notify_instance *ni;
> +
> +	/* Ensure notify_priv is updated */
> +	smp_rmb();
> +	if (unlikely(!handle->notify_priv))
> +		return -ENODEV;
> +	ni = handle->notify_priv;
> +
> +	evt_key = MAKE_HASH_KEY(proto_id, evt_id,
> +				src_id ? *src_id : SCMI_ALL_SRC_IDS);
> +	hndl = scmi_get_handler(ni, evt_key);
> +	if (IS_ERR_OR_NULL(hndl))
> +		return -EINVAL;
> +
> +	blocking_notifier_chain_unregister(&hndl->chain, nb);
> +	scmi_put_handler(ni, hndl);
> +
> +	/*
> +	 * Free the handler (and stop events) if this happens to be the last
> +	 * known user callback for this handler; a possible concurrently ongoing
> +	 * run of @scmi_lookup_and_call_event_chain will cause this to happen
> +	 * in that context safely instead.
> +	 */

Sorry but I don't follow this as the comment states some condition while
this is done unconditionally. So each scmi_unregister_notifier decrements
refcount by 2 ?

> +	scmi_put_handler(ni, hndl);
> +
> +	return 0;
> +}
> +
> +/**
> + * scmi_protocols_late_init()  - Worker for late initialization
> + * @work: The work item to use associated to the proper SCMI instance
> + *
> + * This kicks in whenever a new protocol has completed its own registration via
> + * scmi_register_protocol_events(): it is in charge of scanning the table of
> + * pending handlers (registered by users while the related protocol was still
> + * not initialized) and finalizing their initialization whenever possible;
> + * invalid pending handlers are purged at this point in time.
> + */
> +static void scmi_protocols_late_init(struct work_struct *work)
> +{
> +	int bkt;
> +	struct scmi_event_handler *hndl;
> +	struct scmi_notify_instance *ni;
> +	struct hlist_node *tmp;
> +
> +	ni = container_of(work, struct scmi_notify_instance, init_work);
> +
> +	/* Ensure protocols and events are up to date */
> +	smp_rmb();
> +
> +	mutex_lock(&ni->pending_mtx);
> +	hash_for_each_safe(ni->pending_events_handlers, bkt, tmp, hndl, hash) {
> +		bool ret;
> +
> +		ret = scmi_bind_event_handler(ni, hndl);
> +		if (ret) {
> +			pr_info("SCMI Notifications: finalized PENDING handler - key:%X\n",
> +				hndl->key);
> +			ret = scmi_event_handler_enable_events(hndl);
> +		} else {
> +			ret = scmi_valid_pending_handler(ni, hndl);
> +		}
> +		if (!ret) {
> +			pr_info("SCMI Notifications: purging PENDING handler - key:%X\n",
> +				hndl->key);

Again all these can be debug logs.

> +			/* this hndl can be only a pending one */
> +			scmi_put_handler_unlocked(ni, hndl);
> +		}
> +	}
> +	mutex_unlock(&ni->pending_mtx);
> +}
> +
> +/*
> + * notify_ops are attached to the handle so that can be accessed
> + * directly from an scmi_driver to register its own notifiers.
> + */
> +static struct scmi_notify_ops notify_ops = {
> +	.register_event_notifier = scmi_register_notifier,
> +	.unregister_event_notifier = scmi_unregister_notifier,
> +};
> +
>  /**
>   * scmi_notification_init()  - Initializes Notification Core Support
>   * @handle: The handle identifying the platform instance to initialize
> @@ -401,6 +1090,12 @@ int scmi_notification_init(struct scmi_handle *handle)
>  	if (!ni->registered_protocols)
>  		goto err;
>  
> +	mutex_init(&ni->pending_mtx);
> +	hash_init(ni->pending_events_handlers);
> +
> +	INIT_WORK(&ni->init_work, scmi_protocols_late_init);
> +
> +	handle->notify_ops = &notify_ops;
>  	handle->notify_priv = ni;
>  	/* Ensure handle is up to date */
>  	smp_wmb();
> diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
> index 54094aaf812a..f0561fb30970 100644
> --- a/drivers/firmware/arm_scmi/notify.h
> +++ b/drivers/firmware/arm_scmi/notify.h
> @@ -9,9 +9,21 @@
>  #ifndef _SCMI_NOTIFY_H
>  #define _SCMI_NOTIFY_H
>  
> +#include <linux/bug.h>
>  #include <linux/device.h>
>  #include <linux/types.h>
>  
> +#define MAP_EVT_TO_ENABLE_CMD(id, map)			\
> +({							\
> +	int ret = -1;					\
> +							\
> +	if (likely((id) < ARRAY_SIZE((map))))		\
> +		ret = (map)[(id)];			\
> +	else						\
> +		WARN(1, "UN-KNOWN evt_id:%d\n", (id));	\
> +	ret;						\
> +})
> +

This is used just once, inline it where you use for now.

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 3/9] firmware: arm_scmi: Add notification dispatch and delivery

[Sudeep Holla]

On Wed, May 20, 2020 at 09:11:12AM +0100, Cristian Marussi wrote:
> Add core SCMI Notifications dispatch and delivery support logic which is
> able, at first, to dispatch well-known received events from the RX ISR to
> the dedicated deferred worker, and then, from there, to final deliver the
> events to the registered users' callbacks.
> 
> Dispatch and delivery is just added here, still not enabled.
> 
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> ---
>  drivers/firmware/arm_scmi/notify.c | 354 ++++++++++++++++++++++++++++-
>  drivers/firmware/arm_scmi/notify.h |  10 +
>  2 files changed, 362 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
> index 7cf61dbe2a8e..d582f71fde5b 100644
> --- a/drivers/firmware/arm_scmi/notify.c
> +++ b/drivers/firmware/arm_scmi/notify.c

[...]

> @@ -1085,6 +1422,12 @@ int scmi_notification_init(struct scmi_handle *handle)
>  	ni->gid = gid;
>  	ni->handle = handle;
>  
> +	ni->notify_wq = alloc_workqueue("scmi_notify",
> +					WQ_UNBOUND | WQ_FREEZABLE | WQ_SYSFS,
> +					0);

What's the use of WQ_SYSFS for SCMI notifications ? Do we need it ?

-- 
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 0/9] SCMI Notifications Core Support

[Sudeep Holla]

Hi Cristian,

On Wed, May 20, 2020 at 09:11:09AM +0100, Cristian Marussi wrote:
> Hi all,
>
> this series wants to introduce SCMI Notification Support, built on top of
> the standard Kernel notification chain subsystem.
>
> At initialization time each SCMI Protocol takes care to register with the
> new SCMI notification core the set of its own events which it intends to
> support.
>
> Using the API exposed via scmi_handle.notify_ops a Kernel user can register
> its own notifier_t callback (via a notifier_block as usual) against any
> registered event as identified by the tuple:
>
> 		(proto_id, event_id, src_id)
>
> where src_id represents a generic source identifier which is protocol
> dependent like domain_id, performance_id, sensor_id and so forth.
> (users can anyway do NOT provide any src_id, and subscribe instead to ALL
>  the existing (if any) src_id sources for that proto_id/evt_id combination)
>
> Each of the above tuple-specified eventis will be served on its own
> dedicated blocking notification chain, dynamically allocated on-demand when
> at least one user has shown interest on that event.
>
> Upon a notification delivery all the users' registered notifier_t callbacks
> will be in turn invoked and fed with the event_id as @action param and a
> generated custom per-event struct _report as @data param.
> (as in include/linux/scmi_protocol.h)
>
> The final step of notification delivery via users' callback invocation is
> instead delegated to a pool of deferred workers (Kernel cmwq): each
> SCMI protocol has its own dedicated worker and dedicated queue to push
> events from the rx ISR to the worker.
>
> Based on scmi-next/for-next/scmi 5.7 [1], on top of:
>
> commit f7199cf48902 ("firmware: arm_scmi: Fix return error code in
> 		     smc_send_message")
>
> This series has been tested on JUNO with an experimental firmware only
> supporting Perf Notifications.
>

Thanks for working on this. Looks mostly OK. I have few minor comments
or queries.

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/9] firmware: arm_scmi: Add notification protocol-registration

[Cristian Marussi]

Hi Sudeep

thanks for the review.

On Mon, Jun 08, 2020 at 06:02:58PM +0100, Sudeep Holla wrote:
> On Wed, May 20, 2020 at 09:11:10AM +0100, Cristian Marussi wrote:
> > Add core SCMI Notifications protocol-registration support: allow protocols
> > to register their own set of supported events, during their initialization
> > phase. Notification core can track multiple platform instances by their
> > handles.
> >
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> > ---
> >  drivers/firmware/arm_scmi/Makefile |   2 +-
> >  drivers/firmware/arm_scmi/common.h |   4 +
> >  drivers/firmware/arm_scmi/notify.c | 435 +++++++++++++++++++++++++++++
> >  drivers/firmware/arm_scmi/notify.h |  56 ++++
> >  include/linux/scmi_protocol.h      |   3 +
> >  5 files changed, 499 insertions(+), 1 deletion(-)
> >  create mode 100644 drivers/firmware/arm_scmi/notify.c
> >  create mode 100644 drivers/firmware/arm_scmi/notify.h
> >
> > diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
> > new file mode 100644
> > index 000000000000..8b620fc7df50
> > --- /dev/null
> > +++ b/drivers/firmware/arm_scmi/notify.c
> > @@ -0,0 +1,435 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * System Control and Management Interface (SCMI) Notification support
> > + *
> > + * Copyright (C) 2020 ARM Ltd.
> > + */
> > +/**
> > + * DOC: Theory of operation
> > + *
> > + * SCMI Protocol specification allows the platform to signal events to
> > + * interested agents via notification messages: this is an implementation
> > + * of the dispatch and delivery of such notifications to the interested users
> > + * inside the Linux kernel.
> > + *
> > + * An SCMI Notification core instance is initialized for each active platform
> > + * instance identified by the means of the usual &struct scmi_handle.
> > + *
> > + * Each SCMI Protocol implementation, during its initialization, registers with
> > + * this core its set of supported events using scmi_register_protocol_events():
> > + * all the needed descriptors are stored in the &struct registered_protocols and
> > + * &struct registered_events arrays.
> > + */
> > +
> > +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> > +
> > +#include <linux/bug.h>
> > +#include <linux/compiler.h>
> > +#include <linux/device.h>
> > +#include <linux/err.h>
> > +#include <linux/kernel.h>
> > +#include <linux/kfifo.h>
> > +#include <linux/mutex.h>
> > +#include <linux/refcount.h>
> > +#include <linux/scmi_protocol.h>
> > +#include <linux/slab.h>
> > +#include <linux/types.h>
> > +
> > +#include "notify.h"
> > +
> > +#define	SCMI_MAX_PROTO			256
> 
> [nit] Space after define instead of tab.
> 

Sure. (it was a one-space tab on my code..damn :D)

> > +#define	SCMI_ALL_SRC_IDS		0xffffUL
> 
> Once you move {PROTO,EVT,SRC}_ID_MASK here, you can just {re-}use SRC_ID_MASK
> 
> > +/*
> > + * Builds an unsigned 32bit key from the given input tuple to be used
> > + * as a key in hashtables.
> > + */
> > +#define MAKE_HASH_KEY(p, e, s)			\
> > +	((u32)(((p) << 24) | ((e) << 16) | ((s) & SCMI_ALL_SRC_IDS)))
> > +
> 
> You can use {PROTO,EVT,SRC}_ID_MASK here and FIELD_PREP in the above
> macro.
> 

I'll move the defs around and use the standard bitfield macros on both cases.

> > +#define MAKE_ALL_SRCS_KEY(p, e)			\
> > +	MAKE_HASH_KEY((p), (e), SCMI_ALL_SRC_IDS)
> > +
> > +struct scmi_registered_protocol_events_desc;
> > +
> 
> Just a thought here: can we rename all protocol_event* as just event*.
> The term protocol is kind of implicit and makes the names too long.
> I feel some are really too long :). Similar if you think protocol
> is implicit elsewhere, drop it.
> 

Yes you are right I hated all the time this massive protocols_ naming; I was
trying to highlight all the internal machinery related to protocols registering
with the core, but it was an ill attempt: I removed all redundant protocol_ prefixes
and it seems fine. (all but scmi_register_protocol_events() itself)

> > +/**
> > + * struct scmi_notify_instance  - Represents an instance of the notification
> > + * core
> > + * @gid: GroupID used for devres
> > + * @handle: A reference to the platform instance
> > + * @registered_protocols: A statically allocated array containing pointers to
> > + *			  all the registered protocol-level specific information
> > + *			  related to events' handling
> > + *
> > + * Each platform instance, represented by a handle, has its own instance of
> > + * the notification subsystem represented by this structure.
> > + */
> > +struct scmi_notify_instance {
> > +	void						*gid;
> > +	struct scmi_handle				*handle;
> > +	struct scmi_registered_protocol_events_desc	**registered_protocols;
> > +};
> > +
> > +/**
> > + * struct events_queue  - Describes a queue and its associated worker
> > + * @sz: Size in bytes of the related kfifo
> > + * @kfifo: A dedicated Kernel kfifo descriptor
> > + *
> > + * Each protocol has its own dedicated events_queue descriptor.
> > + */
> > +struct events_queue {
> > +	size_t				sz;
> > +	struct kfifo			kfifo;
> > +};
> > +
> 
> [nit] Add tabs only for alignment, just to keep it uniform throughout.
> 

Sure. I think this two lonely odd-tabbed things are so strangely aligned in
advance for the appearance in a following patch of another massively named
protocol field :D. I'll remove this spacing from this patch anyway.

> > +/**
> > + * struct scmi_event_header  - A utility header
> > + * @timestamp: The timestamp, in nanoseconds (boottime), which was associated
> > + *	       to this event as soon as it entered the SCMI RX ISR
> > + * @evt_id: Event ID (corresponds to the Event MsgID for this Protocol)
> > + * @payld_sz: Effective size of the embedded message payload which follows
> > + * @payld: A reference to the embedded event payload
> > + *
> > + * This header is prepended to each received event message payload before
> > + * queueing it on the related &struct events_queue.
> > + */
> > +struct scmi_event_header {
> > +	u64	timestamp;
> > +	u8	evt_id;
> > +	size_t	payld_sz;
> > +	u8	payld[];
> > +} __packed;
> > +
> 
> Is this directly read from the shmem ? Trying to understand need for
> __packed.
> 

Mmmm, not really, it's the header describing the events which is build and
pushed on the kfifo with the event payload appended to it, so it seemed to me
like a sort of 'packet' flowing somewhere and I wanted to avoid padding.

> > +struct scmi_registered_event;
> > +
> > +/**
> > + * struct scmi_registered_protocol_events_desc  - Protocol Specific information
> > + * @id: Protocol ID
> > + * @ops: Protocol specific and event-related operations
> > + * @equeue: The embedded per-protocol events_queue
> > + * @ni: A reference to the initialized instance descriptor
> > + * @eh: A reference to pre-allocated buffer to be used as a scratch area by the
> > + *	deferred worker when fetching data from the kfifo
> > + * @eh_sz: Size of the pre-allocated buffer @eh
> > + * @in_flight: A reference to an in flight &struct scmi_registered_event
> > + * @num_events: Number of events in @registered_events
> > + * @registered_events: A dynamically allocated array holding all the registered
> > + *		       events' descriptors, whose fixed-size is determined at
> > + *		       compile time.
> > + *
> > + * All protocols that register at least one event have their protocol-specific
> > + * information stored here, together with the embedded allocated events_queue.
> > + * These descriptors are stored in the @registered_protocols array at protocol
> > + * registration time.
> > + *
> > + * Once these descriptors are successfully registered, they are NEVER again
> > + * removed or modified since protocols do not unregister ever, so that, once
> > + * we safely grab a NON-NULL reference from the array we can keep it and use it.
> > + */
> > +struct scmi_registered_protocol_events_desc {
> > +	u8					id;
> > +	const struct scmi_protocol_event_ops	*ops;
> > +	struct events_queue			equeue;
> > +	struct scmi_notify_instance		*ni;
> > +	struct scmi_event_header		*eh;
> > +	size_t					eh_sz;
> > +	void					*in_flight;
> > +	int					num_events;
> > +	struct scmi_registered_event		**registered_events;
> > +};
> > +
> > +/**
> > + * struct scmi_registered_event  - Event Specific Information
> > + * @proto: A reference to the associated protocol descriptor
> > + * @evt: A reference to the associated event descriptor (as provided at
> > + *       registration time)
> > + * @report: A pre-allocated buffer used by the deferred worker to fill a
> > + *	    customized event report
> > + * @num_sources: The number of possible sources for this event as stated at
> > + *		 events' registration time
> > + * @sources: A reference to a dynamically allocated array used to refcount the
> > + *	     events' enable requests for all the existing sources
> > + * @sources_mtx: A mutex to serialize the access to @sources
> > + *
> > + * All registered events are represented by one of these structures that are
> > + * stored in the @registered_events array at protocol registration time.
> > + *
> > + * Once these descriptors are successfully registered, they are NEVER again
> > + * removed or modified since protocols do not unregister ever, so that once we
> > + * safely grab a NON-NULL reference from the table we can keep it and use it.
> > + */
> > +struct scmi_registered_event {
> > +	struct scmi_registered_protocol_events_desc	*proto;
> > +	const struct scmi_event				*evt;
> > +	void						*report;
> > +	u32						num_sources;
> > +	refcount_t					*sources;
> > +	struct mutex					sources_mtx;
> > +};
> > +
> > +/**
> > + * scmi_kfifo_free()  - Devres action helper to free the kfifo
> > + * @kfifo: The kfifo to free
> > + */
> > +static void scmi_kfifo_free(void *kfifo)
> > +{
> > +	kfifo_free((struct kfifo *)kfifo);
> > +}
> > +
> > +/**
> > + * scmi_initialize_events_queue()  - Allocate/Initialize a kfifo buffer
> > + * @ni: A reference to the notification instance to use
> > + * @equeue: The events_queue to initialize
> > + * @sz: Size of the kfifo buffer to allocate
> > + *
> > + * Allocate a buffer for the kfifo and initialize it.
> > + *
> > + * Return: 0 on Success
> > + */
> > +static int scmi_initialize_events_queue(struct scmi_notify_instance *ni,
> > +					struct events_queue *equeue, size_t sz)
> > +{
> > +	if (kfifo_alloc(&equeue->kfifo, sz, GFP_KERNEL))
> > +		return -ENOMEM;
> > +	/* Size could have been roundup to power-of-two */
> > +	equeue->sz = kfifo_size(&equeue->kfifo);
> > +
> > +	return devm_add_action_or_reset(ni->handle->dev, scmi_kfifo_free,
> > +					&equeue->kfifo);
> > +}
> > +
> > +/**
> > + * scmi_allocate_registered_protocol_desc()  - Allocate a registered protocol
> > + * events' descriptor
> > + * @ni: A reference to the &struct scmi_notify_instance notification instance
> > + *	to use
> > + * @proto_id: Protocol ID
> > + * @queue_sz: Size of the associated queue to allocate
> > + * @eh_sz: Size of the event header scratch area to pre-allocate
> > + * @num_events: Number of events to support (size of @registered_events)
> > + * @ops: Pointer to a struct holding references to protocol specific helpers
> > + *	 needed during events handling
> > + *
> > + * It is supposed to be called only once for each protocol at protocol
> > + * initialization time, so it warns if the requested protocol is found already
> > + * registered.
> > + *
> > + * Return: The allocated and registered descriptor on Success
> > + */
> > +static struct scmi_registered_protocol_events_desc *
> > +scmi_allocate_registered_protocol_desc(struct scmi_notify_instance *ni,
> > +				       u8 proto_id, size_t queue_sz,
> > +				       size_t eh_sz, int num_events,
> > +				const struct scmi_protocol_event_ops *ops)
> > +{
> > +	int ret;
> > +	struct scmi_registered_protocol_events_desc *pd;
> > +
> > +	/* Ensure protocols are up to date */
> > +	smp_rmb();
> > +	if (ni->registered_protocols[proto_id]) {
> > +		WARN_ON(1);
> 
> Can't this be if (WARN_ON(ni->registered_protocols[proto_id])) ?
> 

Yes of course.

> > +		return ERR_PTR(-EINVAL);
> > +	}
> > +
> > +	pd = devm_kzalloc(ni->handle->dev, sizeof(*pd), GFP_KERNEL);
> > +	if (!pd)
> > +		return ERR_PTR(-ENOMEM);
> > +	pd->id = proto_id;
> > +	pd->ops = ops;
> > +	pd->ni = ni;
> > +
> > +	ret = scmi_initialize_events_queue(ni, &pd->equeue, queue_sz);
> > +	if (ret)
> > +		return ERR_PTR(ret);
> > +
> > +	pd->eh = devm_kzalloc(ni->handle->dev, eh_sz, GFP_KERNEL);
> > +	if (!pd->eh)
> > +		return ERR_PTR(-ENOMEM);
> > +	pd->eh_sz = eh_sz;
> > +
> > +	pd->registered_events = devm_kcalloc(ni->handle->dev, num_events,
> > +					     sizeof(char *), GFP_KERNEL);
> > +	if (!pd->registered_events)
> > +		return ERR_PTR(-ENOMEM);
> > +	pd->num_events = num_events;
> > +
> > +	return pd;
> > +}
> > +
> > +/**
> > + * scmi_register_protocol_events()  - Register Protocol Events with the core
> > + * @handle: The handle identifying the platform instance against which the
> > + *	    the protocol's events are registered
> > + * @proto_id: Protocol ID
> > + * @queue_sz: Size in bytes of the associated queue to be allocated
> > + * @ops: Protocol specific event-related operations
> > + * @evt: Event descriptor array
> > + * @num_events: Number of events in @evt array
> > + * @num_sources: Number of possible sources for this protocol on this
> > + *		 platform.
> > + *
> > + * Used by SCMI Protocols initialization code to register with the notification
> > + * core the list of supported events and their descriptors: takes care to
> > + * pre-allocate and store all needed descriptors, scratch buffers and event
> > + * queues.
> > + *
> > + * Return: 0 on Success
> > + */
> > +int scmi_register_protocol_events(const struct scmi_handle *handle,
> > +				  u8 proto_id, size_t queue_sz,
> > +				  const struct scmi_protocol_event_ops *ops,
> > +				  const struct scmi_event *evt, int num_events,
> > +				  int num_sources)
> > +{
> > +	int i;
> > +	size_t payld_sz = 0;
> > +	struct scmi_registered_protocol_events_desc *pd;
> > +	struct scmi_notify_instance *ni;
> > +
> > +	if (!ops || !evt || proto_id >= SCMI_MAX_PROTO)
> > +		return -EINVAL;
> > +
> > +	/* Ensure notify_priv is updated */
> > +	smp_rmb();
> > +	if (unlikely(!handle->notify_priv))
> > +		return -ENOMEM;
> > +	ni = handle->notify_priv;
> > +
> > +	/* Attach to the notification main devres group */
> > +	if (!devres_open_group(ni->handle->dev, ni->gid, GFP_KERNEL))
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < num_events; i++)
> > +		payld_sz = max_t(size_t, payld_sz, evt[i].max_payld_sz);
> > +	pd = scmi_allocate_registered_protocol_desc(ni, proto_id, queue_sz,
> > +				    sizeof(struct scmi_event_header) + payld_sz,
> > +						    num_events, ops);
> > +	if (IS_ERR(pd))
> > +		goto err;
> > +
> > +	for (i = 0; i < num_events; i++, evt++) {
> > +		struct scmi_registered_event *r_evt;
> > +
> > +		r_evt = devm_kzalloc(ni->handle->dev, sizeof(*r_evt),
> > +				     GFP_KERNEL);
> > +		if (!r_evt)
> > +			goto err;
> > +		r_evt->proto = pd;
> > +		r_evt->evt = evt;
> > +
> > +		r_evt->sources = devm_kcalloc(ni->handle->dev, num_sources,
> > +					      sizeof(refcount_t), GFP_KERNEL);
> > +		if (!r_evt->sources)
> > +			goto err;
> > +		r_evt->num_sources = num_sources;
> > +		mutex_init(&r_evt->sources_mtx);
> > +
> > +		r_evt->report = devm_kzalloc(ni->handle->dev,
> > +					     evt->max_report_sz, GFP_KERNEL);
> > +		if (!r_evt->report)
> > +			goto err;
> > +
> > +		pd->registered_events[i] = r_evt;
> > +		/* Ensure events are updated */
> > +		smp_wmb();
> > +		pr_info("SCMI Notifications: registered event - %X\n",
> > +			MAKE_ALL_SRCS_KEY(r_evt->proto->id, r_evt->evt->id));
> > +	}
> > +
> > +	/* Register protocol and events...it will never be removed */
> > +	ni->registered_protocols[proto_id] = pd;
> > +	/* Ensure protocols are updated */
> > +	smp_wmb();
> > +
> > +	devres_close_group(ni->handle->dev, ni->gid);
> > +
> > +	return 0;
> > +
> > +err:
> > +	pr_warn("SCMI Notifications - Proto:%X - Registration Failed !\n",
> > +		proto_id);
> > +	/* A failing protocol registration does not trigger full failure */
> > +	devres_close_group(ni->handle->dev, ni->gid);
> > +
> > +	return -ENOMEM;
> > +}
> > +
> > +/**
> > + * scmi_notification_init()  - Initializes Notification Core Support
> > + * @handle: The handle identifying the platform instance to initialize
> > + *
> > + * This function lays out all the basic resources needed by the notification
> > + * core instance identified by the provided handle: once done, all of the
> > + * SCMI Protocols can register their events with the core during their own
> > + * initializations.
> > + *
> > + * Note that failing to initialize the core notifications support does not
> > + * cause the whole SCMI Protocols stack to fail its initialization.
> > + *
> > + * SCMI Notification Initialization happens in 2 steps:
> > + * * initialization: basic common allocations (this function)
> > + * * registration: protocols asynchronously come into life and registers their
> > + *		   own supported list of events with the core; this causes
> > + *		   further per-protocol allocations
> > + *
> > + * Any user's callback registration attempt, referring a still not registered
> > + * event, will be registered as pending and finalized later (if possible)
> > + * by scmi_protocols_late_init() work.
> > + * This allows for lazy initialization of SCMI Protocols due to late (or
> > + * missing) SCMI drivers' modules loading.
> > + *
> > + * Return: 0 on Success
> > + */
> > +int scmi_notification_init(struct scmi_handle *handle)
> > +{
> > +	void *gid;
> > +	struct scmi_notify_instance *ni;
> > +
> > +	gid = devres_open_group(handle->dev, NULL, GFP_KERNEL);
> > +	if (!gid)
> > +		return -ENOMEM;
> > +
> > +	ni = devm_kzalloc(handle->dev, sizeof(*ni), GFP_KERNEL);
> > +	if (!ni)
> > +		goto err;
> > +
> > +	ni->gid = gid;
> > +	ni->handle = handle;
> > +
> > +	ni->registered_protocols = devm_kcalloc(handle->dev, SCMI_MAX_PROTO,
> > +						sizeof(char *), GFP_KERNEL);
> 
> May not be too expensive, do we have to allocate for all 256 possible
> protocols ? Will it help if we share list of implemented protocols.
> I know this may get complex once we add support for registering protocols
> later, but do we need it now ?
> 

So the reasoning behind this was as follows: this structure is accessed heavily
on the rx (interrupt) path to lookup the proper kfifo queue to pick (if any) to
push the SCMI event packets into, so I wanted to keep this as lockless as
possible.

At first I used an RCU and an hash table to access it safely and locklessly, but
then I realized that these slots were written only once for all at initialization time
and then only (frequently) read afterwards on the rx path. (the write-once though
happens at unpredictable time, i.e. when the related underlying protocol is
initialized, action which depends on when the first SCMI driver user of that
protocol is in turn initialzed: usually at boot time, BUT possibly later if the
user driver is a module or also possibly never)

So it seemed overkill to add the complexity of an RCU htable just for writing a
word-sized aligned single-copy-atomic pointer value into an array that is written
once for all and then never changes again: the only thing I really cared here was
to concurrently write and safely read the pointer from the array and check if it
is NOT NULL.

This led indeed to a fairly sparse array mostly empty though (and a few barriers):
it seemed preferable to me at the end though.

> > +	if (!ni->registered_protocols)
> > +		goto err;
> > +
> > +	handle->notify_priv = ni;
> > +	/* Ensure handle is up to date */
> > +	smp_wmb();
> > +
> > +	pr_info("SCMI Notifications Core Enabled.\n");
> > +
> > +	devres_close_group(handle->dev, ni->gid);
> > +
> > +	return 0;
> > +
> > +err:
> > +	pr_warn("SCMI Notifications - Initialization Failed.\n");
> 
> You have defined pr_fmt, do you still need "SCMI Notifications" for
> all the logging(err/warn/info) everywhere in the file. Making use of
> it will help you trim the messages without loosing it in the log. Update
> pr_fmt if needed.
> 

Yes you are right, it just does not make sense this dumb repetition: I'll
fix the pr_fmt

> > +	devres_release_group(handle->dev, NULL);
> > +	return -ENOMEM;
> > +}
> > +
> > +/**
> > + * scmi_notification_exit()  - Shutdown and clean Notification core
> > + * @handle: The handle identifying the platform instance to shutdown
> > + */
> > +void scmi_notification_exit(struct scmi_handle *handle)
> > +{
> > +	struct scmi_notify_instance *ni;
> > +
> > +	/* Ensure notify_priv is updated */
> > +	smp_rmb();
> > +	if (unlikely(!handle->notify_priv))
> > +		return;
> > +	ni = handle->notify_priv;
> > +
> > +	devres_release_group(ni->handle->dev, ni->gid);
> > +}
> > diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
> > new file mode 100644
> > index 000000000000..54094aaf812a
> > --- /dev/null
> > +++ b/drivers/firmware/arm_scmi/notify.h
> > @@ -0,0 +1,56 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * System Control and Management Interface (SCMI) Message Protocol
> > + * notification header file containing some definitions, structures
> > + * and function prototypes related to SCMI Notification handling.
> > + *
> > + * Copyright (C) 2020 ARM Ltd.
> > + */
> > +#ifndef _SCMI_NOTIFY_H
> > +#define _SCMI_NOTIFY_H
> > +
> > +#include <linux/device.h>
> > +#include <linux/types.h>
> > +
> > +/**
> > + * struct scmi_event  - Describes an event to be supported
> > + * @id: Event ID
> > + * @max_payld_sz: Max possible size for the payload of a notif msg of this kind
> > + * @max_report_sz: Max possible size for the report of a notif msg of this kind
> 
> :s/notif msg of this kind/notification message/
> 

I'll do.

Thanks

Cristian
> --
> Regards,
> Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 2/9] firmware: arm_scmi: Add notification callbacks-registration

[Cristian Marussi]

On Mon, Jun 08, 2020 at 06:03:17PM +0100, Sudeep Holla wrote:
> On Wed, May 20, 2020 at 09:11:11AM +0100, Cristian Marussi wrote:
> > Add core SCMI Notifications callbacks-registration support: allow users
> > to register their own callbacks against the desired events.
> > Whenever a registration request is issued against a still non existent
> > event, mark such request as pending for later processing, in order to
> > account for possible late initializations of SCMI Protocols associated
> > to loadable drivers.
> > 
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> > ---
> > V7 --> V8
> > - Fixed init check, un-needed atomics removed
> > V6 --> V7
> > - removed un-needed ktime.h include
> > V4 --> V5
> > - fix kernel-doc
> > - reviewed REVT_NOTIFY_ENABLE macro
> > - added matching barrier for late_init
> > V3 --> V4
> > - split registered_handlers hashtable on a per-protocol basis to reduce
> >   unneeded contention
> > - introduced pending_handlers table and related late_init worker to finalize
> >   handlers registration upon effective protocols' registrations
> > - introduced further safe accessors macros for registered_protocols
> >   and registered_events arrays
> > V2 --> V3
> > - refactored get/put event_handler
> > - removed generic non-handle-based API
> > V1 --> V2
> > - splitted out of V1 patch 04
> > - moved from IDR maps to real HashTables to store event_handlers
> > - added proper enable_events refcounting via __scmi_enable_evt()
> >   [was broken in V1 when using ALL_SRCIDs notification chains]
> > - reviewed hashtable cleanup strategy in scmi_notifications_exit()
> > - added scmi_register_event_notifier()/scmi_unregister_event_notifier()
> >   to include/linux/scmi_protocol.h as a candidate user API
> >   [no EXPORTs still]
> > - added notify_ops to handle during initialization as an additional
> >   internal API for scmi_drivers
> > ---
> >  drivers/firmware/arm_scmi/notify.c | 695 +++++++++++++++++++++++++++++
> >  drivers/firmware/arm_scmi/notify.h |  12 +
> >  include/linux/scmi_protocol.h      |  46 ++
> >  3 files changed, 753 insertions(+)
> > 
> > diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
> > index 8b620fc7df50..7cf61dbe2a8e 100644
> > --- a/drivers/firmware/arm_scmi/notify.c
> > +++ b/drivers/firmware/arm_scmi/notify.c
> > @@ -19,17 +19,49 @@
> >   * this core its set of supported events using scmi_register_protocol_events():
> >   * all the needed descriptors are stored in the &struct registered_protocols and
> >   * &struct registered_events arrays.
> > + *
> > + * Kernel users interested in some specific event can register their callbacks
> > + * providing the usual notifier_block descriptor, since this core implements
> > + * events' delivery using the standard Kernel notification chains machinery.
> > + *
> > + * Given the number of possible events defined by SCMI and the extensibility
> > + * of the SCMI Protocol itself, the underlying notification chains are created
> > + * and destroyed dynamically on demand depending on the number of users
> > + * effectively registered for an event, so that no support structures or chains
> > + * are allocated until at least one user has registered a notifier_block for
> > + * such event. Similarly, events' generation itself is enabled at the platform
> > + * level only after at least one user has registered, and it is shutdown after
> > + * the last user for that event has gone.
> > + *
> > + * All users provided callbacks and allocated notification-chains are stored in
> > + * the @registered_events_handlers hashtable. Callbacks' registration requests
> > + * for still to be registered events are instead kept in the dedicated common
> > + * hashtable @pending_events_handlers.
> > + *
> > + * An event is identified univocally by the tuple (proto_id, evt_id, src_id)
> > + * and is served by its own dedicated notification chain; information contained
> > + * in such tuples is used, in a few different ways, to generate the needed
> > + * hash-keys.
> > + *
> > + * Here proto_id and evt_id are simply the protocol_id and message_id numbers
> > + * as described in the SCMI Protocol specification, while src_id represents an
> > + * optional, protocol dependent, source identifier (like domain_id, perf_id
> > + * or sensor_id and so forth).
> >   */
> >  
> >  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> >  
> > +#include <linux/bitfield.h>
> >  #include <linux/bug.h>
> >  #include <linux/compiler.h>
> >  #include <linux/device.h>
> >  #include <linux/err.h>
> > +#include <linux/hashtable.h>
> >  #include <linux/kernel.h>
> >  #include <linux/kfifo.h>
> > +#include <linux/list.h>
> >  #include <linux/mutex.h>
> > +#include <linux/notifier.h>
> >  #include <linux/refcount.h>
> >  #include <linux/scmi_protocol.h>
> >  #include <linux/slab.h>
> > @@ -49,6 +81,74 @@
> >  #define MAKE_ALL_SRCS_KEY(p, e)			\
> >  	MAKE_HASH_KEY((p), (e), SCMI_ALL_SRC_IDS)
> >  
> > +/*
> > + * Assumes that the stored obj includes its own hash-key in a field named 'key':
> > + * with this simplification this macro can be equally used for all the objects'
> > + * types hashed by this implementation.
> > + *
> > + * @__ht: The hashtable name
> > + * @__obj: A pointer to the object type to be retrieved from the hashtable;
> > + *	   it will be used as a cursor while scanning the hastable and it will
> > + *	   be possibly left as NULL when @__k is not found
> > + * @__k: The key to search for
> > + */
> > +#define KEY_FIND(__ht, __obj, __k)				\
> > +({								\
> > +	hash_for_each_possible((__ht), (__obj), hash, (__k))	\
> > +		if (likely((__obj)->key == (__k)))		\
> > +			break;					\
> > +	__obj;							\
> > +})
> > +
> > +#define PROTO_ID_MASK			GENMASK(31, 24)
> > +#define EVT_ID_MASK			GENMASK(23, 16)
> > +#define SRC_ID_MASK			GENMASK(15, 0)
> > +#define KEY_XTRACT_PROTO_ID(key)	FIELD_GET(PROTO_ID_MASK, (key))
> > +#define KEY_XTRACT_EVT_ID(key)		FIELD_GET(EVT_ID_MASK, (key))
> > +#define KEY_XTRACT_SRC_ID(key)		FIELD_GET(SRC_ID_MASK, (key))
> > +
> 
> You could move this to patch 1 and improve macros as explained there.
> 
Done.

> > +/*
> > + * A set of macros used to access safely @registered_protocols and
> > + * @registered_events arrays; these are fixed in size and each entry is possibly
> > + * populated at protocols' registration time and then only read but NEVER
> > + * modified or removed.
> > + */
> > +#define SCMI_GET_PROTO(__ni, __pid)					\
> > +({									\
> > +	struct scmi_registered_protocol_events_desc *__pd = NULL;	\
> > +									\
> > +	if ((__ni) && (__pid) < SCMI_MAX_PROTO)				\
> > +		__pd = READ_ONCE((__ni)->registered_protocols[(__pid)]);\
> > +	__pd;								\
> > +})
> > +
> > +#define SCMI_GET_REVT_FROM_PD(__pd, __eid)				\
> > +({									\
> > +	struct scmi_registered_event *__revt = NULL;			\
> > +									\
> > +	if ((__pd) && (__eid) < (__pd)->num_events)			\
> > +		__revt = READ_ONCE((__pd)->registered_events[(__eid)]);	\
> > +	__revt;								\
> > +})
> > +
> > +#define SCMI_GET_REVT(__ni, __pid, __eid)				\
> > +({									\
> > +	struct scmi_registered_event *__revt = NULL;			\
> > +	struct scmi_registered_protocol_events_desc *__pd = NULL;	\
> 
> Do we need NULL assignment above ? The 2 macros deal with that already.
> 

No in fact. I'll remove.

> > +									\
> > +	__pd = SCMI_GET_PROTO((__ni), (__pid));				\
> > +	__revt = SCMI_GET_REVT_FROM_PD(__pd, (__eid));			\
> > +	__revt;								\
> > +})
> > +
> > +/* A couple of utility macros to limit cruft when calling protocols' helpers */
> > +#define REVT_NOTIFY_ENABLE(revt, eid, sid)				       \
> > +	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
> > +						(eid), (sid), true))
> > +#define REVT_NOTIFY_DISABLE(revt, eid, sid)				       \
> > +	((revt)->proto->ops->set_notify_enabled((revt)->proto->ni->handle,     \
> > +						(eid), (sid), false))
> > +
> >  struct scmi_registered_protocol_events_desc;
> >  
> >  /**
> > @@ -56,9 +156,13 @@ struct scmi_registered_protocol_events_desc;
> >   * core
> >   * @gid: GroupID used for devres
> >   * @handle: A reference to the platform instance
> > + * @init_work: A work item to perform final initializations of pending handlers
> > + * @pending_mtx: A mutex to protect @pending_events_handlers
> >   * @registered_protocols: A statically allocated array containing pointers to
> >   *			  all the registered protocol-level specific information
> >   *			  related to events' handling
> > + * @pending_events_handlers: An hashtable containing all pending events'
> > + *			     handlers descriptors
> >   *
> >   * Each platform instance, represented by a handle, has its own instance of
> >   * the notification subsystem represented by this structure.
> > @@ -66,7 +170,12 @@ struct scmi_registered_protocol_events_desc;
> >  struct scmi_notify_instance {
> >  	void						*gid;
> >  	struct scmi_handle				*handle;
> > +
> > +	struct work_struct				init_work;
> > +
> > +	struct mutex					pending_mtx;
> >  	struct scmi_registered_protocol_events_desc	**registered_protocols;
> > +	DECLARE_HASHTABLE(pending_events_handlers, 8);
> 
> What's the magic 8 here ? May be worth adding comment or reuse some macro
> if already defined ?
> 

It determines the size of the hashtable (of the underlying array in fact)
as 1 << 8; 256 seemed a reasonable value to avoid most collisions since we
have 256 protocols at max and not expecting so many registerd users' callback
for each.

Avoiding collisions possibly reduces some undeterministic spike in the time
needed to deliver an event but it's anyway looked up in the deferred worker and
probably overkill as 256 (especially the pending one), so I'll reduce those
sizes (reducing size of possibly empty underlying htable arrays) and use some
meaningful macros to identify those.

> >  };
> >  
> >  /**
> > @@ -115,6 +224,9 @@ struct scmi_registered_event;
> >   * @registered_events: A dynamically allocated array holding all the registered
> >   *		       events' descriptors, whose fixed-size is determined at
> >   *		       compile time.
> > + * @registered_mtx: A mutex to protect @registered_events_handlers
> > + * @registered_events_handlers: An hashtable containing all events' handlers
> > + *				descriptors registered for this protocol
> >   *
> >   * All protocols that register at least one event have their protocol-specific
> >   * information stored here, together with the embedded allocated events_queue.
> > @@ -135,6 +247,8 @@ struct scmi_registered_protocol_events_desc {
> >  	void					*in_flight;
> >  	int					num_events;
> >  	struct scmi_registered_event		**registered_events;
> > +	struct mutex				registered_mtx;
> > +	DECLARE_HASHTABLE(registered_events_handlers, 8);
> >  };
> >
> 
> Ditto
> 

I'll fix as above.

> >  /**
> > @@ -166,6 +280,38 @@ struct scmi_registered_event {
> >  	struct mutex					sources_mtx;
> >  };
> >  
> > +/**
> > + * struct scmi_event_handler  - Event handler information
> > + * @key: The used hashkey
> > + * @users: A reference count for number of active users for this handler
> > + * @r_evt: A reference to the associated registered event; when this is NULL
> > + *	   this handler is pending, which means that identifies a set of
> > + *	   callbacks intended to be attached to an event which is still not
> > + *	   known nor registered by any protocol at that point in time
> > + * @chain: The notification chain dedicated to this specific event tuple
> > + * @hash: The hlist_node used for collision handling
> > + * @enabled: A boolean which records if event's generation has been already
> > + *	     enabled for this handler as a whole
> > + *
> > + * This structure collects all the information needed to process a received
> > + * event identified by the tuple (proto_id, evt_id, src_id).
> > + * These descriptors are stored in a per-protocol @registered_events_handlers
> > + * table using as a key a value derived from that tuple.
> > + */
> > +struct scmi_event_handler {
> > +	u32				key;
> > +	refcount_t			users;
> > +	struct scmi_registered_event	*r_evt;
> > +	struct blocking_notifier_head	chain;
> > +	struct hlist_node		hash;
> > +	bool				enabled;
> > +};
> > +
> > +#define IS_HNDL_PENDING(hndl)	((hndl)->r_evt == NULL)
> > +
> > +static void scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
> > +				      struct scmi_event_handler *hndl);
> > +
> >  /**
> >   * scmi_kfifo_free()  - Devres action helper to free the kfifo
> >   * @kfifo: The kfifo to free
> > @@ -253,6 +399,10 @@ scmi_allocate_registered_protocol_desc(struct scmi_notify_instance *ni,
> >  		return ERR_PTR(-ENOMEM);
> >  	pd->num_events = num_events;
> >  
> > +	/* Initialize per protocol handlers table */
> > +	mutex_init(&pd->registered_mtx);
> > +	hash_init(pd->registered_events_handlers);
> > +
> >  	return pd;
> >  }
> >  
> > @@ -343,6 +493,12 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
> >  
> >  	devres_close_group(ni->handle->dev, ni->gid);
> >  
> > +	/*
> > +	 * Finalize any pending events' handler which could have been waiting
> > +	 * for this protocol's events registration.
> > +	 */
> > +	schedule_work(&ni->init_work);
> > +
> >  	return 0;
> >  
> >  err:
> > @@ -354,6 +510,539 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
> >  	return -ENOMEM;
> >  }
> >  
> > +/**
> > + * scmi_allocate_event_handler()  - Allocate Event handler
> > + * @ni: A reference to the notification instance to use
> > + * @evt_key: 32bit key uniquely bind to the event identified by the tuple
> > + *	     (proto_id, evt_id, src_id)
> > + *
> > + * Allocate an event handler and related notification chain associated with
> > + * the provided event handler key.
> > + * Note that, at this point, a related registered_event is still to be
> > + * associated to this handler descriptor (hndl->r_evt == NULL), so the handler
> > + * is initialized as pending.
> > + *
> > + * Context: Assumes to be called with @pending_mtx already acquired.
> > + * Return: the freshly allocated structure on Success
> > + */
> > +static struct scmi_event_handler *
> > +scmi_allocate_event_handler(struct scmi_notify_instance *ni, u32 evt_key)
> > +{
> > +	struct scmi_event_handler *hndl;
> > +
> > +	hndl = kzalloc(sizeof(*hndl), GFP_KERNEL);
> > +	if (!hndl)
> > +		return ERR_PTR(-ENOMEM);
> > +	hndl->key = evt_key;
> > +	BLOCKING_INIT_NOTIFIER_HEAD(&hndl->chain);
> > +	refcount_set(&hndl->users, 1);
> > +	/* New handlers are created pending */
> > +	hash_add(ni->pending_events_handlers, &hndl->hash, hndl->key);
> > +
> > +	return hndl;
> > +}
> > +
> > +/**
> > + * scmi_free_event_handler()  - Free the provided Event handler
> > + * @hndl: The event handler structure to free
> > + *
> > + * Context: Assumes to be called with proper locking acquired depending
> > + *	    on the situation.
> > + */
> > +static void scmi_free_event_handler(struct scmi_event_handler *hndl)
> > +{
> > +	hash_del(&hndl->hash);
> > +	kfree(hndl);
> > +}
> > +
> > +/**
> > + * scmi_bind_event_handler()  - Helper to attempt binding an handler to an event
> > + * @ni: A reference to the notification instance to use
> > + * @hndl: The event handler to bind
> > + *
> > + * If an associated registered event is found, move the handler from the pending
> > + * into the registered table.
> > + *
> > + * Context: Assumes to be called with @pending_mtx already acquired.
> > + * Return: True if bind was successful, False otherwise
> > + */
> > +static inline bool scmi_bind_event_handler(struct scmi_notify_instance *ni,
> > +					   struct scmi_event_handler *hndl)
> > +{
> > +	struct scmi_registered_event *r_evt;
> > +
> > +
> > +	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(hndl->key),
> > +			      KEY_XTRACT_EVT_ID(hndl->key));
> > +	if (unlikely(!r_evt))
> > +		return false;
> > +
> > +	/* Remove from pending and insert into registered */
> > +	hash_del(&hndl->hash);
> > +	hndl->r_evt = r_evt;
> > +	mutex_lock(&r_evt->proto->registered_mtx);
> > +	hash_add(r_evt->proto->registered_events_handlers,
> > +		 &hndl->hash, hndl->key);
> > +	mutex_unlock(&r_evt->proto->registered_mtx);
> > +
> > +	return true;
> > +}
> > +
> > +/**
> > + * scmi_valid_pending_handler()  - Helper to check pending status of handlers
> > + * @ni: A reference to the notification instance to use
> > + * @hndl: The event handler to check
> > + *
> > + * An handler is considered pending when its r_evt == NULL, because the related
> > + * event was still unknown at handler's registration time; anyway, since all
> > + * protocols register their supported events once for all at protocols'
> > + * initialization time, a pending handler cannot be considered valid anymore if
> > + * the underlying event (which it is waiting for), belongs to an already
> > + * initialized and registered protocol.
> > + *
> > + * Return: True if pending registration is still valid, False otherwise.
> > + */
> > +static inline bool scmi_valid_pending_handler(struct scmi_notify_instance *ni,
> > +					      struct scmi_event_handler *hndl)
> > +{
> > +	struct scmi_registered_protocol_events_desc *pd;
> > +
> > +	if (unlikely(!IS_HNDL_PENDING(hndl)))
> > +		return false;
> > +
> > +	pd = SCMI_GET_PROTO(ni, KEY_XTRACT_PROTO_ID(hndl->key));
> > +	if (pd)
> > +		return false;
> > +
> > +	return true;
> > +}
> > +
> > +/**
> > + * scmi_register_event_handler()  - Register whenever possible an Event handler
> > + * @ni: A reference to the notification instance to use
> > + * @hndl: The event handler to register
> > + *
> > + * At first try to bind an event handler to its associated event, then check if
> > + * it was at least a valid pending handler: if it was not bound nor valid return
> > + * false.
> > + *
> > + * Valid pending incomplete bindings will be periodically retried by a dedicated
> > + * worker which is kicked each time a new protocol completes its own
> > + * registration phase.
> > + *
> > + * Context: Assumes to be called with @pending_mtx acquired.
> > + * Return: True if a normal or a valid pending registration has been completed,
> > + *	   False otherwise
> > + */
> > +static bool scmi_register_event_handler(struct scmi_notify_instance *ni,
> > +					struct scmi_event_handler *hndl)
> > +{
> > +	bool ret;
> > +
> > +	ret = scmi_bind_event_handler(ni, hndl);
> > +	if (ret) {
> > +		pr_info("SCMI Notifications: registered NEW handler - key:%X\n",
> > +			hndl->key);
> > +	} else {
> > +		ret = scmi_valid_pending_handler(ni, hndl);
> > +		if (ret)
> > +			pr_info("SCMI Notifications: registered PENDING handler - key:%X\n",
> > +				hndl->key);
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +/**
> > + * __scmi_event_handler_get_ops()  - Utility to get or create an event handler
> > + * @ni: A reference to the notification instance to use
> > + * @evt_key: The event key to use
> > + * @create: A boolean flag to specify if a handler must be created when
> > + *	    not already existent
> > + *
> > + * Search for the desired handler matching the key in both the per-protocol
> > + * registered table and the common pending table:
> > + * * if found adjust users refcount
> > + * * if not found and @create is true, create and register the new handler:
> > + *   handler could end up being registered as pending if no matching event
> > + *   could be found.
> > + *
> > + * An handler is guaranteed to reside in one and only one of the tables at
> > + * any one time; to ensure this the whole search and create is performed
> > + * holding the @pending_mtx lock, with @registered_mtx additionally acquired
> > + * if needed.
> > + *
> > + * Note that when a nested acquisition of these mutexes is needed the locking
> > + * order is always (same as in @init_work):
> > + * 1. pending_mtx
> > + * 2. registered_mtx
> > + *
> > + * Events generation is NOT enabled right after creation within this routine
> > + * since at creation time we usually want to have all setup and ready before
> > + * events really start flowing.
> > + *
> > + * Return: A properly refcounted handler on Success, NULL on Failure
> > + */
> > +static inline struct scmi_event_handler *
> > +__scmi_event_handler_get_ops(struct scmi_notify_instance *ni,
> > +			     u32 evt_key, bool create)
> > +{
> > +	struct scmi_registered_event *r_evt;
> > +	struct scmi_event_handler *hndl = NULL;
> > +
> > +	r_evt = SCMI_GET_REVT(ni, KEY_XTRACT_PROTO_ID(evt_key),
> > +			      KEY_XTRACT_EVT_ID(evt_key));
> > +
> > +	mutex_lock(&ni->pending_mtx);
> > +	/* Search registered events at first ... if possible at all */
> > +	if (likely(r_evt)) {
> > +		mutex_lock(&r_evt->proto->registered_mtx);
> > +		hndl = KEY_FIND(r_evt->proto->registered_events_handlers,
> > +				hndl, evt_key);
> > +		if (likely(hndl))
> > +			refcount_inc(&hndl->users);
> > +		mutex_unlock(&r_evt->proto->registered_mtx);
> > +	}
> > +
> > +	/* ...then amongst pending. */
> > +	if (unlikely(!hndl)) {
> > +		hndl = KEY_FIND(ni->pending_events_handlers, hndl, evt_key);
> > +		if (likely(hndl))
> > +			refcount_inc(&hndl->users);
> > +	}
> > +
> > +	/* Create if still not found and required */
> > +	if (!hndl && create) {
> > +		hndl = scmi_allocate_event_handler(ni, evt_key);
> > +		if (!IS_ERR_OR_NULL(hndl)) {
> > +			if (!scmi_register_event_handler(ni, hndl)) {
> > +				pr_info("SCMI Notifications: purging UNKNOWN handler - key:%X\n",
> > +					hndl->key);
> > +				/* this hndl can be only a pending one */
> > +				scmi_put_handler_unlocked(ni, hndl);
> > +				hndl = NULL;
> > +			}
> > +		}
> > +	}
> > +	mutex_unlock(&ni->pending_mtx);
> > +
> > +	return hndl;
> > +}
> > +
> > +static struct scmi_event_handler *
> > +scmi_get_handler(struct scmi_notify_instance *ni, u32 evt_key)
> > +{
> > +	return __scmi_event_handler_get_ops(ni, evt_key, false);
> > +}
> > +
> > +static struct scmi_event_handler *
> > +scmi_get_or_create_handler(struct scmi_notify_instance *ni, u32 evt_key)
> > +{
> > +	return __scmi_event_handler_get_ops(ni, evt_key, true);
> > +}
> > +
> > +/**
> > + * __scmi_enable_evt()  - Enable/disable events generation
> > + * @r_evt: The registered event to act upon
> > + * @src_id: The src_id to act upon
> > + * @enable: The action to perform: true->Enable, false->Disable
> > + *
> > + * Takes care of proper refcounting while performing enable/disable: handles
> > + * the special case of ALL sources requests by itself.
> > + *
> > + * Return: True when the required action has been successfully executed
> > + */
> > +static inline bool __scmi_enable_evt(struct scmi_registered_event *r_evt,
> > +				     u32 src_id, bool enable)
> > +{
> > +	int ret = 0;
> > +	u32 num_sources;
> > +	refcount_t *sid;
> > +
> > +	if (src_id == SCMI_ALL_SRC_IDS) {
> > +		src_id = 0;
> > +		num_sources = r_evt->num_sources;
> > +	} else if (src_id < r_evt->num_sources) {
> > +		num_sources = 1;
> > +	} else {
> > +		return ret;
> > +	}
> > +
> > +	mutex_lock(&r_evt->sources_mtx);
> > +	if (enable) {
> > +		for (; num_sources; src_id++, num_sources--) {
> > +			bool r;
> > +
> > +			sid = &r_evt->sources[src_id];
> > +			if (refcount_read(sid) == 0) {
> > +				r = REVT_NOTIFY_ENABLE(r_evt,
> > +						       r_evt->evt->id, src_id);
> > +				if (r)
> > +					refcount_set(sid, 1);
> > +			} else {
> > +				refcount_inc(sid);
> > +				r = true;
> > +			}
> > +			ret += r;
> > +		}
> > +	} else {
> > +		for (; num_sources; src_id++, num_sources--) {
> > +			sid = &r_evt->sources[src_id];
> > +			if (refcount_dec_and_test(sid))
> > +				REVT_NOTIFY_DISABLE(r_evt,
> > +						    r_evt->evt->id, src_id);
> > +		}
> > +		ret = 1;
> > +	}
> > +	mutex_unlock(&r_evt->sources_mtx);
> > +
> > +	return ret;
> > +}
> > +
> > +static bool scmi_enable_events(struct scmi_event_handler *hndl)
> > +{
> > +	if (!hndl->enabled)
> > +		hndl->enabled = __scmi_enable_evt(hndl->r_evt,
> > +						  KEY_XTRACT_SRC_ID(hndl->key),
> > +						  true);
> > +	return hndl->enabled;
> > +}
> > +
> > +static bool scmi_disable_events(struct scmi_event_handler *hndl)
> > +{
> > +	if (hndl->enabled)
> > +		hndl->enabled = !__scmi_enable_evt(hndl->r_evt,
> > +						   KEY_XTRACT_SRC_ID(hndl->key),
> > +						   false);
> > +	return !hndl->enabled;
> > +}
> > +
> > +/**
> > + * scmi_put_handler_unlocked()  - Put an event handler
> > + * @ni: A reference to the notification instance to use
> > + * @hndl: The event handler to act upon
> > + *
> > + * After having got exclusive access to the registered handlers hashtable,
> > + * update the refcount and if @hndl is no more in use by anyone:
> > + * * ask for events' generation disabling
> > + * * unregister and free the handler itself
> > + *
> > + * Context: Assumes all the proper locking has been managed by the caller.
> > + */
> > +static void
> > +scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
> > +				struct scmi_event_handler *hndl)
> > +{
> > +	if (refcount_dec_and_test(&hndl->users)) {
> > +		if (likely(!IS_HNDL_PENDING(hndl)))
> > +			scmi_disable_events(hndl);
> > +		scmi_free_event_handler(hndl);
> > +	}
> > +}
> > +
> > +static void scmi_put_handler(struct scmi_notify_instance *ni,
> > +			     struct scmi_event_handler *hndl)
> > +{
> > +	struct scmi_registered_event *r_evt = hndl->r_evt;
> > +
> > +	mutex_lock(&ni->pending_mtx);
> > +	if (r_evt)
> > +		mutex_lock(&r_evt->proto->registered_mtx);
> > +
> > +	scmi_put_handler_unlocked(ni, hndl);
> > +
> > +	if (r_evt)
> > +		mutex_unlock(&r_evt->proto->registered_mtx);
> > +	mutex_unlock(&ni->pending_mtx);
> > +}
> > +
> > +/**
> > + * scmi_event_handler_enable_events()  - Enable events associated to an handler
> > + * @hndl: The Event handler to act upon
> > + *
> > + * Return: True on success
> > + */
> > +static bool scmi_event_handler_enable_events(struct scmi_event_handler *hndl)
> > +{
> > +	if (!scmi_enable_events(hndl)) {
> > +		pr_err("SCMI Notifications: Failed to ENABLE events for key:%X !\n",
> > +		       hndl->key);
> > +		return false;
> > +	}
> > +
> > +	return true;
> > +}
> > +
> > +/**
> > + * scmi_register_notifier()  - Register a notifier_block for an event
> > + * @handle: The handle identifying the platform instance against which the
> > + *	    callback is registered
> > + * @proto_id: Protocol ID
> > + * @evt_id: Event ID
> > + * @src_id: Source ID, when NULL register for events coming form ALL possible
> > + *	    sources
> > + * @nb: A standard notifier block to register for the specified event
> > + *
> > + * Generic helper to register a notifier_block against a protocol event.
> > + *
> > + * A notifier_block @nb will be registered for each distinct event identified
> > + * by the tuple (proto_id, evt_id, src_id) on a dedicated notification chain
> > + * so that:
> > + *
> > + *	(proto_X, evt_Y, src_Z) --> chain_X_Y_Z
> > + *
> > + * @src_id meaning is protocol specific and identifies the origin of the event
> > + * (like domain_id, sensor_id and so forth).
> > + *
> > + * @src_id can be NULL to signify that the caller is interested in receiving
> > + * notifications from ALL the available sources for that protocol OR simply that
> > + * the protocol does not support distinct sources.
> > + *
> > + * As soon as one user for the specified tuple appears, an handler is created,
> > + * and that specific event's generation is enabled at the platform level, unless
> > + * an associated registered event is found missing, meaning that the needed
> > + * protocol is still to be initialized and the handler has just been registered
> > + * as still pending.
> > + *
> > + * Return: Return 0 on Success
> > + */
> > +static int scmi_register_notifier(const struct scmi_handle *handle,
> > +				  u8 proto_id, u8 evt_id, u32 *src_id,
> > +				  struct notifier_block *nb)
> > +{
> > +	int ret = 0;
> > +	u32 evt_key;
> > +	struct scmi_event_handler *hndl;
> > +	struct scmi_notify_instance *ni;
> > +
> > +	/* Ensure notify_priv is updated */
> > +	smp_rmb();
> > +	if (unlikely(!handle->notify_priv))
> > +		return -ENODEV;
> > +	ni = handle->notify_priv;
> > +
> > +	evt_key = MAKE_HASH_KEY(proto_id, evt_id,
> > +				src_id ? *src_id : SCMI_ALL_SRC_IDS);
> > +	hndl = scmi_get_or_create_handler(ni, evt_key);
> > +	if (IS_ERR_OR_NULL(hndl))
> > +		return PTR_ERR(hndl);
> > +
> > +	blocking_notifier_chain_register(&hndl->chain, nb);
> > +
> > +	/* Enable events for not pending handlers */
> > +	if (likely(!IS_HNDL_PENDING(hndl))) {
> > +		if (!scmi_event_handler_enable_events(hndl)) {
> > +			scmi_put_handler(ni, hndl);
> > +			ret = -EINVAL;
> > +		}
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +/**
> > + * scmi_unregister_notifier()  - Unregister a notifier_block for an event
> > + * @handle: The handle identifying the platform instance against which the
> > + *	    callback is unregistered
> > + * @proto_id: Protocol ID
> > + * @evt_id: Event ID
> > + * @src_id: Source ID
> > + * @nb: The notifier_block to unregister
> > + *
> > + * Takes care to unregister the provided @nb from the notification chain
> > + * associated to the specified event and, if there are no more users for the
> > + * event handler, frees also the associated event handler structures.
> > + * (this could possibly cause disabling of event's generation at platform level)
> > + *
> > + * Return: 0 on Success
> > + */
> > +static int scmi_unregister_notifier(const struct scmi_handle *handle,
> > +				    u8 proto_id, u8 evt_id, u32 *src_id,
> > +				    struct notifier_block *nb)
> > +{
> > +	u32 evt_key;
> > +	struct scmi_event_handler *hndl;
> > +	struct scmi_notify_instance *ni;
> > +
> > +	/* Ensure notify_priv is updated */
> > +	smp_rmb();
> > +	if (unlikely(!handle->notify_priv))
> > +		return -ENODEV;
> > +	ni = handle->notify_priv;
> > +
> > +	evt_key = MAKE_HASH_KEY(proto_id, evt_id,
> > +				src_id ? *src_id : SCMI_ALL_SRC_IDS);
> > +	hndl = scmi_get_handler(ni, evt_key);
> > +	if (IS_ERR_OR_NULL(hndl))
> > +		return -EINVAL;
> > +
> > +	blocking_notifier_chain_unregister(&hndl->chain, nb);
> > +	scmi_put_handler(ni, hndl);
> > +
> > +	/*
> > +	 * Free the handler (and stop events) if this happens to be the last
> > +	 * known user callback for this handler; a possible concurrently ongoing
> > +	 * run of @scmi_lookup_and_call_event_chain will cause this to happen
> > +	 * in that context safely instead.
> > +	 */
> 
> Sorry but I don't follow this as the comment states some condition while
> this is done unconditionally. So each scmi_unregister_notifier decrements
> refcount by 2 ?
> 

Basically yes...the explanation is not so short though (:D) and I tried to badly
summarized it in the comment above.

Events dispatching is handled with custom dynamically created notification
chains; such chains are created on demand as soon as the first user registers a
notifier against such event/chain and destroyed whenever the last user deregisters
the last notifier callback for the same chain: such chains and related data are
stored in an event_handler struct and every currently existing handler is stored
in a per-protocol hashtable.

Beside register/unregister ops such htable can be concurrently looked up in search
of a matching handler also during the dispatching of the received events in the
deferred workqueues; the access to the hashtable itself is protected by a table
mutex (acquired and released inside each get/put) and the handler itself is reference
counted to effectively track whenever is no more required by any user and can be freed.

So upon each get/put the mutex is temporarily acquired to atomically access the htable
and adjust the handler refcount and then released: if, at register time did not exist
already, it is created straight away with refcount 1.

But, the important thing is that, for this to work, wherever an handler is accessed
it has to be get at first and then put when done; this happens also during the delivery
phase, so that, if I'm looking up event X for delivery and some user is unregistering
those last callback for event X, I'm sure that the current lookup won't unexpectedly see
its handler vanishing while operating on it.
In such a racy case the unregister completes regularly on its side but the refcount does
not drop to zero because there is still one pending (get) lookup ongoing: once the lookup
completes and issues its own put, the handler will be finally freed in the lookup context.

So, finally, looking at how this function behaves, I have to put the handler twice, one at
first to balance the get issued early inside this same function to access the handler and
then another time to balance the get issued previously inside the register routine: if this
was the last user, in a normal non-racy scenario, this will cause, here, the refcount to
drop to zero and the handler to be freed straight away, while, in the racy scenario depicted
above, it will be the final put issue by the completion of the concurrent lookup ops that
will finally zero the refcount and free the handler.

I'll try to review the above comment in a more sensible way: my point was to give some
explanation of the double put in a concise way....does not seem to have had success :D

> > +	scmi_put_handler(ni, hndl);
> > +
> > +	return 0;
> > +}
> > +
> > +/**
> > + * scmi_protocols_late_init()  - Worker for late initialization
> > + * @work: The work item to use associated to the proper SCMI instance
> > + *
> > + * This kicks in whenever a new protocol has completed its own registration via
> > + * scmi_register_protocol_events(): it is in charge of scanning the table of
> > + * pending handlers (registered by users while the related protocol was still
> > + * not initialized) and finalizing their initialization whenever possible;
> > + * invalid pending handlers are purged at this point in time.
> > + */
> > +static void scmi_protocols_late_init(struct work_struct *work)
> > +{
> > +	int bkt;
> > +	struct scmi_event_handler *hndl;
> > +	struct scmi_notify_instance *ni;
> > +	struct hlist_node *tmp;
> > +
> > +	ni = container_of(work, struct scmi_notify_instance, init_work);
> > +
> > +	/* Ensure protocols and events are up to date */
> > +	smp_rmb();
> > +
> > +	mutex_lock(&ni->pending_mtx);
> > +	hash_for_each_safe(ni->pending_events_handlers, bkt, tmp, hndl, hash) {
> > +		bool ret;
> > +
> > +		ret = scmi_bind_event_handler(ni, hndl);
> > +		if (ret) {
> > +			pr_info("SCMI Notifications: finalized PENDING handler - key:%X\n",
> > +				hndl->key);
> > +			ret = scmi_event_handler_enable_events(hndl);
> > +		} else {
> > +			ret = scmi_valid_pending_handler(ni, hndl);
> > +		}
> > +		if (!ret) {
> > +			pr_info("SCMI Notifications: purging PENDING handler - key:%X\n",
> > +				hndl->key);
> 
> Again all these can be debug logs.
> 

Fine I'll move almost all to dev_dbg()

> > +			/* this hndl can be only a pending one */
> > +			scmi_put_handler_unlocked(ni, hndl);
> > +		}
> > +	}
> > +	mutex_unlock(&ni->pending_mtx);
> > +}
> > +
> > +/*
> > + * notify_ops are attached to the handle so that can be accessed
> > + * directly from an scmi_driver to register its own notifiers.
> > + */
> > +static struct scmi_notify_ops notify_ops = {
> > +	.register_event_notifier = scmi_register_notifier,
> > +	.unregister_event_notifier = scmi_unregister_notifier,
> > +};
> > +
> >  /**
> >   * scmi_notification_init()  - Initializes Notification Core Support
> >   * @handle: The handle identifying the platform instance to initialize
> > @@ -401,6 +1090,12 @@ int scmi_notification_init(struct scmi_handle *handle)
> >  	if (!ni->registered_protocols)
> >  		goto err;
> >  
> > +	mutex_init(&ni->pending_mtx);
> > +	hash_init(ni->pending_events_handlers);
> > +
> > +	INIT_WORK(&ni->init_work, scmi_protocols_late_init);
> > +
> > +	handle->notify_ops = &notify_ops;
> >  	handle->notify_priv = ni;
> >  	/* Ensure handle is up to date */
> >  	smp_wmb();
> > diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
> > index 54094aaf812a..f0561fb30970 100644
> > --- a/drivers/firmware/arm_scmi/notify.h
> > +++ b/drivers/firmware/arm_scmi/notify.h
> > @@ -9,9 +9,21 @@
> >  #ifndef _SCMI_NOTIFY_H
> >  #define _SCMI_NOTIFY_H
> >  
> > +#include <linux/bug.h>
> >  #include <linux/device.h>
> >  #include <linux/types.h>
> >  
> > +#define MAP_EVT_TO_ENABLE_CMD(id, map)			\
> > +({							\
> > +	int ret = -1;					\
> > +							\
> > +	if (likely((id) < ARRAY_SIZE((map))))		\
> > +		ret = (map)[(id)];			\
> > +	else						\
> > +		WARN(1, "UN-KNOWN evt_id:%d\n", (id));	\
> > +	ret;						\
> > +})
> > +
> 
> This is used just once, inline it where you use for now.
> 

 Yes I'll do, it was used more at first then version after version I dropped it
and forgot to drop it here too. (Not sure if it will be used again in other
in_progress SCMI series, but anyway is not worth having it as of now)


Thanks

Cristian

> --
> Regards,
> Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 3/9] firmware: arm_scmi: Add notification dispatch and delivery

[Cristian Marussi]

On Mon, Jun 08, 2020 at 06:03:46PM +0100, Sudeep Holla wrote:
> On Wed, May 20, 2020 at 09:11:12AM +0100, Cristian Marussi wrote:
> > Add core SCMI Notifications dispatch and delivery support logic which is
> > able, at first, to dispatch well-known received events from the RX ISR to
> > the dedicated deferred worker, and then, from there, to final deliver the
> > events to the registered users' callbacks.
> > 
> > Dispatch and delivery is just added here, still not enabled.
> > 
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> > ---
> >  drivers/firmware/arm_scmi/notify.c | 354 ++++++++++++++++++++++++++++-
> >  drivers/firmware/arm_scmi/notify.h |  10 +
> >  2 files changed, 362 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
> > index 7cf61dbe2a8e..d582f71fde5b 100644
> > --- a/drivers/firmware/arm_scmi/notify.c
> > +++ b/drivers/firmware/arm_scmi/notify.c
> 
> [...]
> 
> > @@ -1085,6 +1422,12 @@ int scmi_notification_init(struct scmi_handle *handle)
> >  	ni->gid = gid;
> >  	ni->handle = handle;
> >  
> > +	ni->notify_wq = alloc_workqueue("scmi_notify",
> > +					WQ_UNBOUND | WQ_FREEZABLE | WQ_SYSFS,
> > +					0);
> 
> What's the use of WQ_SYSFS for SCMI notifications ? Do we need it ?
> 

Lukasz asked for it, when we were talking about workqueues' priorities configurability.
(not implemented in this series)

Thanks

Cristian
> -- 
> Regards,
> Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 3/9] firmware: arm_scmi: Add notification dispatch and delivery

[Lukasz Luba]

On 6/18/20 12:31 AM, Cristian Marussi wrote:
> On Mon, Jun 08, 2020 at 06:03:46PM +0100, Sudeep Holla wrote:
>> On Wed, May 20, 2020 at 09:11:12AM +0100, Cristian Marussi wrote:
>>> Add core SCMI Notifications dispatch and delivery support logic which is
>>> able, at first, to dispatch well-known received events from the RX ISR to
>>> the dedicated deferred worker, and then, from there, to final deliver the
>>> events to the registered users' callbacks.
>>>
>>> Dispatch and delivery is just added here, still not enabled.
>>>
>>> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
>>> Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
>>> ---
>>>   drivers/firmware/arm_scmi/notify.c | 354 ++++++++++++++++++++++++++++-
>>>   drivers/firmware/arm_scmi/notify.h |  10 +
>>>   2 files changed, 362 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
>>> index 7cf61dbe2a8e..d582f71fde5b 100644
>>> --- a/drivers/firmware/arm_scmi/notify.c
>>> +++ b/drivers/firmware/arm_scmi/notify.c
>>
>> [...]
>>
>>> @@ -1085,6 +1422,12 @@ int scmi_notification_init(struct scmi_handle *handle)
>>>   	ni->gid = gid;
>>>   	ni->handle = handle;
>>>   
>>> +	ni->notify_wq = alloc_workqueue("scmi_notify",
>>> +					WQ_UNBOUND | WQ_FREEZABLE | WQ_SYSFS,
>>> +					0);
>>
>> What's the use of WQ_SYSFS for SCMI notifications ? Do we need it ?
>>
> 
> Lukasz asked for it, when we were talking about workqueues' priorities configurability.
> (not implemented in this series)

I confirm, I've asked if we can have a mechanism to control these
workqueues. They will be running concurrently with other CFS
tasks which could cause delays for them. They could also be scheduled
on a random core: big or little (depends on its utilization) but maybe
we would like to pin them explicitly to some cores, i.e
little only. We have also discussed a possible mechanism based on RT
threads (which could avoid CFS delays), but that would require a lot of
changes, so this flag here gives us some control.
But if you decide to remove this flag, we would probably find a solution
using uclamp or similar when needed.

Regards,
Lukasz

> 
> Thanks
> 
> Cristian
>> -- 
>> Regards,
>> Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 9/9] firmware: arm_scmi: Add Base notifications support

[Cristian Marussi]

On Mon, Jun 08, 2020 at 06:02:39PM +0100, Sudeep Holla wrote:
> On Wed, May 20, 2020 at 09:11:18AM +0100, Cristian Marussi wrote:
> > Make SCMI Base protocol register with the notification core.
> >
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Signed-off-by: Cristian Marussi <cristian.marussi@arm.com>
> > ---
> > V6 --> V7
> > - fixed report.timestamp type
> > - fix max_payld_sz initialization
> > - fix report layout and initialization
> > - expose SCMI_EVENT_ in linux/scmi_protocol.h
> > V5 --> V6
> > - added handle argument to fill_custom_report()
> > V4 --> V5
> > - fixed unsual return construct
> > V3 --> V4
> > - scmi_event field renamed
> > V2 --> V3
> > - added handle awareness
> > V1 --> V2
> > - simplified .set_notify_enabled() implementation moving the ALL_SRCIDs
> >   logic out of protocol. ALL_SRCIDs logic is now in charge of the
> >   notification core, together with proper reference counting of enables
> > - switched to devres protocol-registration
> > ---
> >  drivers/firmware/arm_scmi/base.c | 118 +++++++++++++++++++++++++++++--
> >  include/linux/scmi_protocol.h    |   9 +++
> >  2 files changed, 123 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/firmware/arm_scmi/base.c b/drivers/firmware/arm_scmi/base.c
> > index ce7d9203e41b..dcb098d8d823 100644
> > --- a/drivers/firmware/arm_scmi/base.c
> > +++ b/drivers/firmware/arm_scmi/base.c
> > @@ -5,7 +5,13 @@
> >   * Copyright (C) 2018 ARM Ltd.
> >   */
> >
> > +#include <linux/scmi_protocol.h>
> > +
> >  #include "common.h"
> > +#include "notify.h"
> > +
> > +#define SCMI_BASE_NUM_SOURCES		1
> > +#define SCMI_BASE_MAX_CMD_ERR_COUNT	1024
> >
> 
> I am not sure of this, see below.
> 
Ok

> >  enum scmi_base_protocol_cmd {
> >  	BASE_DISCOVER_VENDOR = 0x3,
> > @@ -19,16 +25,25 @@ enum scmi_base_protocol_cmd {
> >  	BASE_RESET_AGENT_CONFIGURATION = 0xb,
> >  };
> >
> > -enum scmi_base_protocol_notify {
> > -	BASE_ERROR_EVENT = 0x0,
> > -};
> > -
> >  struct scmi_msg_resp_base_attributes {
> >  	u8 num_protocols;
> >  	u8 num_agents;
> >  	__le16 reserved;
> >  };
> >
> > +struct scmi_msg_base_error_notify {
> > +	__le32 event_control;
> > +#define BASE_TP_NOTIFY_ALL	BIT(0)
> > +};
> > +
> > +struct scmi_base_error_notify_payld {
> > +	__le32 agent_id;
> > +	__le32 error_status;
> > +#define IS_FATAL_ERROR(x)	((x) & BIT(31))
> > +#define ERROR_CMD_COUNT(x)	FIELD_GET(GENMASK(9, 0), (x))
> > +	__le64 msg_reports[SCMI_BASE_MAX_CMD_ERR_COUNT];
> 
> This entire payload needs to fit in shmem and having huge shmem just
> for this sounds not so good to me. If this can be large, it needs to
> be iterated multiple times to get the full log.
> 

I think it's a good point, if a platform implementation would generate
such jumbo payloads for this events no matter what the actual trasport
message size was, we're definitely going to receive corrupted packets.

Spec says about BASE_ERROR_EVENT:

Bits[9:0]	Command count, number of commands in the
		command list. A value of zero is possible if the
		error cannot be attributed.

I'll ask Souvik if it's not the case to amend the spec to highlight this
possibility. (being errors notifications I suppose platform could simply
chunk the big packet in multiple pieces to fit into the current transport)

Anyway reviewing this implementation this payld struct here is defined as
to be big enough to contain the maximum allowed size by the current spec,
it is not that I am expecting to strictly receive packets sized exacly as
that; it's the same appproach I use thorughout the notifications: I reserve
an area big enough to hold any possible packet (.max_payld_sz) and then I
just check that I received a packets that fits (sizeof(*p) < payld_sz), in
case I received a variable length payload event packet as this, or check,
for other fixed-size payload events, that the received packet is of the
exact specified length.

It's just that I'm trying to pre-allocate spare areas that can fit any
possible packet length (if variable) for any possible transport.

> > +};
> > +
> >  /**
> >   * scmi_base_attributes_get() - gets the implementation details
> >   *	that are associated with the base protocol.
> > @@ -222,6 +237,95 @@ static int scmi_base_discover_agent_get(const struct scmi_handle *handle,
> >  	return ret;
> >  }
> >
> > +static int scmi_base_error_notify(const struct scmi_handle *handle, bool enable)
> > +{
> > +	int ret;
> > +	u32 evt_cntl = enable ? BASE_TP_NOTIFY_ALL : 0;
> > +	struct scmi_xfer *t;
> > +	struct scmi_msg_base_error_notify *cfg;
> > +
> > +	ret = scmi_xfer_get_init(handle, BASE_NOTIFY_ERRORS,
> > +				 SCMI_PROTOCOL_BASE, sizeof(*cfg), 0, &t);
> > +	if (ret)
> > +		return ret;
> > +
> > +	cfg = t->tx.buf;
> > +	cfg->event_control = cpu_to_le32(evt_cntl);
> > +
> > +	ret = scmi_do_xfer(handle, t);
> > +
> > +	scmi_xfer_put(handle, t);
> > +	return ret;
> > +}
> > +
> > +static bool scmi_base_set_notify_enabled(const struct scmi_handle *handle,
> > +					 u8 evt_id, u32 src_id, bool enable)
> > +{
> > +	int ret;
> > +
> > +	ret = scmi_base_error_notify(handle, enable);
> > +	if (ret)
> > +		pr_warn("SCMI Notifications - Proto:%X - FAIL_ENABLED - evt[%X] ret:%d\n",
> > +			SCMI_PROTOCOL_BASE, evt_id, ret);
> > +
> 
> I would make all these debug as they are not fatal. The users can decide
> if they are fatal and log it appropriately.
> 

Ok.

> > +	return !ret;
> > +}
> > +
> > +static void *scmi_base_fill_custom_report(const struct scmi_handle *handle,
> > +					  u8 evt_id, u64 timestamp,
> > +					  const void *payld, size_t payld_sz,
> > +					  void *report, u32 *src_id)
> > +{
> > +	void *rep = NULL;
> > +
> > +	switch (evt_id) {
> > +	case SCMI_EVENT_BASE_ERROR_EVENT:
> 
> Drop switch for now, just check for evt_id to be SCMI_EVENT_BASE_ERROR_EVENT.
> 

Ok
> > +	{
> > +		int i;
> > +		const struct scmi_base_error_notify_payld *p = payld;
> > +		struct scmi_base_error_report *r = report;
> > +
> > +		/*
> > +		 * BaseError notification payload is variable in size but
> > +		 * up to a maximum length determined by the struct ponted by p.
> > +		 * Instead payld_sz is the effective length of this notification
> > +		 * payload so cannot be greater of the maximum allowed size as
> > +		 * pointed by p.
> > +		 */
> > +		if (sizeof(*p) < payld_sz)
> > +			break;
> > +
> > +		r->timestamp = timestamp;
> > +		r->agent_id = le32_to_cpu(p->agent_id);
> > +		r->fatal = IS_FATAL_ERROR(le32_to_cpu(p->error_status));
> > +		r->cmd_count = ERROR_CMD_COUNT(le32_to_cpu(p->error_status));
> > +		for (i = 0; i < r->cmd_count; i++)
> > +			r->reports[i] = le64_to_cpu(p->msg_reports[i]);
> > +		*src_id = 0;
> > +		rep = r;
> > +		break;
> > +	}
> > +	default:
> > +		break;
> > +	}
> > +
> > +	return rep;
> > +}
> > +
> > +static const struct scmi_event base_events[] = {
> > +	{
> > +		.id = SCMI_EVENT_BASE_ERROR_EVENT,
> > +		.max_payld_sz = sizeof(struct scmi_base_error_notify_payld),
> > +		.max_report_sz = sizeof(struct scmi_base_error_report) +
> > +				  SCMI_BASE_MAX_CMD_ERR_COUNT * sizeof(u64),
> > +	},
> > +};
> > +
> > +static const struct scmi_protocol_event_ops base_event_ops = {
> > +	.set_notify_enabled = scmi_base_set_notify_enabled,
> > +	.fill_custom_report = scmi_base_fill_custom_report,
> > +};
> > +
> >  int scmi_base_protocol_init(struct scmi_handle *h)
> >  {
> >  	int id, ret;
> > @@ -256,6 +360,12 @@ int scmi_base_protocol_init(struct scmi_handle *h)
> >  	dev_dbg(dev, "Found %d protocol(s) %d agent(s)\n", rev->num_protocols,
> >  		rev->num_agents);
> >
> > +	scmi_register_protocol_events(handle,
> > +				      SCMI_PROTOCOL_BASE, (4 * PAGE_SIZE),
> 
> The size 4 * PAGE_SZ is not clear. For me this can't be more that
> max_msg_size.

This is the size required by this protocol for the per-protocol kfifo queue
allocation: it is anyway wrong as my aim was to default to 4k (or 16k for this
possibly jumbo sized packet), but this way is needlessly bound to the configured
PAGE_SIZE.

I'll introduce a define for a base 4k proto queue size and use that.

> 
> The comments in this patch applies to last 5 patches(all protocols basically)
> 

Ok I'll do.


Thanks

Cristian

> --
> Regards,
> Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel