[RFC PATCH v2 0/2] drivers/perf: hisi: Add driver for HNS3 PMU

[RFC PATCH v2 0/2] drivers/perf: hisi: Add driver for HNS3 PMU

[Guangbin Huang]

This patch-set adds driver for HNS3(HiSilicon network system version 3)
PMU and doc to descript it.

This patchset is based on 5.12-rc4. Before applying this patchset, please
apply patchset "drivers/perf: hisi: Add support for PCIe PMU" firstly,
because the code of this driver is the same directory as HiSilicon PCIe
PMU.

change log:
V2:
 - Address the comments from John.
 - Link: https://lore.kernel.org/linux-arm-kernel/1616377787-1550-1-git-send-email-huangguangbin2@huawei.com/

Guangbin Huang (2):
  drivers/perf: hisi: Add description for HNS3 PMU driver
  drivers/perf: hisi: add driver for HNS3 PMU

 Documentation/admin-guide/perf/hns3-pmu.rst | 129 +++++++++++
 MAINTAINERS                                 |   6 +
 drivers/perf/pci/Kconfig                    |   8 +
 drivers/perf/pci/hisilicon/Makefile         |   1 +
 drivers/perf/pci/hisilicon/hns3_pmu.c       | 331 ++++++++++++++++++++++++++++
 include/linux/cpuhotplug.h                  |   1 +
 6 files changed, 476 insertions(+)
 create mode 100644 Documentation/admin-guide/perf/hns3-pmu.rst
 create mode 100644 drivers/perf/pci/hisilicon/hns3_pmu.c

-- 
2.8.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC PATCH v2 1/2] drivers/perf: hisi: Add description for HNS3 PMU driver

[Guangbin Huang]

HNS3 PMU End Point device is supported on HiSilicon HIP09 platform, so
add document hns3-pmu.rst to provide guidance on how to use it.

Signed-off-by: Guangbin Huang <huangguangbin2@huawei.com>
---
 Documentation/admin-guide/perf/hns3-pmu.rst | 129 ++++++++++++++++++++++++++++
 1 file changed, 129 insertions(+)
 create mode 100644 Documentation/admin-guide/perf/hns3-pmu.rst

diff --git a/Documentation/admin-guide/perf/hns3-pmu.rst b/Documentation/admin-guide/perf/hns3-pmu.rst
new file mode 100644
index 0000000..9695688
--- /dev/null
+++ b/Documentation/admin-guide/perf/hns3-pmu.rst
@@ -0,0 +1,129 @@
+======================================
+HNS3 Performance Monitoring Unit (PMU)
+======================================
+
+HNS3(HiSilicon network system 3) Performance Monitoring Unit (PMU) is an
+End Point device to collect performance statistics of HiSilicon SoC NIC.
+On Hip09, each SICL(Super I/O cluster) has one PMU device.
+
+HNS3 PMU is supported to collect performance statistics of bandwidth,
+latency, packet rate and interrupt rate.
+
+Each HNS3 PMU supports up to 8 hardware events.
+
+HNS3 PMU driver
+===============
+
+The HNS3 PMU driver registers a perf PMU with the name of its device id.::
+
+  /sys/devices/hns3_pmu_<device_id>
+
+The device_id is read from hardware register, it contains information of
+chip_id(bit 31:2) and SICL_ID(bit 1:0). One chip may have one or more SICL.
+
+PMU driver provides description of available events, filter modes, format,
+identifier and cpumask in sysfs.
+
+The "events" directory describes the event code and subevent code of all
+supported events shown in perf list.
+
+The "filtermode" directory describes the supported filter modes of each
+event.
+
+The "format" directory describes all formats of the config (events) and
+config1 (filter options) fields of the perf_event_attr structure.
+
+The "identifier" file shows version of PMU hardware device.
+
+Example usage of checking event code and subevent code::
+
+  $# cat /sys/devices/hns3_pmu_0/events/bw_igu_ssu
+  config=0x0000
+
+The upper 8 bits of config is event code, lower 8 bits of config is
+subevent code.
+
+Example usage of checking supported filter mode::
+
+  $# cat /sys/devices/hns3_pmu_0/filtermode/bw_igu_ssu
+  filter mode supported: global/port/port-tc/
+
+Example usage of perf::
+
+  $# perf list
+  hns3_pmu_0/bw_igu_ssu/ [kernel PMU event]
+  ------------------------------------------
+
+  $# perf stat -a -e hns3_pmu_0/bw_igu_ssu,global=1/ -I 1000
+  or
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,global=1/ -I 1000
+
+The current driver does not support sampling. So "perf record" is unsupported.
+Also attach to a task is unsupported for HNS3 PMU.
+
+Filter modes
+--------------
+
+1. global mode
+PMU collect performance statistic of all functions of IO DIE. Set the
+"global" filter option to 1 will enable this mode.
+Example usage of perf::
+
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,global=1/ -I 1000
+
+2. port mode
+PMU collect performance statistic of one whole physical port. The port id
+is same as mac id. The "tc" filter option must be set to 0xF in this mode.
+Example usage of perf::
+
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,port=0,tc=0xF/ -I 1000
+
+3. port-tc mode
+PMU collect performance statistic of one tc of physical port. The port id
+is same as mac id. The "tc" filter option must be set to 0 ~ 7 in this
+mode.
+Example usage of perf::
+
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,port=0,tc=0/ -I 1000
+
+4. func mode
+PMU collect performance statistic of one PF/VF. The function id is BDF of
+PF/VF, its conversion formula::
+
+  func = (bus << 8) + (device << 3) + (function)
+
+for example:
+  BDF         func
+  35:00.0    0x3500
+  35:00.1    0x3501
+  35:01.0    0x3508
+
+In this mode, the "queue" filter option must be set to 0xFFFF.
+Example usage of perf::
+
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,bdf=0x3500,queue=0xFFFF/ -I 1000
+
+5. func-queue mode
+PMU collect performance statistic of one queue of PF/VF. The function id
+is BDF of PF/VF, its conversion formula::
+
+  func = (bus << 8) + (device << 3) + (function)
+
+In this mode, the "queue" filter option must be set to the exact queue id
+of function.
+Example usage of perf::
+
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,bdf=0x3500,queue=0/ -I 1000
+
+
+6. func-intr mode
+PMU collect performance statistic of one interrupt of PF/VF. The function
+id is BDF of PF/VF, its conversion formula::
+
+  func = (bus << 8) + (device << 3) + (function)
+
+In this mode, the "intr" filter option must be set to the exact interrupt
+id of function.
+Example usage of perf::
+
+  $# perf stat -a -e hns3_pmu_0/event=0,subevent=0,bdf=0x3500,intr=0/ -I 1000
-- 
2.8.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC PATCH v2 2/2] drivers/perf: hisi: add driver for HNS3 PMU

[Guangbin Huang]

HNS3 PMU End Point device supports to collect performance statistics
of bandwidth, latency, packet rate, interrupt rate in HiSilicon SoC
NIC.

NIC of each IO DIE has one PMU device for it. Driver registers each
PMU device to perf, and exports information of supported events,
filter mode of each event, identifier and so on via sysfs.

Each PMU device has its own control, counter and interrupt registers,
and supports up to 8 events by hardware.

Filter options contains:
event        - select event
subevent     - select subevent
port         - select physical port of nic
tc           - select tc(must be used with port)
func         - select PF/VF
queue        - select queue of PF/VF(must be used with func)
intr         - select interrupt number(must be used with func)
global       - select all functions of IO DIE

Signed-off-by: Guangbin Huang <huangguangbin2@huawei.com>
---
 MAINTAINERS                           |   6 +
 drivers/perf/pci/Kconfig              |   8 +
 drivers/perf/pci/hisilicon/Makefile   |   1 +
 drivers/perf/pci/hisilicon/hns3_pmu.c | 331 ++++++++++++++++++++++++++++++++++
 include/linux/cpuhotplug.h            |   1 +
 5 files changed, 347 insertions(+)
 create mode 100644 drivers/perf/pci/hisilicon/hns3_pmu.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 00e5583..6198177 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8088,6 +8088,12 @@ S:	Maintained
 F:	Documentation/admin-guide/perf/hisi-pcie-pmu.rst
 F:	drivers/perf/pci/hisilicon/hisi_pcie_pmu.c
 
+HISILICON HNS3 PMU DRIVER
+M:	Guangbin Huang <huangguangbin2@huawei.com>
+S:	Supported
+F:	Documentation/admin-guide/perf/hns3-pmu.rst
+F:	drivers/perf/pci/hisilicon/hns3_pmu.c
+
 HISILICON QM AND ZIP Controller DRIVER
 M:	Zhou Wang <wangzhou1@hisilicon.com>
 L:	linux-crypto@vger.kernel.org
diff --git a/drivers/perf/pci/Kconfig b/drivers/perf/pci/Kconfig
index 9f30291..4f85afd 100644
--- a/drivers/perf/pci/Kconfig
+++ b/drivers/perf/pci/Kconfig
@@ -13,4 +13,12 @@ config HISI_PCIE_PMU
 	  Adds the PCIe PMU into perf events system for monitoring latency,
 	  bandwidth etc.
 
+config HNS3_PMU
+	tristate "HNS3 PERF PMU"
+	depends on ARM64 && PCI
+	help
+	  Provide support for HNS3 performance monitoring unit (PMU) IEP
+	  devices.
+	  Adds the HNS3 PMU into perf events system for monitoring latency,
+	  bandwidth etc.
 endmenu
diff --git a/drivers/perf/pci/hisilicon/Makefile b/drivers/perf/pci/hisilicon/Makefile
index 65b0bd7..708326d0 100644
--- a/drivers/perf/pci/hisilicon/Makefile
+++ b/drivers/perf/pci/hisilicon/Makefile
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 obj-$(CONFIG_HISI_PCIE_PMU) += hisi_pcie_pmu.o
+obj-$(CONFIG_HNS3_PMU) += hns3_pmu.o
diff --git a/drivers/perf/pci/hisilicon/hns3_pmu.c b/drivers/perf/pci/hisilicon/hns3_pmu.c
new file mode 100644
index 0000000..97b2a40
--- /dev/null
+++ b/drivers/perf/pci/hisilicon/hns3_pmu.c
@@ -0,0 +1,331 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This driver adds support for HNS3 PMU iEP device. Related perf events are
+ * bandwidth, latency, packet rate, interrupt rate etc.
+ *
+ * Copyright (C) 2021 HiSilicon Limited
+ */
+#include <linux/bitfield.h>
+#include <linux/bitmap.h>
+#include <linux/bug.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpumask.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/perf_event.h>
+#include <linux/smp.h>
+
+/* registers offset address */
+#define HNS3_PMU_REG_GLOBAL_CTRL		0x0000
+#define HNS3_PMU_REG_CLOCK_FREQ			0x0020
+#define HNS3_PMU_REG_BDF			0x0FE0
+#define HNS3_PMU_REG_VERSION			0x0FE4
+#define HNS3_PMU_REG_DEVICE_ID			0x0FE8
+
+#define HNS3_PMU_INVALID_CPU_ID			(-1)
+
+struct hns3_pmu {
+	struct hlist_node node;
+	struct pci_dev *pdev;
+	struct pmu pmu;
+	void __iomem *base;
+	int on_cpu;
+	u32 identifier;
+	u32 hw_clk_freq; /* hardware clock frequency of PMU*/
+	/* maximum and minimun bdf allowed by PMU */
+	u16 bdf_min;
+	u16 bdf_max;
+};
+
+#define to_hns3_pmu(p)  (container_of((p), struct hns3_pmu, pmu))
+
+static ssize_t identifier_show(struct device *dev,
+			       struct device_attribute *attr, char *buf)
+{
+	struct hns3_pmu *hns3_pmu;
+
+	hns3_pmu = to_hns3_pmu(dev_get_drvdata(dev));
+
+	return sysfs_emit(buf, "0x%x\n", hns3_pmu->identifier);
+}
+static DEVICE_ATTR_RO(identifier);
+
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
+			    char *buf)
+{
+	struct hns3_pmu *hns3_pmu = to_hns3_pmu(dev_get_drvdata(dev));
+
+	return sysfs_emit(buf, "%d\n", hns3_pmu->on_cpu);
+}
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *hns3_pmu_cpumask_attrs[] = {
+	&dev_attr_cpumask.attr,
+	NULL
+};
+
+static struct attribute_group hns3_pmu_cpumask_attr_group = {
+	.attrs = hns3_pmu_cpumask_attrs,
+};
+
+static struct attribute *hns3_pmu_identifier_attrs[] = {
+	&dev_attr_identifier.attr,
+	NULL
+};
+
+static struct attribute_group hns3_pmu_identifier_attr_group = {
+	.attrs = hns3_pmu_identifier_attrs,
+};
+
+static const struct attribute_group *hns3_pmu_attr_groups[] = {
+	&hns3_pmu_cpumask_attr_group,
+	&hns3_pmu_identifier_attr_group,
+	NULL
+};
+
+static int hns3_pmu_alloc_pmu(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
+{
+	u16 device_id;
+	char *name;
+	u32 val;
+	int ret;
+
+	hns3_pmu->base = pci_ioremap_bar(pdev, 2);
+	if (!hns3_pmu->base) {
+		pci_err(pdev, "ioremap failed for hns3_pmu resource\n");
+		ret = -ENOMEM;
+		goto err_ioremap_bar;
+	}
+
+	hns3_pmu->hw_clk_freq = readl(hns3_pmu->base + HNS3_PMU_REG_CLOCK_FREQ);
+
+	val = readl(hns3_pmu->base + HNS3_PMU_REG_BDF);
+	hns3_pmu->bdf_min = val & 0xffff;
+	hns3_pmu->bdf_max = val >> 16;
+
+	val = readl(hns3_pmu->base + HNS3_PMU_REG_DEVICE_ID);
+	device_id = val & 0xffff;
+	name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "hns3_pmu_%u", device_id);
+	if (!name) {
+		pci_err(pdev, "failed to allocate name for HNS3 PMU.\n");
+		ret = -ENOMEM;
+		goto err_alloc_dev_name;
+	}
+
+	hns3_pmu->pdev = pdev;
+	hns3_pmu->on_cpu = HNS3_PMU_INVALID_CPU_ID;
+	hns3_pmu->identifier = readl(hns3_pmu->base + HNS3_PMU_REG_VERSION);
+	hns3_pmu->pmu = (struct pmu) {
+		.name		= name,
+		.module		= THIS_MODULE,
+		.task_ctx_nr	= perf_invalid_context,
+		.attr_groups	= hns3_pmu_attr_groups,
+		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE,
+	};
+
+	return 0;
+
+err_alloc_dev_name:
+	iounmap(hns3_pmu->base);
+err_ioremap_bar:
+	pci_release_regions(pdev);
+
+	return ret;
+}
+
+static int hns3_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct hns3_pmu *hns3_pmu;
+
+	hns3_pmu = hlist_entry_safe(node, struct hns3_pmu, node);
+	if (!hns3_pmu)
+		return -ENODEV;
+
+	if (hns3_pmu->on_cpu == HNS3_PMU_INVALID_CPU_ID)
+		hns3_pmu->on_cpu = cpu;
+
+	return 0;
+}
+
+static int hns3_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct hns3_pmu *hns3_pmu;
+	unsigned int target;
+
+	hns3_pmu = hlist_entry_safe(node, struct hns3_pmu, node);
+	if (!hns3_pmu)
+		return -ENODEV;
+
+	/* Nothing to do if this CPU doesn't own the PMU */
+	if (hns3_pmu->on_cpu != cpu)
+		return 0;
+
+	/* Choose a new CPU from all online cpus */
+	target = cpumask_any_but(cpu_online_mask, cpu);
+	if (target >= nr_cpu_ids)
+		return 0;
+
+	perf_pmu_migrate_context(&hns3_pmu->pmu, cpu, target);
+	hns3_pmu->on_cpu = target;
+
+	return 0;
+}
+
+static int hns3_pmu_init(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
+{
+	int ret;
+
+	ret = hns3_pmu_alloc_pmu(pdev, hns3_pmu);
+	if (ret)
+		return ret;
+
+	ret = cpuhp_state_add_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
+				       &hns3_pmu->node);
+	if (ret) {
+		pci_err(pdev, "failed to register hotplug, ret = %d.\n", ret);
+		goto err_register_hotplug;
+	}
+
+	ret = perf_pmu_register(&hns3_pmu->pmu, hns3_pmu->pmu.name,
+				HNS3_PMU_INVALID_CPU_ID);
+	if (ret) {
+		pci_err(pdev, "failed to register HNS3 PMU, ret = %d.\n", ret);
+		goto err_register_pmu;
+	}
+
+	return ret;
+
+err_register_pmu:
+	cpuhp_state_remove_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
+				    &hns3_pmu->node);
+
+err_register_hotplug:
+	iounmap(hns3_pmu->base);
+
+	return ret;
+}
+
+static void hns3_pmu_uninit(struct pci_dev *pdev)
+{
+	struct hns3_pmu *hns3_pmu = (struct hns3_pmu *)pci_get_drvdata(pdev);
+
+	perf_pmu_unregister(&hns3_pmu->pmu);
+	cpuhp_state_remove_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
+				    &hns3_pmu->node);
+	iounmap(hns3_pmu->base);
+}
+
+static int hns3_pmu_init_dev(struct pci_dev *pdev)
+{
+	int ret;
+
+	ret = pci_enable_device(pdev);
+	if (ret) {
+		pci_err(pdev, "failed to enable pci device, ret = %d.\n", ret);
+		return ret;
+	}
+
+	ret = pci_request_mem_regions(pdev, "hns3_pmu");
+	if (ret < 0) {
+		pci_err(pdev, "failed to request pci mem regions, ret = %d.\n",
+			ret);
+		pci_disable_device(pdev);
+		return ret;
+	}
+
+	pci_set_master(pdev);
+
+	return 0;
+}
+
+static void hns3_pmu_uninit_dev(struct pci_dev *pdev)
+{
+	pci_clear_master(pdev);
+	pci_release_mem_regions(pdev);
+	pci_disable_device(pdev);
+}
+
+static int hns3_pmu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+	struct hns3_pmu *hns3_pmu;
+	int ret;
+
+	hns3_pmu = devm_kzalloc(&pdev->dev, sizeof(*hns3_pmu), GFP_KERNEL);
+	if (!hns3_pmu)
+		return -ENOMEM;
+
+	ret = hns3_pmu_init_dev(pdev);
+	if (ret)
+		return ret;
+
+	ret = hns3_pmu_init(pdev, hns3_pmu);
+	if (ret) {
+		hns3_pmu_uninit_dev(pdev);
+		return ret;
+	}
+
+	pci_set_drvdata(pdev, hns3_pmu);
+
+	return ret;
+}
+
+static void hns3_pmu_remove(struct pci_dev *pdev)
+{
+	hns3_pmu_uninit(pdev);
+	hns3_pmu_uninit_dev(pdev);
+	pci_set_drvdata(pdev, NULL);
+}
+
+static const struct pci_device_id hns3_pmu_ids[] = {
+	{ PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, 0xA22B) },
+	{ 0, }
+};
+MODULE_DEVICE_TABLE(pci, hns3_pmu_ids);
+
+static struct pci_driver hns3_pmu_driver = {
+	.name = "hns3_pmu",
+	.id_table = hns3_pmu_ids,
+	.probe = hns3_pmu_probe,
+	.remove = hns3_pmu_remove,
+};
+
+static int __init hns3_pmu_module_init(void)
+{
+	int ret;
+
+	ret = cpuhp_setup_state_multi(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
+				      "AP_PERF_ARM_HNS3_PMU_ONLINE",
+				      hns3_pmu_online_cpu,
+				      hns3_pmu_offline_cpu);
+	if (ret) {
+		pr_err("failed to setup HNS3 PMU hotplug, ret = %d.\n", ret);
+		return ret;
+	}
+
+	ret = pci_register_driver(&hns3_pmu_driver);
+	if (ret) {
+		pr_err("failed to register pci driver, ret = %d.\n", ret);
+		cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE);
+	}
+
+	return ret;
+}
+module_init(hns3_pmu_module_init);
+
+static void __exit hns3_pmu_module_exit(void)
+{
+	pci_unregister_driver(&hns3_pmu_driver);
+	cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE);
+}
+module_exit(hns3_pmu_module_exit);
+
+MODULE_DESCRIPTION("HNS3 PMU driver");
+MODULE_LICENSE("GPL v2");
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 6a765cb..e68bed9 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -176,6 +176,7 @@ enum cpuhp_state {
 	CPUHP_AP_PERF_ARM_HISI_HHA_ONLINE,
 	CPUHP_AP_PERF_ARM_HISI_L3_ONLINE,
 	CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE,
+	CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
 	CPUHP_AP_PERF_ARM_L2X0_ONLINE,
 	CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
 	CPUHP_AP_PERF_ARM_QCOM_L3_ONLINE,
-- 
2.8.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v2 2/2] drivers/perf: hisi: add driver for HNS3 PMU

[shenjian (K)]

在 2021/5/7 21:08, Guangbin Huang 写道:
> HNS3 PMU End Point device supports to collect performance statistics
> of bandwidth, latency, packet rate, interrupt rate in HiSilicon SoC
> NIC.
>
> NIC of each IO DIE has one PMU device for it. Driver registers each
> PMU device to perf, and exports information of supported events,
> filter mode of each event, identifier and so on via sysfs.
>
> Each PMU device has its own control, counter and interrupt registers,
> and supports up to 8 events by hardware.
>
> Filter options contains:
> event        - select event
> subevent     - select subevent
> port         - select physical port of nic
> tc           - select tc(must be used with port)
> func         - select PF/VF
> queue        - select queue of PF/VF(must be used with func)
> intr         - select interrupt number(must be used with func)
> global       - select all functions of IO DIE
>
> Signed-off-by: Guangbin Huang <huangguangbin2@huawei.com>
> ---
>   MAINTAINERS                           |   6 +
>   drivers/perf/pci/Kconfig              |   8 +
>   drivers/perf/pci/hisilicon/Makefile   |   1 +
>   drivers/perf/pci/hisilicon/hns3_pmu.c | 331 ++++++++++++++++++++++++++++++++++
>   include/linux/cpuhotplug.h            |   1 +
>   5 files changed, 347 insertions(+)
>   create mode 100644 drivers/perf/pci/hisilicon/hns3_pmu.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 00e5583..6198177 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8088,6 +8088,12 @@ S:	Maintained
>   F:	Documentation/admin-guide/perf/hisi-pcie-pmu.rst
>   F:	drivers/perf/pci/hisilicon/hisi_pcie_pmu.c
>   
> +HISILICON HNS3 PMU DRIVER
> +M:	Guangbin Huang <huangguangbin2@huawei.com>
> +S:	Supported
> +F:	Documentation/admin-guide/perf/hns3-pmu.rst
> +F:	drivers/perf/pci/hisilicon/hns3_pmu.c
> +
>   HISILICON QM AND ZIP Controller DRIVER
>   M:	Zhou Wang <wangzhou1@hisilicon.com>
>   L:	linux-crypto@vger.kernel.org
> diff --git a/drivers/perf/pci/Kconfig b/drivers/perf/pci/Kconfig
> index 9f30291..4f85afd 100644
> --- a/drivers/perf/pci/Kconfig
> +++ b/drivers/perf/pci/Kconfig
> @@ -13,4 +13,12 @@ config HISI_PCIE_PMU
>   	  Adds the PCIe PMU into perf events system for monitoring latency,
>   	  bandwidth etc.
>   
> +config HNS3_PMU
> +	tristate "HNS3 PERF PMU"
> +	depends on ARM64 && PCI
> +	help
> +	  Provide support for HNS3 performance monitoring unit (PMU) IEP
> +	  devices.
> +	  Adds the HNS3 PMU into perf events system for monitoring latency,
> +	  bandwidth etc.
>   endmenu
> diff --git a/drivers/perf/pci/hisilicon/Makefile b/drivers/perf/pci/hisilicon/Makefile
> index 65b0bd7..708326d0 100644
> --- a/drivers/perf/pci/hisilicon/Makefile
> +++ b/drivers/perf/pci/hisilicon/Makefile
> @@ -1,3 +1,4 @@
>   # SPDX-License-Identifier: GPL-2.0-only
>   
>   obj-$(CONFIG_HISI_PCIE_PMU) += hisi_pcie_pmu.o
> +obj-$(CONFIG_HNS3_PMU) += hns3_pmu.o
> diff --git a/drivers/perf/pci/hisilicon/hns3_pmu.c b/drivers/perf/pci/hisilicon/hns3_pmu.c
> new file mode 100644
> index 0000000..97b2a40
> --- /dev/null
> +++ b/drivers/perf/pci/hisilicon/hns3_pmu.c
> @@ -0,0 +1,331 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * This driver adds support for HNS3 PMU iEP device. Related perf events are
> + * bandwidth, latency, packet rate, interrupt rate etc.
> + *
> + * Copyright (C) 2021 HiSilicon Limited
> + */
> +#include <linux/bitfield.h>
> +#include <linux/bitmap.h>
> +#include <linux/bug.h>
> +#include <linux/cpuhotplug.h>
> +#include <linux/cpumask.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/err.h>
> +#include <linux/interrupt.h>
> +#include <linux/iopoll.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/list.h>
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include <linux/perf_event.h>
> +#include <linux/smp.h>
> +
> +/* registers offset address */
> +#define HNS3_PMU_REG_GLOBAL_CTRL		0x0000
> +#define HNS3_PMU_REG_CLOCK_FREQ			0x0020
> +#define HNS3_PMU_REG_BDF			0x0FE0
> +#define HNS3_PMU_REG_VERSION			0x0FE4
> +#define HNS3_PMU_REG_DEVICE_ID			0x0FE8
> +
> +#define HNS3_PMU_INVALID_CPU_ID			(-1)
> +
> +struct hns3_pmu {
> +	struct hlist_node node;
> +	struct pci_dev *pdev;
> +	struct pmu pmu;
> +	void __iomem *base;
> +	int on_cpu;
> +	u32 identifier;
> +	u32 hw_clk_freq; /* hardware clock frequency of PMU*/
miss a blank before "*/"
> +	/* maximum and minimun bdf allowed by PMU */
> +	u16 bdf_min;
> +	u16 bdf_max;
> +};
> +
> +#define to_hns3_pmu(p)  (container_of((p), struct hns3_pmu, pmu))
> +
> +static ssize_t identifier_show(struct device *dev,
> +			       struct device_attribute *attr, char *buf)
> +{
> +	struct hns3_pmu *hns3_pmu;
> +
> +	hns3_pmu = to_hns3_pmu(dev_get_drvdata(dev));
> +
> +	return sysfs_emit(buf, "0x%x\n", hns3_pmu->identifier);
> +}
> +static DEVICE_ATTR_RO(identifier);
> +
> +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
> +			    char *buf)
> +{
> +	struct hns3_pmu *hns3_pmu = to_hns3_pmu(dev_get_drvdata(dev));
> +
> +	return sysfs_emit(buf, "%d\n", hns3_pmu->on_cpu);
> +}
> +static DEVICE_ATTR_RO(cpumask);
> +
> +static struct attribute *hns3_pmu_cpumask_attrs[] = {
> +	&dev_attr_cpumask.attr,
> +	NULL
> +};
> +
> +static struct attribute_group hns3_pmu_cpumask_attr_group = {
> +	.attrs = hns3_pmu_cpumask_attrs,
> +};
> +
> +static struct attribute *hns3_pmu_identifier_attrs[] = {
> +	&dev_attr_identifier.attr,
> +	NULL
> +};
> +
> +static struct attribute_group hns3_pmu_identifier_attr_group = {
> +	.attrs = hns3_pmu_identifier_attrs,
> +};
> +
> +static const struct attribute_group *hns3_pmu_attr_groups[] = {
> +	&hns3_pmu_cpumask_attr_group,
> +	&hns3_pmu_identifier_attr_group,
> +	NULL
> +};
> +
> +static int hns3_pmu_alloc_pmu(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
> +{
> +	u16 device_id;
> +	char *name;
> +	u32 val;
> +	int ret;
> +
> +	hns3_pmu->base = pci_ioremap_bar(pdev, 2);
> +	if (!hns3_pmu->base) {
> +		pci_err(pdev, "ioremap failed for hns3_pmu resource\n");
> +		ret = -ENOMEM;
> +		goto err_ioremap_bar;
> +	}
> +
> +	hns3_pmu->hw_clk_freq = readl(hns3_pmu->base + HNS3_PMU_REG_CLOCK_FREQ);
> +
> +	val = readl(hns3_pmu->base + HNS3_PMU_REG_BDF);
> +	hns3_pmu->bdf_min = val & 0xffff;
> +	hns3_pmu->bdf_max = val >> 16;
> +
> +	val = readl(hns3_pmu->base + HNS3_PMU_REG_DEVICE_ID);
> +	device_id = val & 0xffff;
> +	name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "hns3_pmu_%u", device_id);
> +	if (!name) {
> +		pci_err(pdev, "failed to allocate name for HNS3 PMU.\n");
> +		ret = -ENOMEM;
> +		goto err_alloc_dev_name;
> +	}
> +
> +	hns3_pmu->pdev = pdev;
> +	hns3_pmu->on_cpu = HNS3_PMU_INVALID_CPU_ID;
> +	hns3_pmu->identifier = readl(hns3_pmu->base + HNS3_PMU_REG_VERSION);
> +	hns3_pmu->pmu = (struct pmu) {
> +		.name		= name,
> +		.module		= THIS_MODULE,
> +		.task_ctx_nr	= perf_invalid_context,
> +		.attr_groups	= hns3_pmu_attr_groups,
> +		.capabilities	= PERF_PMU_CAP_NO_EXCLUDE,
> +	};
> +
> +	return 0;
> +
> +err_alloc_dev_name:
> +	iounmap(hns3_pmu->base);
> +err_ioremap_bar:
> +	pci_release_regions(pdev);
> +
> +	return ret;
> +}
> +
> +static int hns3_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
> +{
> +	struct hns3_pmu *hns3_pmu;
> +
> +	hns3_pmu = hlist_entry_safe(node, struct hns3_pmu, node);
> +	if (!hns3_pmu)
> +		return -ENODEV;
> +
> +	if (hns3_pmu->on_cpu == HNS3_PMU_INVALID_CPU_ID)
> +		hns3_pmu->on_cpu = cpu;
> +
> +	return 0;
> +}
> +
> +static int hns3_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
> +{
> +	struct hns3_pmu *hns3_pmu;
> +	unsigned int target;
> +
> +	hns3_pmu = hlist_entry_safe(node, struct hns3_pmu, node);
> +	if (!hns3_pmu)
> +		return -ENODEV;
> +
> +	/* Nothing to do if this CPU doesn't own the PMU */
> +	if (hns3_pmu->on_cpu != cpu)
> +		return 0;
> +
> +	/* Choose a new CPU from all online cpus */
> +	target = cpumask_any_but(cpu_online_mask, cpu);
> +	if (target >= nr_cpu_ids)
> +		return 0;
> +
> +	perf_pmu_migrate_context(&hns3_pmu->pmu, cpu, target);
> +	hns3_pmu->on_cpu = target;
> +
> +	return 0;
> +}
> +
> +static int hns3_pmu_init(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
> +{
> +	int ret;
> +
> +	ret = hns3_pmu_alloc_pmu(pdev, hns3_pmu);
> +	if (ret)
> +		return ret;
> +
> +	ret = cpuhp_state_add_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
> +				       &hns3_pmu->node);
> +	if (ret) {
> +		pci_err(pdev, "failed to register hotplug, ret = %d.\n", ret);
> +		goto err_register_hotplug;
> +	}
> +
> +	ret = perf_pmu_register(&hns3_pmu->pmu, hns3_pmu->pmu.name,
> +				HNS3_PMU_INVALID_CPU_ID);
> +	if (ret) {
> +		pci_err(pdev, "failed to register HNS3 PMU, ret = %d.\n", ret);
> +		goto err_register_pmu;
> +	}
> +
> +	return ret;
> +
> +err_register_pmu:
> +	cpuhp_state_remove_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
> +				    &hns3_pmu->node);
> +
> +err_register_hotplug:
> +	iounmap(hns3_pmu->base);
> +
> +	return ret;
> +}
> +
> +static void hns3_pmu_uninit(struct pci_dev *pdev)
> +{
> +	struct hns3_pmu *hns3_pmu = (struct hns3_pmu *)pci_get_drvdata(pdev);
> +
> +	perf_pmu_unregister(&hns3_pmu->pmu);
> +	cpuhp_state_remove_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
> +				    &hns3_pmu->node);
> +	iounmap(hns3_pmu->base);
> +}
> +
> +static int hns3_pmu_init_dev(struct pci_dev *pdev)
> +{
> +	int ret;
> +
> +	ret = pci_enable_device(pdev);
> +	if (ret) {
> +		pci_err(pdev, "failed to enable pci device, ret = %d.\n", ret);
> +		return ret;
> +	}
> +
> +	ret = pci_request_mem_regions(pdev, "hns3_pmu");
> +	if (ret < 0) {
> +		pci_err(pdev, "failed to request pci mem regions, ret = %d.\n",
> +			ret);
> +		pci_disable_device(pdev);
> +		return ret;
> +	}
> +
> +	pci_set_master(pdev);
> +
> +	return 0;
> +}
> +
> +static void hns3_pmu_uninit_dev(struct pci_dev *pdev)
> +{
> +	pci_clear_master(pdev);
> +	pci_release_mem_regions(pdev);
> +	pci_disable_device(pdev);
> +}
> +
> +static int hns3_pmu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> +{
> +	struct hns3_pmu *hns3_pmu;
> +	int ret;
> +
> +	hns3_pmu = devm_kzalloc(&pdev->dev, sizeof(*hns3_pmu), GFP_KERNEL);
> +	if (!hns3_pmu)
> +		return -ENOMEM;
> +
> +	ret = hns3_pmu_init_dev(pdev);
> +	if (ret)
> +		return ret;
> +
> +	ret = hns3_pmu_init(pdev, hns3_pmu);
> +	if (ret) {
> +		hns3_pmu_uninit_dev(pdev);
> +		return ret;
> +	}
> +
> +	pci_set_drvdata(pdev, hns3_pmu);
> +
> +	return ret;
> +}
> +
> +static void hns3_pmu_remove(struct pci_dev *pdev)
> +{
> +	hns3_pmu_uninit(pdev);
> +	hns3_pmu_uninit_dev(pdev);
> +	pci_set_drvdata(pdev, NULL);
> +}
> +
> +static const struct pci_device_id hns3_pmu_ids[] = {
> +	{ PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, 0xA22B) },
> +	{ 0, }
> +};
> +MODULE_DEVICE_TABLE(pci, hns3_pmu_ids);
> +
> +static struct pci_driver hns3_pmu_driver = {
> +	.name = "hns3_pmu",
> +	.id_table = hns3_pmu_ids,
> +	.probe = hns3_pmu_probe,
> +	.remove = hns3_pmu_remove,
> +};
> +
> +static int __init hns3_pmu_module_init(void)
> +{
> +	int ret;
> +
> +	ret = cpuhp_setup_state_multi(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
> +				      "AP_PERF_ARM_HNS3_PMU_ONLINE",
> +				      hns3_pmu_online_cpu,
> +				      hns3_pmu_offline_cpu);
> +	if (ret) {
> +		pr_err("failed to setup HNS3 PMU hotplug, ret = %d.\n", ret);
> +		return ret;
> +	}
> +
> +	ret = pci_register_driver(&hns3_pmu_driver);
> +	if (ret) {
> +		pr_err("failed to register pci driver, ret = %d.\n", ret);
> +		cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE);
> +	}
> +
> +	return ret;
> +}
> +module_init(hns3_pmu_module_init);
> +
> +static void __exit hns3_pmu_module_exit(void)
> +{
> +	pci_unregister_driver(&hns3_pmu_driver);
> +	cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE);
> +}
> +module_exit(hns3_pmu_module_exit);
> +
> +MODULE_DESCRIPTION("HNS3 PMU driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 6a765cb..e68bed9 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -176,6 +176,7 @@ enum cpuhp_state {
>   	CPUHP_AP_PERF_ARM_HISI_HHA_ONLINE,
>   	CPUHP_AP_PERF_ARM_HISI_L3_ONLINE,
>   	CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE,
> +	CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
>   	CPUHP_AP_PERF_ARM_L2X0_ONLINE,
>   	CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
>   	CPUHP_AP_PERF_ARM_QCOM_L3_ONLINE,


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v2 2/2] drivers/perf: hisi: add driver for HNS3 PMU

[huangguangbin (A)]

On 2021/5/7 22:31, shenjian (K) wrote:
> 
> 
> 在 2021/5/7 21:08, Guangbin Huang 写道:
>> HNS3 PMU End Point device supports to collect performance statistics
>> of bandwidth, latency, packet rate, interrupt rate in HiSilicon SoC
>> NIC.
>>
>> NIC of each IO DIE has one PMU device for it. Driver registers each
>> PMU device to perf, and exports information of supported events,
>> filter mode of each event, identifier and so on via sysfs.
>>
>> Each PMU device has its own control, counter and interrupt registers,
>> and supports up to 8 events by hardware.
>>
>> Filter options contains:
>> event        - select event
>> subevent     - select subevent
>> port         - select physical port of nic
>> tc           - select tc(must be used with port)
>> func         - select PF/VF
>> queue        - select queue of PF/VF(must be used with func)
>> intr         - select interrupt number(must be used with func)
>> global       - select all functions of IO DIE
>>
>> Signed-off-by: Guangbin Huang <huangguangbin2@huawei.com>
>> ---
>>   MAINTAINERS                           |   6 +
>>   drivers/perf/pci/Kconfig              |   8 +
>>   drivers/perf/pci/hisilicon/Makefile   |   1 +
>>   drivers/perf/pci/hisilicon/hns3_pmu.c | 331 ++++++++++++++++++++++++++++++++++
>>   include/linux/cpuhotplug.h            |   1 +
>>   5 files changed, 347 insertions(+)
>>   create mode 100644 drivers/perf/pci/hisilicon/hns3_pmu.c
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 00e5583..6198177 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -8088,6 +8088,12 @@ S:    Maintained
>>   F:    Documentation/admin-guide/perf/hisi-pcie-pmu.rst
>>   F:    drivers/perf/pci/hisilicon/hisi_pcie_pmu.c
>> +HISILICON HNS3 PMU DRIVER
>> +M:    Guangbin Huang <huangguangbin2@huawei.com>
>> +S:    Supported
>> +F:    Documentation/admin-guide/perf/hns3-pmu.rst
>> +F:    drivers/perf/pci/hisilicon/hns3_pmu.c
>> +
>>   HISILICON QM AND ZIP Controller DRIVER
>>   M:    Zhou Wang <wangzhou1@hisilicon.com>
>>   L:    linux-crypto@vger.kernel.org
>> diff --git a/drivers/perf/pci/Kconfig b/drivers/perf/pci/Kconfig
>> index 9f30291..4f85afd 100644
>> --- a/drivers/perf/pci/Kconfig
>> +++ b/drivers/perf/pci/Kconfig
>> @@ -13,4 +13,12 @@ config HISI_PCIE_PMU
>>         Adds the PCIe PMU into perf events system for monitoring latency,
>>         bandwidth etc.
>> +config HNS3_PMU
>> +    tristate "HNS3 PERF PMU"
>> +    depends on ARM64 && PCI
>> +    help
>> +      Provide support for HNS3 performance monitoring unit (PMU) IEP
>> +      devices.
>> +      Adds the HNS3 PMU into perf events system for monitoring latency,
>> +      bandwidth etc.
>>   endmenu
>> diff --git a/drivers/perf/pci/hisilicon/Makefile b/drivers/perf/pci/hisilicon/Makefile
>> index 65b0bd7..708326d0 100644
>> --- a/drivers/perf/pci/hisilicon/Makefile
>> +++ b/drivers/perf/pci/hisilicon/Makefile
>> @@ -1,3 +1,4 @@
>>   # SPDX-License-Identifier: GPL-2.0-only
>>   obj-$(CONFIG_HISI_PCIE_PMU) += hisi_pcie_pmu.o
>> +obj-$(CONFIG_HNS3_PMU) += hns3_pmu.o
>> diff --git a/drivers/perf/pci/hisilicon/hns3_pmu.c b/drivers/perf/pci/hisilicon/hns3_pmu.c
>> new file mode 100644
>> index 0000000..97b2a40
>> --- /dev/null
>> +++ b/drivers/perf/pci/hisilicon/hns3_pmu.c
>> @@ -0,0 +1,331 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * This driver adds support for HNS3 PMU iEP device. Related perf events are
>> + * bandwidth, latency, packet rate, interrupt rate etc.
>> + *
>> + * Copyright (C) 2021 HiSilicon Limited
>> + */
>> +#include <linux/bitfield.h>
>> +#include <linux/bitmap.h>
>> +#include <linux/bug.h>
>> +#include <linux/cpuhotplug.h>
>> +#include <linux/cpumask.h>
>> +#include <linux/delay.h>
>> +#include <linux/device.h>
>> +#include <linux/err.h>
>> +#include <linux/interrupt.h>
>> +#include <linux/iopoll.h>
>> +#include <linux/irq.h>
>> +#include <linux/kernel.h>
>> +#include <linux/list.h>
>> +#include <linux/module.h>
>> +#include <linux/pci.h>
>> +#include <linux/perf_event.h>
>> +#include <linux/smp.h>
>> +
>> +/* registers offset address */
>> +#define HNS3_PMU_REG_GLOBAL_CTRL        0x0000
>> +#define HNS3_PMU_REG_CLOCK_FREQ            0x0020
>> +#define HNS3_PMU_REG_BDF            0x0FE0
>> +#define HNS3_PMU_REG_VERSION            0x0FE4
>> +#define HNS3_PMU_REG_DEVICE_ID            0x0FE8
>> +
>> +#define HNS3_PMU_INVALID_CPU_ID            (-1)
>> +
>> +struct hns3_pmu {
>> +    struct hlist_node node;
>> +    struct pci_dev *pdev;
>> +    struct pmu pmu;
>> +    void __iomem *base;
>> +    int on_cpu;
>> +    u32 identifier;
>> +    u32 hw_clk_freq; /* hardware clock frequency of PMU*/
> miss a blank before "*/"

Ok, thanks.

>> +    /* maximum and minimun bdf allowed by PMU */
>> +    u16 bdf_min;
>> +    u16 bdf_max;
>> +};
>> +
>> +#define to_hns3_pmu(p)  (container_of((p), struct hns3_pmu, pmu))
>> +
>> +static ssize_t identifier_show(struct device *dev,
>> +                   struct device_attribute *attr, char *buf)
>> +{
>> +    struct hns3_pmu *hns3_pmu;
>> +
>> +    hns3_pmu = to_hns3_pmu(dev_get_drvdata(dev));
>> +
>> +    return sysfs_emit(buf, "0x%x\n", hns3_pmu->identifier);
>> +}
>> +static DEVICE_ATTR_RO(identifier);
>> +
>> +static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
>> +                char *buf)
>> +{
>> +    struct hns3_pmu *hns3_pmu = to_hns3_pmu(dev_get_drvdata(dev));
>> +
>> +    return sysfs_emit(buf, "%d\n", hns3_pmu->on_cpu);
>> +}
>> +static DEVICE_ATTR_RO(cpumask);
>> +
>> +static struct attribute *hns3_pmu_cpumask_attrs[] = {
>> +    &dev_attr_cpumask.attr,
>> +    NULL
>> +};
>> +
>> +static struct attribute_group hns3_pmu_cpumask_attr_group = {
>> +    .attrs = hns3_pmu_cpumask_attrs,
>> +};
>> +
>> +static struct attribute *hns3_pmu_identifier_attrs[] = {
>> +    &dev_attr_identifier.attr,
>> +    NULL
>> +};
>> +
>> +static struct attribute_group hns3_pmu_identifier_attr_group = {
>> +    .attrs = hns3_pmu_identifier_attrs,
>> +};
>> +
>> +static const struct attribute_group *hns3_pmu_attr_groups[] = {
>> +    &hns3_pmu_cpumask_attr_group,
>> +    &hns3_pmu_identifier_attr_group,
>> +    NULL
>> +};
>> +
>> +static int hns3_pmu_alloc_pmu(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
>> +{
>> +    u16 device_id;
>> +    char *name;
>> +    u32 val;
>> +    int ret;
>> +
>> +    hns3_pmu->base = pci_ioremap_bar(pdev, 2);
>> +    if (!hns3_pmu->base) {
>> +        pci_err(pdev, "ioremap failed for hns3_pmu resource\n");
>> +        ret = -ENOMEM;
>> +        goto err_ioremap_bar;
>> +    }
>> +
>> +    hns3_pmu->hw_clk_freq = readl(hns3_pmu->base + HNS3_PMU_REG_CLOCK_FREQ);
>> +
>> +    val = readl(hns3_pmu->base + HNS3_PMU_REG_BDF);
>> +    hns3_pmu->bdf_min = val & 0xffff;
>> +    hns3_pmu->bdf_max = val >> 16;
>> +
>> +    val = readl(hns3_pmu->base + HNS3_PMU_REG_DEVICE_ID);
>> +    device_id = val & 0xffff;
>> +    name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "hns3_pmu_%u", device_id);
>> +    if (!name) {
>> +        pci_err(pdev, "failed to allocate name for HNS3 PMU.\n");
>> +        ret = -ENOMEM;
>> +        goto err_alloc_dev_name;
>> +    }
>> +
>> +    hns3_pmu->pdev = pdev;
>> +    hns3_pmu->on_cpu = HNS3_PMU_INVALID_CPU_ID;
>> +    hns3_pmu->identifier = readl(hns3_pmu->base + HNS3_PMU_REG_VERSION);
>> +    hns3_pmu->pmu = (struct pmu) {
>> +        .name        = name,
>> +        .module        = THIS_MODULE,
>> +        .task_ctx_nr    = perf_invalid_context,
>> +        .attr_groups    = hns3_pmu_attr_groups,
>> +        .capabilities    = PERF_PMU_CAP_NO_EXCLUDE,
>> +    };
>> +
>> +    return 0;
>> +
>> +err_alloc_dev_name:
>> +    iounmap(hns3_pmu->base);
>> +err_ioremap_bar:
>> +    pci_release_regions(pdev);
>> +
>> +    return ret;
>> +}
>> +
>> +static int hns3_pmu_online_cpu(unsigned int cpu, struct hlist_node *node)
>> +{
>> +    struct hns3_pmu *hns3_pmu;
>> +
>> +    hns3_pmu = hlist_entry_safe(node, struct hns3_pmu, node);
>> +    if (!hns3_pmu)
>> +        return -ENODEV;
>> +
>> +    if (hns3_pmu->on_cpu == HNS3_PMU_INVALID_CPU_ID)
>> +        hns3_pmu->on_cpu = cpu;
>> +
>> +    return 0;
>> +}
>> +
>> +static int hns3_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node)
>> +{
>> +    struct hns3_pmu *hns3_pmu;
>> +    unsigned int target;
>> +
>> +    hns3_pmu = hlist_entry_safe(node, struct hns3_pmu, node);
>> +    if (!hns3_pmu)
>> +        return -ENODEV;
>> +
>> +    /* Nothing to do if this CPU doesn't own the PMU */
>> +    if (hns3_pmu->on_cpu != cpu)
>> +        return 0;
>> +
>> +    /* Choose a new CPU from all online cpus */
>> +    target = cpumask_any_but(cpu_online_mask, cpu);
>> +    if (target >= nr_cpu_ids)
>> +        return 0;
>> +
>> +    perf_pmu_migrate_context(&hns3_pmu->pmu, cpu, target);
>> +    hns3_pmu->on_cpu = target;
>> +
>> +    return 0;
>> +}
>> +
>> +static int hns3_pmu_init(struct pci_dev *pdev, struct hns3_pmu *hns3_pmu)
>> +{
>> +    int ret;
>> +
>> +    ret = hns3_pmu_alloc_pmu(pdev, hns3_pmu);
>> +    if (ret)
>> +        return ret;
>> +
>> +    ret = cpuhp_state_add_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
>> +                       &hns3_pmu->node);
>> +    if (ret) {
>> +        pci_err(pdev, "failed to register hotplug, ret = %d.\n", ret);
>> +        goto err_register_hotplug;
>> +    }
>> +
>> +    ret = perf_pmu_register(&hns3_pmu->pmu, hns3_pmu->pmu.name,
>> +                HNS3_PMU_INVALID_CPU_ID);
>> +    if (ret) {
>> +        pci_err(pdev, "failed to register HNS3 PMU, ret = %d.\n", ret);
>> +        goto err_register_pmu;
>> +    }
>> +
>> +    return ret;
>> +
>> +err_register_pmu:
>> +    cpuhp_state_remove_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
>> +                    &hns3_pmu->node);
>> +
>> +err_register_hotplug:
>> +    iounmap(hns3_pmu->base);
>> +
>> +    return ret;
>> +}
>> +
>> +static void hns3_pmu_uninit(struct pci_dev *pdev)
>> +{
>> +    struct hns3_pmu *hns3_pmu = (struct hns3_pmu *)pci_get_drvdata(pdev);
>> +
>> +    perf_pmu_unregister(&hns3_pmu->pmu);
>> +    cpuhp_state_remove_instance(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
>> +                    &hns3_pmu->node);
>> +    iounmap(hns3_pmu->base);
>> +}
>> +
>> +static int hns3_pmu_init_dev(struct pci_dev *pdev)
>> +{
>> +    int ret;
>> +
>> +    ret = pci_enable_device(pdev);
>> +    if (ret) {
>> +        pci_err(pdev, "failed to enable pci device, ret = %d.\n", ret);
>> +        return ret;
>> +    }
>> +
>> +    ret = pci_request_mem_regions(pdev, "hns3_pmu");
>> +    if (ret < 0) {
>> +        pci_err(pdev, "failed to request pci mem regions, ret = %d.\n",
>> +            ret);
>> +        pci_disable_device(pdev);
>> +        return ret;
>> +    }
>> +
>> +    pci_set_master(pdev);
>> +
>> +    return 0;
>> +}
>> +
>> +static void hns3_pmu_uninit_dev(struct pci_dev *pdev)
>> +{
>> +    pci_clear_master(pdev);
>> +    pci_release_mem_regions(pdev);
>> +    pci_disable_device(pdev);
>> +}
>> +
>> +static int hns3_pmu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>> +{
>> +    struct hns3_pmu *hns3_pmu;
>> +    int ret;
>> +
>> +    hns3_pmu = devm_kzalloc(&pdev->dev, sizeof(*hns3_pmu), GFP_KERNEL);
>> +    if (!hns3_pmu)
>> +        return -ENOMEM;
>> +
>> +    ret = hns3_pmu_init_dev(pdev);
>> +    if (ret)
>> +        return ret;
>> +
>> +    ret = hns3_pmu_init(pdev, hns3_pmu);
>> +    if (ret) {
>> +        hns3_pmu_uninit_dev(pdev);
>> +        return ret;
>> +    }
>> +
>> +    pci_set_drvdata(pdev, hns3_pmu);
>> +
>> +    return ret;
>> +}
>> +
>> +static void hns3_pmu_remove(struct pci_dev *pdev)
>> +{
>> +    hns3_pmu_uninit(pdev);
>> +    hns3_pmu_uninit_dev(pdev);
>> +    pci_set_drvdata(pdev, NULL);
>> +}
>> +
>> +static const struct pci_device_id hns3_pmu_ids[] = {
>> +    { PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, 0xA22B) },
>> +    { 0, }
>> +};
>> +MODULE_DEVICE_TABLE(pci, hns3_pmu_ids);
>> +
>> +static struct pci_driver hns3_pmu_driver = {
>> +    .name = "hns3_pmu",
>> +    .id_table = hns3_pmu_ids,
>> +    .probe = hns3_pmu_probe,
>> +    .remove = hns3_pmu_remove,
>> +};
>> +
>> +static int __init hns3_pmu_module_init(void)
>> +{
>> +    int ret;
>> +
>> +    ret = cpuhp_setup_state_multi(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
>> +                      "AP_PERF_ARM_HNS3_PMU_ONLINE",
>> +                      hns3_pmu_online_cpu,
>> +                      hns3_pmu_offline_cpu);
>> +    if (ret) {
>> +        pr_err("failed to setup HNS3 PMU hotplug, ret = %d.\n", ret);
>> +        return ret;
>> +    }
>> +
>> +    ret = pci_register_driver(&hns3_pmu_driver);
>> +    if (ret) {
>> +        pr_err("failed to register pci driver, ret = %d.\n", ret);
>> +        cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE);
>> +    }
>> +
>> +    return ret;
>> +}
>> +module_init(hns3_pmu_module_init);
>> +
>> +static void __exit hns3_pmu_module_exit(void)
>> +{
>> +    pci_unregister_driver(&hns3_pmu_driver);
>> +    cpuhp_remove_multi_state(CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE);
>> +}
>> +module_exit(hns3_pmu_module_exit);
>> +
>> +MODULE_DESCRIPTION("HNS3 PMU driver");
>> +MODULE_LICENSE("GPL v2");
>> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
>> index 6a765cb..e68bed9 100644
>> --- a/include/linux/cpuhotplug.h
>> +++ b/include/linux/cpuhotplug.h
>> @@ -176,6 +176,7 @@ enum cpuhp_state {
>>       CPUHP_AP_PERF_ARM_HISI_HHA_ONLINE,
>>       CPUHP_AP_PERF_ARM_HISI_L3_ONLINE,
>>       CPUHP_AP_PERF_ARM_HISI_PCIE_PMU_ONLINE,
>> +    CPUHP_AP_PERF_ARM_HNS3_PMU_ONLINE,
>>       CPUHP_AP_PERF_ARM_L2X0_ONLINE,
>>       CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
>>       CPUHP_AP_PERF_ARM_QCOM_L3_ONLINE,
> 
> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel