From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zhi Li Subject: Re: [PATCH V5 2/4] drivers/perf: imx_ddr: Add ddr performance counter support Date: Wed, 24 Apr 2019 14:13:23 -0500 Message-ID: References: <1555447156-28306-1-git-send-email-Frank.Li@nxp.com> <1555447156-28306-2-git-send-email-Frank.Li@nxp.com> <20190424105822.GA14614@fuggles.cambridge.arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20190424105822.GA14614@fuggles.cambridge.arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org To: Will Deacon Cc: "mark.rutland@arm.com" , Aisheng Dong , "devicetree@vger.kernel.org" , "festevam@gmail.com" , "s.hauer@pengutronix.de" , Frank Li , "robh+dt@kernel.org" , dl-linux-imx , "kernel@pengutronix.de" , "shawnguo@kernel.org" , "linux-arm-kernel@lists.infradead.org" List-Id: devicetree@vger.kernel.org On Wed, Apr 24, 2019 at 5:58 AM Will Deacon wrote: > > Hi Frank, > > On Tue, Apr 16, 2019 at 08:39:33PM +0000, Frank Li wrote: > > Add ddr performance monitor support for iMX8QXP > > > > There are 4 counters for ddr perfomance events. > > counter 0 is dedicated for cycles. > > you choose any up to 3 no cycles events. > > I was about to pick this up, but I still have a few questions/comments > on the code: > > > +static irqreturn_t ddr_perf_irq_handler(int irq, void *p) > > +{ > > + int i; > > + u8 reg; > > + int val; > > + int counter; > > + struct ddr_pmu *pmu = (struct ddr_pmu *) p; > > + struct perf_event *event; > > + > > + /* Only cycles counter overflowed can issue irq. all counters will > > + * be stopped when cycles counter overflow. but other counter don't stop > > + * when overflow happen. Update all of the local counter values, > > + * then reset the cycles counter, so the others can continue > > + * counting. cycles counter is fastest counter in all events. at last > > + * 4 times than other counters. > > + */ > > + for (i = 0; i < NUM_COUNTER; i++) { > > + > > + if (!pmu->active_events[i]) > > + continue; > > + > > + event = pmu->active_events[i]; > > + counter = event->hw.idx; > > + reg = counter * 4 + COUNTER_CNTL; > > + val = readl(pmu->base + reg); > > Does this read have a side-effect, or can it be removed given that you don't > use val for anything else? It can be removed. > > > + ddr_perf_event_update(event); > > + > > + if (counter == EVENT_CYCLES_COUNTER) { > > + ddr_perf_event_enable(pmu, > > + EVENT_CYCLES_ID, > > + EVENT_CYCLES_COUNTER, > > + true); > > + ddr_perf_event_update(event); > > + } > > + } > > + > > + return IRQ_HANDLED; > > +} > > What stops the IRQ handler running concurrently with perf callbacks? I can't > see any locking here, or is the IRQ supposed to be affine to the same CPU > that's handling the perf context? I don't under your means here. ddr_perf_event_update is lockless design. I can't found problem if run ddr_perf_event_enable at both perf context and IRQ concurrently. > > > +static int ddr_perf_offline_cpu(unsigned int cpu, struct hlist_node *node) > > +{ > > + struct ddr_pmu *pmu = hlist_entry_safe(node, struct ddr_pmu, node); > > + int target; > > + > > + if (!cpumask_test_and_clear_cpu(cpu, &pmu->cpu)) > > + return 0; > > + > > + target = cpumask_any_but(cpu_online_mask, cpu); > > + if (target >= nr_cpu_ids) > > + return 0; > > + > > + perf_pmu_migrate_context(&pmu->pmu, cpu, target); > > + cpumask_set_cpu(target, &pmu->cpu); > > + > > + return 0; > > +} > > + > > +static int ddr_perf_probe(struct platform_device *pdev) > > +{ > > + struct ddr_pmu *pmu; > > + struct device_node *np; > > + void __iomem *base; > > + struct resource *iomem; > > + char *name; > > + char *hpname; > > + int num; > > + int ret; > > + u32 irq; > > + const struct of_device_id *of_id = > > + of_match_device(imx_ddr_pmu_dt_ids, &pdev->dev); > > + > > + iomem = platform_get_resource(pdev, IORESOURCE_MEM, 0); > > + base = devm_ioremap_resource(&pdev->dev, iomem); > > + if (IS_ERR(base)) > > + return PTR_ERR(base); > > + > > + np = pdev->dev.of_node; > > + > > + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); > > + if (!pmu) > > + return -ENOMEM; > > + > > + num = ddr_perf_init(pmu, base, &pdev->dev); > > + > > + name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "ddr%d", num); > > + if (!name) > > + return -ENOMEM; > > + > > + hpname = devm_kasprintf(&pdev->dev, GFP_KERNEL, > > + "perf/imx/ddr%d:online", num); > > + if (!hpname) > > + return -ENOMEM; > > + > > + pmu->flags = (uintptr_t) of_id->data; > > + > > + cpumask_set_cpu(raw_smp_processor_id(), &pmu->cpu); > > + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, hpname, NULL, > > + ddr_perf_offline_cpu); > > + > > + if (ret < 0) { > > + dev_err(&pdev->dev, "cpuhp_setup_state_multi failed\n"); > > + goto ddr_perf_err; > > + } > > + > > + pmu->cpuhp_state = ret; > > + > > + /* Register the pmu instance for cpu hotplug */ > > + cpuhp_state_add_instance_nocalls(pmu->cpuhp_state, &pmu->node); > > + > > + ret = perf_pmu_register(&(pmu->pmu), name, -1); > > + if (ret) > > + goto ddr_perf_err; > > + > > + /* Request irq */ > > + irq = of_irq_get(np, 0); > > + if (irq < 0) { > > irq is a u32 so this will never execute. You need to make it an int. > > > + dev_err(&pdev->dev, "Failed to get irq: %d", irq); > > + ret = irq; > > + goto ddr_perf_irq_err; > > + } > > + > > + ret = devm_request_irq(&pdev->dev, irq, > > + ddr_perf_irq_handler, > > + IRQF_TRIGGER_RISING | IRQF_ONESHOT, > > Can you explain why you need to pass these specific IRQ flags, please? I'm > struggling to see what the ONESHOT is buying you. I think default should be okay. I will update at next version > > Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A41BAC10F11 for ; Wed, 24 Apr 2019 19:13:45 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73292205ED for ; Wed, 24 Apr 2019 19:13:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JxjperCu"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dQI9XtCL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73292205ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZZSyLz0qCyofkX3CdSw5+/h6Z5sdMatmEHlCIf1uUKQ=; b=JxjperCu8ewpxB MucLvRLEggrlMaIhE1xJxUAj8Bcyno983pAsh1bLI6t3eh0F8LVRVKmcs8H532qNoS8RFAIxYT8bv Xb01cqYp+BhM6GgS6oqGiT69t8EqVxS9cdkDlTLxYwrbz/+dFDcPbtnKi6H/LWP3F1z7MrcDTDhXE iJW9vAxWOuokyRwp+ulxFFb3Gx/Arjo/LxuMLM9iFCSV5O/hZjMaFD2aTlEtZPNOm2SdV4OaIvaQ1 CqhDuVxNj16wD9nDpvM1bIpFkpqPtSRaJAtoWQtH6OrWk/9zCS6hemyXYC72WtEVciMTNOK2+QaEs KNZ2n5bFTbZgwWb7KAVQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hJNKs-0008Jl-1H; Wed, 24 Apr 2019 19:13:38 +0000 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1hJNKp-0008JQ-LR for linux-arm-kernel@lists.infradead.org; Wed, 24 Apr 2019 19:13:37 +0000 Received: by mail-pf1-x442.google.com with SMTP id e24so9776499pfi.12 for ; Wed, 24 Apr 2019 12:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8WWA9TvmT5JZyrgtx2M7YMkyXEZYigTZU2OGVx2T5KQ=; b=dQI9XtCLtqpFzSW6AVoqw11XKq0hUGRWkBbKrs5nYkC359+UHF+cTwJ18pMySuJ6TE TUlVjoPnYXtODQo1Usq2DmSx8z5jfh8icWzJ6MSqfnh9cx7l+ORJDzNDhm2Wm/P1N2Fp uI8/b69I/DCxFp9O2KhxqutMayMtgtyGCUyvuT1x9BbO1pXrVNsEA2fFbdX+y9UkoJaS PAfMrQDVW95SHyIC5ccPwOQBMgH+XnBDKbIzMeXFaQg22qDyBAGrauCB/cYvFOlMiItc yDd8ffkoGhuhg1l5b5iqBkJhYFTTQ2d6v23XjZtLe9qxyw0P/YR18/W7ZaVS5lbhkmkJ lcBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8WWA9TvmT5JZyrgtx2M7YMkyXEZYigTZU2OGVx2T5KQ=; b=ZvPk+zUkamm0Y+/fEHx/YiT84GmijwAAAP9ZsaDxKxrphoCg/Bz16TvQDQeLbiqOxT oQtlJBWY76Er65EzMcVs88hzWNurPq4hY3Nn6ha2pae/13FhzHQ8/KTFRKJUAMhbsmw2 IeJND6K30emJYPeWz6ivPClxlB37YCgkXrihRqLxS6+K6g7a2huppubUkJxNbpivoJna KIQNIciAOXvmdv2O8OGkcjvOlPYCxl0SsMdTrw9sJ9iQ8hiuWaTD1CGM7ivuRUCbPGv6 RC8mhag5+s8XJcxX40typi/bugQLlwR2hRfUcJNyGBpZ19JegT/4GCQQjgxRGuK5BcDB r9yQ== X-Gm-Message-State: APjAAAX6eFXwGHJEnV51acfePAYCUz3X8q9RK2OFZdb9YNfJ2lSyNvy8 h0d52YkdtZmuufy+5yS7HLx1B1UE0ZjSOlA6Jz1JvR8dFOo= X-Google-Smtp-Source: APXvYqwJBQH1b16hqUla2qPcjYCkegyigHN6RnkolZogA8IyRemISTSMsvjA3tfoZiIA+4BdmcIPLzM5ainGgSaJqBI= X-Received: by 2002:a63:5b24:: with SMTP id p36mr31260649pgb.84.1556133214332; Wed, 24 Apr 2019 12:13:34 -0700 (PDT) MIME-Version: 1.0 References: <1555447156-28306-1-git-send-email-Frank.Li@nxp.com> <1555447156-28306-2-git-send-email-Frank.Li@nxp.com> <20190424105822.GA14614@fuggles.cambridge.arm.com> In-Reply-To: <20190424105822.GA14614@fuggles.cambridge.arm.com> From: Zhi Li Date: Wed, 24 Apr 2019 14:13:23 -0500 Message-ID: Subject: Re: [PATCH V5 2/4] drivers/perf: imx_ddr: Add ddr performance counter support To: Will Deacon X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190424_121335_708346_C3776C67 X-CRM114-Status: GOOD ( 24.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "mark.rutland@arm.com" , Aisheng Dong , "devicetree@vger.kernel.org" , "festevam@gmail.com" , "s.hauer@pengutronix.de" , Frank Li , "robh+dt@kernel.org" , dl-linux-imx , "kernel@pengutronix.de" , "shawnguo@kernel.org" , "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Apr 24, 2019 at 5:58 AM Will Deacon wrote: > > Hi Frank, > > On Tue, Apr 16, 2019 at 08:39:33PM +0000, Frank Li wrote: > > Add ddr performance monitor support for iMX8QXP > > > > There are 4 counters for ddr perfomance events. > > counter 0 is dedicated for cycles. > > you choose any up to 3 no cycles events. > > I was about to pick this up, but I still have a few questions/comments > on the code: > > > +static irqreturn_t ddr_perf_irq_handler(int irq, void *p) > > +{ > > + int i; > > + u8 reg; > > + int val; > > + int counter; > > + struct ddr_pmu *pmu = (struct ddr_pmu *) p; > > + struct perf_event *event; > > + > > + /* Only cycles counter overflowed can issue irq. all counters will > > + * be stopped when cycles counter overflow. but other counter don't stop > > + * when overflow happen. Update all of the local counter values, > > + * then reset the cycles counter, so the others can continue > > + * counting. cycles counter is fastest counter in all events. at last > > + * 4 times than other counters. > > + */ > > + for (i = 0; i < NUM_COUNTER; i++) { > > + > > + if (!pmu->active_events[i]) > > + continue; > > + > > + event = pmu->active_events[i]; > > + counter = event->hw.idx; > > + reg = counter * 4 + COUNTER_CNTL; > > + val = readl(pmu->base + reg); > > Does this read have a side-effect, or can it be removed given that you don't > use val for anything else? It can be removed. > > > + ddr_perf_event_update(event); > > + > > + if (counter == EVENT_CYCLES_COUNTER) { > > + ddr_perf_event_enable(pmu, > > + EVENT_CYCLES_ID, > > + EVENT_CYCLES_COUNTER, > > + true); > > + ddr_perf_event_update(event); > > + } > > + } > > + > > + return IRQ_HANDLED; > > +} > > What stops the IRQ handler running concurrently with perf callbacks? I can't > see any locking here, or is the IRQ supposed to be affine to the same CPU > that's handling the perf context? I don't under your means here. ddr_perf_event_update is lockless design. I can't found problem if run ddr_perf_event_enable at both perf context and IRQ concurrently. > > > +static int ddr_perf_offline_cpu(unsigned int cpu, struct hlist_node *node) > > +{ > > + struct ddr_pmu *pmu = hlist_entry_safe(node, struct ddr_pmu, node); > > + int target; > > + > > + if (!cpumask_test_and_clear_cpu(cpu, &pmu->cpu)) > > + return 0; > > + > > + target = cpumask_any_but(cpu_online_mask, cpu); > > + if (target >= nr_cpu_ids) > > + return 0; > > + > > + perf_pmu_migrate_context(&pmu->pmu, cpu, target); > > + cpumask_set_cpu(target, &pmu->cpu); > > + > > + return 0; > > +} > > + > > +static int ddr_perf_probe(struct platform_device *pdev) > > +{ > > + struct ddr_pmu *pmu; > > + struct device_node *np; > > + void __iomem *base; > > + struct resource *iomem; > > + char *name; > > + char *hpname; > > + int num; > > + int ret; > > + u32 irq; > > + const struct of_device_id *of_id = > > + of_match_device(imx_ddr_pmu_dt_ids, &pdev->dev); > > + > > + iomem = platform_get_resource(pdev, IORESOURCE_MEM, 0); > > + base = devm_ioremap_resource(&pdev->dev, iomem); > > + if (IS_ERR(base)) > > + return PTR_ERR(base); > > + > > + np = pdev->dev.of_node; > > + > > + pmu = devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); > > + if (!pmu) > > + return -ENOMEM; > > + > > + num = ddr_perf_init(pmu, base, &pdev->dev); > > + > > + name = devm_kasprintf(&pdev->dev, GFP_KERNEL, "ddr%d", num); > > + if (!name) > > + return -ENOMEM; > > + > > + hpname = devm_kasprintf(&pdev->dev, GFP_KERNEL, > > + "perf/imx/ddr%d:online", num); > > + if (!hpname) > > + return -ENOMEM; > > + > > + pmu->flags = (uintptr_t) of_id->data; > > + > > + cpumask_set_cpu(raw_smp_processor_id(), &pmu->cpu); > > + ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, hpname, NULL, > > + ddr_perf_offline_cpu); > > + > > + if (ret < 0) { > > + dev_err(&pdev->dev, "cpuhp_setup_state_multi failed\n"); > > + goto ddr_perf_err; > > + } > > + > > + pmu->cpuhp_state = ret; > > + > > + /* Register the pmu instance for cpu hotplug */ > > + cpuhp_state_add_instance_nocalls(pmu->cpuhp_state, &pmu->node); > > + > > + ret = perf_pmu_register(&(pmu->pmu), name, -1); > > + if (ret) > > + goto ddr_perf_err; > > + > > + /* Request irq */ > > + irq = of_irq_get(np, 0); > > + if (irq < 0) { > > irq is a u32 so this will never execute. You need to make it an int. > > > + dev_err(&pdev->dev, "Failed to get irq: %d", irq); > > + ret = irq; > > + goto ddr_perf_irq_err; > > + } > > + > > + ret = devm_request_irq(&pdev->dev, irq, > > + ddr_perf_irq_handler, > > + IRQF_TRIGGER_RISING | IRQF_ONESHOT, > > Can you explain why you need to pass these specific IRQ flags, please? I'm > struggling to see what the ONESHOT is buying you. I think default should be okay. I will update at next version > > Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel