From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C43F5C433E0 for ; Sat, 18 Jul 2020 03:27:40 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 79C71206BE for ; Sat, 18 Jul 2020 03:27:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="IPencFEM"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=mg.codeaurora.org header.i=@mg.codeaurora.org header.b="qRCl2liJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79C71206BE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:References:In-Reply-To:Subject:To:From: Date:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JOfsCQAOqBV7apAXKde5eRmptpxNm77wwK09GCvYqmA=; b=IPencFEMZUgq8zrGwVH9BJWsY Cl2ZyBmIfTFbbqsPkUxBUBU+Rhq4gOJoVwYlLPXNSHr+0oL4wG+l5pmdWJCMgDITCj9U0MZ3mgHu6 FS3MyjPMApoD7k2J2lXlCExf/yDr6DIw9k27AtKh8lCyUsbkXDhjyktuH23ZRshR65sRGBslDqR8I +Tc0IT8XCpf8i28dznlf7YjAsSi9Zrp6MbPT3TIpaaBT/pTWCpXUJGnrGIjO/aY70k6g93t0LnF4A Oho52RsF4qqUAdM9MI4HF/p2/9YvCM91B0fXQ0SGTNyC75Zk1HLWpZzWL9+PCJ9TDeNWfiUkYkSYg 82dSEwosA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwdUG-0002A4-Ef; Sat, 18 Jul 2020 03:26:08 +0000 Received: from m43-7.mailgun.net ([69.72.43.7]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwdUA-00029L-7g for linux-arm-kernel@lists.infradead.org; Sat, 18 Jul 2020 03:26:06 +0000 DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=mg.codeaurora.org; q=dns/txt; s=smtp; t=1595042764; h=Message-ID: References: In-Reply-To: Subject: Cc: To: From: Date: Content-Transfer-Encoding: Content-Type: MIME-Version: Sender; bh=C527DAcO7cltMLJFfivyRxsbfT1CjmqoZ9gYUHyhur0=; b=qRCl2liJSvHWrApzWUTedk5Unp9maIb/h0ahqe3rj+y19io/fs4D9KszaqJefjZEP830K7E0 RVr3MXv7k+ByCGtN44IyRor2Em6Ba4sKyilRlf51HUcH8+K0EpdECqkHZZG3AFxKRlQXsnWB 2rtvvo4qunEqNZ+mBhQTWlgpr44= X-Mailgun-Sending-Ip: 69.72.43.7 X-Mailgun-Sid: WyJiYzAxZiIsICJsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmciLCAiYmU5ZTRhIl0= Received: from smtp.codeaurora.org (ec2-35-166-182-171.us-west-2.compute.amazonaws.com [35.166.182.171]) by smtp-out-n20.prod.us-west-2.postgun.com with SMTP id 5f126bb9166c1c54943fb81b (version=TLS1.2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256); Sat, 18 Jul 2020 03:25:45 GMT Received: by smtp.codeaurora.org (Postfix, from userid 1001) id DDEC9C433A0; Sat, 18 Jul 2020 03:25:44 +0000 (UTC) Received: from mail.codeaurora.org (localhost.localdomain [127.0.0.1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: tingwei) by smtp.codeaurora.org (Postfix) with ESMTPSA id BD307C433C9; Sat, 18 Jul 2020 03:25:42 +0000 (UTC) MIME-Version: 1.0 Date: Sat, 18 Jul 2020 11:25:42 +0800 From: tingwei@codeaurora.org To: Mike Leach Subject: Re: [PATCH v3 08/20] coresight: allow etm4x to be built as a module In-Reply-To: References: <20200717054536.7052-1-tingwei@codeaurora.org> <20200717054536.7052-9-tingwei@codeaurora.org> Message-ID: X-Sender: tingwei@codeaurora.org User-Agent: Roundcube Webmail/1.3.9 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200717_232604_746743_12832438 X-CRM114-Status: GOOD ( 31.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: tsoni@codeaurora.org, Sai Prakash Ranjan , Kim Phillips , Mathieu Poirier , Suzuki K Poulose , Alexander Shishkin , Greg Kroah-Hartman , Coresight ML , Mao Jinlong , Mian Yousaf Kaukab , Russell King , Randy Dunlap , Leo Yan , linux-arm-kernel , linux-arm-kernel Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Mike, Thanks for reporting this. This is a good catch. I was testing on db845c but I didn't encounter this issue with multiple module load/unload test. I guess it's due to timing difference. The issue is etm4_cpu_pm_notify() is removed from cpu pm notification after last etm4 device is removed. There's racing condition when some etm4 device say etm0 is removed but etm4_cpu_pm_notify() is still called. I reproduced this issue by manually unbind etm4x device. The fix is clean etmdrvdata[drvdata->cpu] in etm4_remove() so etm4_cpu_pm_notify() is gracefully returned. I've verified this on db845c with manually unbind etm4x device test. Do you mind test on your db410 as well? I'll put it into v4. @@ -1542,6 +1543,7 @@ static int __exit etm4_remove(struct amba_device *adev) etm_perf_symlink(drvdata->csdev, false); + etmdrvdata[drvdata->cpu] = NULL; if (--etm4_count == 0) { etm4_cpu_pm_unregister(); On 2020-07-18 01:05, Mike Leach wrote: > Hi Tingwei, > > Couldn't see 00/20 cover note for this set, so reporting this here as > it relates to etmv4 in part. > > The following sequence:- > (There is a load dependency here of course, if I try to load ETMv4 > without the coresight core, then I get a lot of missing symbol error > messages - I assume this is expected behaviour?) Coresight_etm4x module depends on coresight module, so it's expected behavior. > > root@linaro-developer:/home/linaro/cs-mods# insmod coresight.ko > root@linaro-developer:/home/linaro/cs-mods# insmod coresight-etm4x.ko > > correctly loads the coresight core then ETMv4 module. > > [ 1208.214674] coresight etm0: CPU0: ETM v4.0 initialized > [ 1208.215534] coresight etm1: CPU1: ETM v4.0 initialized > [ 1208.221020] coresight etm2: CPU2: ETM v4.0 initialized > [ 1208.224757] coresight etm3: CPU3: ETM v4.0 initialized > > However, if I then unload the ETMv4 module:- > > root@linaro-developer:/home/linaro/cs-mods# rmmod coresight-etm4x.ko > > I get a crash:- > > [ 1215.963689] ------------[ cut here ]------------ > [ 1215.963741] WARNING: CPU: 3 PID: 0 at > drivers/hwtracing/coresight/coresight-etm4x-core.c:1364 > etm4_cpu_pm_notify+0xad8/0xb48 [coresight_etm4x] > [ 1215.967373] Modules linked in: coresight_etm4x(-) coresight [last > unloaded: coresight] > [ 1215.979960] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G W > 5.8.0-rc5cs-modscs-mods-00020-gc03fe910680d #282 > [ 1215.987856] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC > (DT) > [ 1215.998531] pstate: 80000085 (Nzcv daIf -PAN -UAO BTYPE=--) > [ 1216.005396] pc : etm4_cpu_pm_notify+0xad8/0xb48 [coresight_etm4x] > [ 1216.010687] lr : notifier_call_chain+0x5c/0xa0 > [ 1216.016926] sp : ffff000009fe7d50 > [ 1216.021265] x29: ffff000009fe7d50 x28: 0000000000000000 > [ 1216.024653] x27: 0000011b1d1464b5 x26: 0000000000000001 > [ 1216.030035] x25: ffff00003c79f888 x24: 0000000000000001 > [ 1216.035330] x23: 0000000000000000 x22: 0000000000000000 > [ 1216.040626] x21: 0000000000000000 x20: 0000000000000000 > [ 1216.045921] x19: ffff000038396480 x18: 0000000000000000 > [ 1216.051216] x17: 0000000000000000 x16: 0000000000000000 > [ 1216.056510] x15: 0000000000000000 x14: ffffffffffffffff > [ 1216.061806] x13: ffffffffffffffff x12: 0000000000000000 > [ 1216.067101] x11: 0000000000000000 x10: 00000000000009b0 > [ 1216.072396] x9 : ffff000009fe7e70 x8 : ffff000009fd4010 > [ 1216.077690] x7 : 0000000000000000 x6 : 0000000023c2c562 > [ 1216.082986] x5 : 00ffffffffffffff x4 : 0000000000000003 > [ 1216.088281] x3 : ffff000038396880 x2 : ffff800008d8d280 > [ 1216.093576] x1 : ffff80002b1aa000 x0 : 000000005f454c42 > [ 1216.098873] Call trace: > [ 1216.104178] etm4_cpu_pm_notify+0xad8/0xb48 [coresight_etm4x] > [ 1216.106343] notifier_call_chain+0x5c/0xa0 > [ 1216.112242] __atomic_notifier_call_chain+0x48/0x60 > [ 1216.116236] cpu_pm_notify+0x44/0x70 > [ 1216.121008] cpu_pm_enter+0x3c/0x80 > [ 1216.124829] psci_enter_domain_idle_state+0x38/0xa8 > [ 1216.128040] cpuidle_enter_state+0x88/0x478 > [ 1216.132900] cpuidle_enter+0x44/0x58 > [ 1216.137069] call_cpuidle+0x40/0x70 > [ 1216.140888] do_idle+0x1e0/0x248 > [ 1216.144099] cpu_startup_entry+0x28/0x98 > [ 1216.147574] secondary_start_kernel+0x1a0/0x1f8 > [ 1216.151476] ---[ end trace 97bcd7b8bdd7f9a7 ]--- > > This happens each time, though the CPU number can change. > Test platform is Dragonboard DB410, linux 5.8-rc5 + this patch set. > > Regards > > Mike > > > On Fri, 17 Jul 2020 at 06:48, Tingwei Zhang > wrote: >> >> From: Kim Phillips >> >> Allow to build coresight-etm4x as a module, for ease of development. >> >> - Kconfig becomes a tristate, to allow =m >> - append -core to source file name to allow module to >> be called coresight-etm4x by the Makefile >> - add an etm4_remove function, for module unload >> - add a MODULE_DEVICE_TABLE for autoloading on boot >> >> Cc: Mathieu Poirier >> Cc: Leo Yan >> Cc: Alexander Shishkin >> Cc: Randy Dunlap >> Cc: Suzuki K Poulose >> Cc: Greg Kroah-Hartman >> Cc: Russell King >> Signed-off-by: Kim Phillips >> Signed-off-by: Tingwei Zhang >> --- >> drivers/hwtracing/coresight/Kconfig | 5 ++- >> drivers/hwtracing/coresight/Makefile | 4 +-- >> ...resight-etm4x.c => coresight-etm4x-core.c} | 31 >> ++++++++++++++++++- >> 3 files changed, 36 insertions(+), 4 deletions(-) >> rename drivers/hwtracing/coresight/{coresight-etm4x.c => > coresight-etm4x-core.c} (98%) >> >> diff --git a/drivers/hwtracing/coresight/Kconfig > b/drivers/hwtracing/coresight/Kconfig >> index 8fd9fd139cf3..d6e107bbd30b 100644 >> --- a/drivers/hwtracing/coresight/Kconfig >> +++ b/drivers/hwtracing/coresight/Kconfig >> @@ -78,7 +78,7 @@ config CORESIGHT_SOURCE_ETM3X >> module will be called coresight-etm3x. >> >> config CORESIGHT_SOURCE_ETM4X >> - bool "CoreSight Embedded Trace Macrocell 4.x driver" >> + tristate "CoreSight Embedded Trace Macrocell 4.x driver" >> depends on ARM64 >> select CORESIGHT_LINKS_AND_SINKS >> select PID_IN_CONTEXTIDR >> @@ -88,6 +88,9 @@ config CORESIGHT_SOURCE_ETM4X >> for instruction level tracing. Depending on the implemented > version >> data tracing may also be available. >> >> + To compile this driver as a module, choose M here: the >> + module will be called coresight-etm4x. >> + >> config CORESIGHT_STM >> tristate "CoreSight System Trace Macrocell driver" >> depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || > ARM64 >> diff --git a/drivers/hwtracing/coresight/Makefile > b/drivers/hwtracing/coresight/Makefile >> index d619cfd0abd8..271dc255454f 100644 >> --- a/drivers/hwtracing/coresight/Makefile >> +++ b/drivers/hwtracing/coresight/Makefile >> @@ -14,8 +14,8 @@ obj-$(CONFIG_CORESIGHT_LINKS_AND_SINKS) += > coresight-funnel.o \ >> obj-$(CONFIG_CORESIGHT_SOURCE_ETM3X) += coresight-etm3x.o >> coresight-etm3x-y := coresight-etm3x-core.o coresight-etm-cp14.o \ >> coresight-etm3x-sysfs.o >> -obj-$(CONFIG_CORESIGHT_SOURCE_ETM4X) += coresight-etm4x.o \ >> - coresight-etm4x-sysfs.o >> +obj-$(CONFIG_CORESIGHT_SOURCE_ETM4X) += coresight-etm4x.o >> +coresight-etm4x-y := coresight-etm4x-core.o coresight-etm4x-sysfs.o >> obj-$(CONFIG_CORESIGHT_STM) += coresight-stm.o >> obj-$(CONFIG_CORESIGHT_CPU_DEBUG) += coresight-cpu-debug.o >> obj-$(CONFIG_CORESIGHT_CATU) += coresight-catu.o >> diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c > b/drivers/hwtracing/coresight/coresight-etm4x-core.c >> similarity index 98% >> rename from drivers/hwtracing/coresight/coresight-etm4x.c >> rename to drivers/hwtracing/coresight/coresight-etm4x-core.c >> index 747afc875f91..b5945f62794c 100644 >> --- a/drivers/hwtracing/coresight/coresight-etm4x.c >> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c >> @@ -1536,6 +1536,26 @@ static struct amba_cs_uci_id uci_id_etm4[] = { >> } >> }; >> >> +static int __exit etm4_remove(struct amba_device *adev) >> +{ >> + struct etmv4_drvdata *drvdata = dev_get_drvdata(&adev->dev); >> + >> + etm_perf_symlink(drvdata->csdev, false); >> + >> + if (--etm4_count == 0) { >> + etm4_cpu_pm_unregister(); >> + >> + > cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); >> + if (hp_online) >> + cpuhp_remove_state_nocalls(hp_online); >> + } >> + >> + coresight_unregister(drvdata->csdev); >> + >> + return 0; >> +} >> + >> + >> static const struct amba_id etm4_ids[] = { >> CS_AMBA_ID(0x000bb95d), /* Cortex-A53 */ >> CS_AMBA_ID(0x000bb95e), /* Cortex-A57 */ >> @@ -1553,12 +1573,21 @@ static const struct amba_id etm4_ids[] = { >> {}, >> }; >> >> +MODULE_DEVICE_TABLE(amba, etm4_ids); >> + >> static struct amba_driver etm4x_driver = { >> .drv = { >> .name = "coresight-etm4x", >> + .owner = THIS_MODULE, >> .suppress_bind_attrs = true, >> }, >> .probe = etm4_probe, >> + .remove = etm4_remove, >> .id_table = etm4_ids, >> }; >> -builtin_amba_driver(etm4x_driver); >> +module_amba_driver(etm4x_driver); >> + >> +MODULE_AUTHOR("Pratik Patel "); >> +MODULE_AUTHOR("Mathieu Poirier "); >> +MODULE_DESCRIPTION("Arm CoreSight Program Flow Trace v4 driver"); >> +MODULE_LICENSE("GPL v2"); >> -- >> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora > Forum, >> a Linux Foundation Collaborative Project >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel