From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38820C43381 for ; Mon, 4 Mar 2019 01:28:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 049C020830 for ; Mon, 4 Mar 2019 01:28:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="dpNUWQcD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 049C020830 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=jp.panasonic.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: From:Subject:To:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=onHL4QPKDgvw1pLz+d4+sU06t871+VswXoc18Vs+PWE=; b=dpNUWQcDrsg5Xb eeM6HDaOXJv5JodDHrKNwbCVdZOn4jKmVqbMwASxMUJucFuOIWxhhRlgcErp6uKJTlMcY2pMe16QB /ZgXCAaxpZk3F4P5rvRQpLeHRLILrjJTrhys9AlShXxjFX+JXxLUwlh8QaaGxtx1ezb/gXAAoN42X 68KikibN7nFWLZCxlfB7dCx2Oiglbw+2LuLjSoZ8uyc56Qfl8EVu4gzmBYF/qg8fu5wwww7KSxtoy JMlJM2j9Gj73qRuouv2OtB5HgeWt3bz/KvfgciEZ8T/qBQpqL08C8v50iNMzMpymeKgknQtON5dUy dcVJkrBsh4XFgO3xxXzQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h0cP5-00015m-EF; Mon, 04 Mar 2019 01:28:27 +0000 Received: from smtp.mei.co.jp ([133.183.100.20]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h0cP1-00015L-5h for linux-arm-kernel@lists.infradead.org; Mon, 04 Mar 2019 01:28:25 +0000 Received: from mail-gw.jp.panasonic.com ([157.8.1.157]) by smtp.mei.co.jp (8.14.4/8.14.4/lvzy12) with ESMTP id x241SIH2026302; Mon, 4 Mar 2019 10:28:18 +0900 Received: from epochmail.jp.panasonic.com ([157.8.1.130]) by mail.jp.panasonic.com (8.11.6p2/3.7W/kc-maili11) with ESMTP id x241SHA10473; Mon, 4 Mar 2019 10:28:17 +0900 Received: by epochmail.jp.panasonic.com (8.12.11.20060308/3.7W/pml-send2) id x241SHCw018186; Mon, 4 Mar 2019 10:28:17 +0900 Received: from jpa000secmn21.palet.jp.panasonic.com [10.74.216.56] by JPA000SGWMN01.palet.jp.panasonic.com with ESMTP id LAA09242; Mon, 4 Mar 2019 10:28:17 +0900 Received: from localhost (10.74.216.181) by JPA000SECMN21.palet.jp.panasonic.com (10.74.216.56) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 4 Mar 2019 10:28:17 +0900 Date: Mon, 4 Mar 2019 10:28:14 +0900 Message-ID: <20190304.102814.1664010168009276853.okuno.kohji@jp.panasonic.com> To: Subject: Re: [PATCH v3] ARM: imx6q: cpuidle: fix bug that CPU might not wake up at expected time From: Kohji Okuno In-Reply-To: <20190301092341.GP26041@dragon> References: <20190222.174953.431124434951497467.okuno.kohji@jp.panasonic.com> <20190226023413.19531-1-okuno.kohji@jp.panasonic.com> <20190301092341.GP26041@dragon> Organization: Panasonic Corporation X-Mailer: Mew version 6.7 on Emacs 25.2 / Mule 6.0 (HANACHIRUSATO) MIME-Version: 1.0 X-Originating-IP: [10.74.216.181] X-ClientProxiedBy: JPA000SECMN11.palet.jp.panasonic.com (10.74.216.20) To JPA000SECMN21.palet.jp.panasonic.com (10.74.216.56) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190303_172823_443147_637DE040 X-CRM114-Status: GOOD ( 18.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: s.hauer@pengutronix.de, okuno.kohji@jp.panasonic.com, linux-imx@nxp.com, kernel@pengutronix.de, fabio.estevam@nxp.com, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Shawn, I found this bug by delaying periodic communication. For example, it is the source codes as below. while (1) { sleep(2); write(fd, "Hello", strlen("Hello")); } Of course, it dows not occur frequently. However, in the worst case, it stopped for more than 1 minute. If there are few running processes and few hrtimer events, this issue will occur. For example: (*) time is abstracting. WFI idle : the CPU will be waked up by local timer. WAIT idle: the CPU will be waked up by broad cast(bc) timer. [time] CPU#0 CPU#1 ---------------------------------------------------------------------- enter "WAIT" idle [next event=10] enter "WAIT" idle [next event=60] all cpus are IDLE = > CPU#1 sets WAIT_UNCLOCKED 10: bc timer fires exit idle enter "WFI" idle [next event=12] 12: ** local timer is not fired ** CPU#0 will not be waked up until the next bc timer interrupt. >> all processes stop << 60: bc timer fires exit idle exit idle CPU#1 sets WAIT_CLOCKED local timer fires If CPU#0 sets WAIT_CLOCKED at time=10, this issue will not occur. Does that make any sense? Best regards, Kohji Okuno Shawn Guo wrote: > Hi Kohji, > > On Tue, Feb 26, 2019 at 11:34:13AM +0900, Kohji Okuno wrote: >> In the current cpuidle implementation for i.MX6q, the CPU that sets >> 'WAIT_UNCLOCKED' and the CPU that returns to 'WAIT_CLOCKED' are always >> the same. While the CPU that sets 'WAIT_UNCLOCKED' is in IDLE state of >> "WAIT", if the other CPU wakes up and enters IDLE state of "WFI" >> istead of "WAIT", this CPU can not wake up at expired time. >> Because, in the case of "WFI", the CPU must be waked up by the local >> timer interrupt. But, while 'WAIT_UNCLOCKED' is set, the local timer >> is stopped, when all CPUs execute "wfi" instruction. As a result, the >> local timer interrupt is not fired. >> In this situation, this CPU will wake up by IRQ different from local >> timer. (e.g. broacast tiemr) >> >> So, this fix changes CPU to return to 'WAIT_CLOCKED'. > > Just out of curiosity, how did you observe the bug? I'm trying to > understand the impact of the bug. > > Shawn > >> >> Signed-off-by: Kohji Okuno >> --- >> >> Changes for v3: >> - remove the braces for a single statement inside the if block. >> >> Changes for v2: >> - change "master" variable name. >> >> arch/arm/mach-imx/cpuidle-imx6q.c | 27 ++++++++++----------------- >> 1 file changed, 10 insertions(+), 17 deletions(-) >> >> diff --git a/arch/arm/mach-imx/cpuidle-imx6q.c b/arch/arm/mach-imx/cpuidle-imx6q.c >> index bfeb25aaf9a2..326e870d7123 100644 >> --- a/arch/arm/mach-imx/cpuidle-imx6q.c >> +++ b/arch/arm/mach-imx/cpuidle-imx6q.c >> @@ -16,30 +16,23 @@ >> #include "cpuidle.h" >> #include "hardware.h" >> >> -static atomic_t master = ATOMIC_INIT(0); >> -static DEFINE_SPINLOCK(master_lock); >> +static int num_idle_cpus = 0; >> +static DEFINE_SPINLOCK(cpuidle_lock); >> >> static int imx6q_enter_wait(struct cpuidle_device *dev, >> struct cpuidle_driver *drv, int index) >> { >> - if (atomic_inc_return(&master) == num_online_cpus()) { >> - /* >> - * With this lock, we prevent other cpu to exit and enter >> - * this function again and become the master. >> - */ >> - if (!spin_trylock(&master_lock)) >> - goto idle; >> + spin_lock(&cpuidle_lock); >> + if (++num_idle_cpus == num_online_cpus()) >> imx6_set_lpm(WAIT_UNCLOCKED); >> - cpu_do_idle(); >> - imx6_set_lpm(WAIT_CLOCKED); >> - spin_unlock(&master_lock); >> - goto done; >> - } >> + spin_unlock(&cpuidle_lock); >> >> -idle: >> cpu_do_idle(); >> -done: >> - atomic_dec(&master); >> + >> + spin_lock(&cpuidle_lock); >> + if (num_idle_cpus-- == num_online_cpus()) >> + imx6_set_lpm(WAIT_CLOCKED); >> + spin_unlock(&cpuidle_lock); >> >> return index; >> } >> -- >> 2.17.1 >> _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel