From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 769B1C43387 for ; Fri, 18 Jan 2019 12:08:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3856D2087E for ; Fri, 18 Jan 2019 12:08:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PEcZ4Iks" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727549AbfARMIp (ORCPT ); Fri, 18 Jan 2019 07:08:45 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:34574 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727020AbfARMIn (ORCPT ); Fri, 18 Jan 2019 07:08:43 -0500 Received: by mail-pg1-f193.google.com with SMTP id j10so5995762pga.1; Fri, 18 Jan 2019 04:08:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Upm3BBmIrcMx9tgTplaaoFy2zP31xnGYQuE4efmwbJg=; b=PEcZ4IkszRekRCge7q5JF3QhlDyfhhkmJBBAs+PPoLh7Twc0bOyLl0ukM3zU40aFkA r0oMuHjW8ziQH4h08fx49Zfgw0kgXjwTe37Tx/9OZX7VcWbcqHhQ6Yg/FWFSfOaEBdYz dysTi5RLhR3K0MW3M80J/Bgr304obnfDmDAZ1BtcbBMiaHS9zHPvnrqD4FBzBKHtRzud mj4AVg/xg8tHBnlwuvfTPZBZBlz27/mZT+Vm/GTXshkvKR30fxgqoAMGsqeUF3Jxwisq 3QfNlZp3KrRFIZyOL01eWLRoHHZjo3+nfg1qbp9sUqDfVkaSD7Waa7FawEBIQumWQ8T0 opNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Upm3BBmIrcMx9tgTplaaoFy2zP31xnGYQuE4efmwbJg=; b=rSriTR+3ER2QMfvgNUWVATSW4tQenIQ5wmXPBD5NKoYdDuxgtZ95n+XESUd3RbtvSO YaFTfqe5qKo83kWx+3X4Ow2pe5NSFmxWkB2rwzinjQuZKW4uWAJgm1LPO2LZE+V9im3I Nd2ycfuqPxj9+IVN9Kb+tsSHMoHj6DTPPSDDfawOrOmfaj71tfJBrgvEP/uwBL0x9LCp NiPgHlpWDXjZVCHdg+rdCBX1x1QohZ1GoVfnnvYB16R3I7rZnVaYeuqYrAT2NJJoPV0d 9eubSHGhzSNgU9R8wK7oiMEBByFL6tHkOIsRCmmQygYOsakOFdwHpeFmKV9KmZAQKddr 7cLA== X-Gm-Message-State: AJcUukc9OOASc2T2l2hYP3cUUjWEG5hRa8Jk3QL2BS13bLVve1b9ZODH 3XFin3KJXfJgyUkowvZwFGA= X-Google-Smtp-Source: ALg8bN5elhXdbfnoLRJxNH9st1YeS9tfgSR7lkgMtSeuXuxoTr0TmHjisCb9nqS9Ufi44kNh2ZBZFg== X-Received: by 2002:a63:61c8:: with SMTP id v191mr17536961pgb.242.1547813322140; Fri, 18 Jan 2019 04:08:42 -0800 (PST) Received: from server.roeck-us.net ([2600:1700:e321:62f0:329c:23ff:fee3:9d7c]) by smtp.gmail.com with ESMTPSA id e86sm6581388pfb.6.2019.01.18.04.08.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Jan 2019 04:08:41 -0800 (PST) Subject: Re: [PATCH v5 3/3] PM/runtime:Replace jiffies based accounting with ktime based accounting To: "Rafael J. Wysocki" , Vincent Guittot Cc: "open list:THERMAL" , linux-kernel , "Rafael J. Wysocki" , Thara Gopinath , Jani Nikula , Joonas Lahtinen , rodrigo.vivi@intel.com, David Airlie , Intel graphics driver community testing & development , dri-devel , Ulf Hansson References: <1545388436-7489-1-git-send-email-vincent.guittot@linaro.org> <1545388436-7489-4-git-send-email-vincent.guittot@linaro.org> <20190117221628.GA21979@roeck-us.net> <20190118104215.GA13168@linaro.org> From: Guenter Roeck Message-ID: <9ff86048-fca5-11b9-174c-f0a9338dd231@roeck-us.net> Date: Fri, 18 Jan 2019 04:08:39 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 1/18/19 3:05 AM, Rafael J. Wysocki wrote: > On Fri, Jan 18, 2019 at 11:53 AM Vincent Guittot > wrote: >> >> On Fri, 18 Jan 2019 at 11:42, Vincent Guittot >> wrote: >>> >>> Hi Guenter, >>> >>> Le Thursday 17 Jan 2019 à 14:16:28 (-0800), Guenter Roeck a écrit : >>>> On Fri, Dec 21, 2018 at 11:33:56AM +0100, Vincent Guittot wrote: >>>>> From: Thara Gopinath >>>>> >>>>> This patch replaces jiffies based accounting for runtime_active_time >>>>> and runtime_suspended_time with ktime base accounting. This makes the >>>>> runtime debug counters inline with genpd and other pm subsytems which >>>>> uses ktime based accounting. >>>>> >>>>> timekeeping is initialized before pm_runtime_init() so ktime_get() will >>>>> be ready before first call. In fact, timekeeping_init() is called early >>>>> in start_kernel() which is way before driver_init() (and that's when >>>>> devices can start to be initialized) called from rest_init() via >>>>> kernel_init_freeable() and do_basic_setup(). >>>>> >>>> This is not (always) correct. My qemu "collie" boot test fails with this >>>> patch applied. Reverting the patch fixes the problem. Bisect log attached. >>>> >>> >>> Can you try the patch below ? >>> ktime_get_mono_fast_ns() has the advantage of being init with dummy clock so >>> it can be used at early_init. >> >> Another possibility would be delay the init of the gpiochip > > Well, right. > > Initializing devices before timekeeping doesn't feel particularly > robust from the design perspective. > > How exactly does that happen? > With an added 'initialized' flag and backtrace into the timekeeping code, with the change suggested earlier applied: ------------[ cut here ]------------ WARNING: CPU: 0 PID: 0 at kernel/time/timekeeping.c:453 ktime_get_mono_fast_ns+0x114/0x12c Timekeeping not initialized CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc2-next-20190117-dirty #2 Hardware name: Sharp-Collie Backtrace: [] (dump_backtrace) from [] (show_stack+0x18/0x1c) r7:00000009 r6:00000000 r5:c065ba90 r4:c06d3e54 [] (show_stack) from [] (dump_stack+0x20/0x28) [] (dump_stack) from [] (__warn+0xcc/0xf4) [] (__warn) from [] (warn_slowpath_fmt+0x4c/0x6c) r8:df407b08 r7:00000000 r6:c0c01550 r5:c065bad8 r4:c06dd028 [] (warn_slowpath_fmt) from [] (ktime_get_mono_fast_ns+0x114/0x12c) r3:00000000 r2:c065bad8 r5:00000000 r4:df407b08 [] (ktime_get_mono_fast_ns) from [] (pm_runtime_init+0x38/0xb8) r9:c06c9a5c r8:df407b08 r7:00000000 r6:c0c01550 r5:00000000 r4:df407b08 [] (pm_runtime_init) from [] (device_initialize+0xb0/0xec) r7:00000000 r6:c0c01550 r5:00000000 r4:df407b08 [] (device_initialize) from [] (gpiochip_add_data_with_key+0x9c/0x884) r7:00000000 r6:c06fca34 r5:00000000 r4:00000000 [] (gpiochip_add_data_with_key) from [] (sa1100_init_gpio+0x40/0x98) r10:dfffcd60 r9:c06c9a5c r8:c06dd020 r7:c06dd028 r6:ffffffff r5:00000000 r4:c06fca34 [] (sa1100_init_gpio) from [] (sa1100_init_irq+0x2c/0x3c) r7:c06dd028 r6:ffffffff r5:c0713300 r4:c06e1070 [] (sa1100_init_irq) from [] (init_IRQ+0x20/0x28) r5:c0713300 r4:00000000 [] (init_IRQ) from [] (start_kernel+0x254/0x4cc) [] (start_kernel) from [<00000000>] ( (null)) r10:0000717f r9:6901b119 r8:c0000100 r7:00000092 r6:0000313d r5:00000053 r4:c06a7330 ---[ end trace 91e1bd00dd7cce32 ]--- Guenter From mboxrd@z Thu Jan 1 00:00:00 1970 From: Guenter Roeck Subject: Re: [PATCH v5 3/3] PM/runtime:Replace jiffies based accounting with ktime based accounting Date: Fri, 18 Jan 2019 04:08:39 -0800 Message-ID: <9ff86048-fca5-11b9-174c-f0a9338dd231@roeck-us.net> References: <1545388436-7489-1-git-send-email-vincent.guittot@linaro.org> <1545388436-7489-4-git-send-email-vincent.guittot@linaro.org> <20190117221628.GA21979@roeck-us.net> <20190118104215.GA13168@linaro.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: "Rafael J. Wysocki" , Vincent Guittot Cc: Ulf Hansson , dri-devel , "open list:THERMAL" , David Airlie , Intel graphics driver community testing & development , "Rafael J. Wysocki" , linux-kernel , Thara Gopinath List-Id: linux-pm@vger.kernel.org T24gMS8xOC8xOSAzOjA1IEFNLCBSYWZhZWwgSi4gV3lzb2NraSB3cm90ZToKPiBPbiBGcmksIEph biAxOCwgMjAxOSBhdCAxMTo1MyBBTSBWaW5jZW50IEd1aXR0b3QKPiA8dmluY2VudC5ndWl0dG90 QGxpbmFyby5vcmc+IHdyb3RlOgo+Pgo+PiBPbiBGcmksIDE4IEphbiAyMDE5IGF0IDExOjQyLCBW aW5jZW50IEd1aXR0b3QKPj4gPHZpbmNlbnQuZ3VpdHRvdEBsaW5hcm8ub3JnPiB3cm90ZToKPj4+ Cj4+PiBIaSBHdWVudGVyLAo+Pj4KPj4+IExlIFRodXJzZGF5IDE3IEphbiAyMDE5IMOgIDE0OjE2 OjI4ICgtMDgwMCksIEd1ZW50ZXIgUm9lY2sgYSDDqWNyaXQgOgo+Pj4+IE9uIEZyaSwgRGVjIDIx LCAyMDE4IGF0IDExOjMzOjU2QU0gKzAxMDAsIFZpbmNlbnQgR3VpdHRvdCB3cm90ZToKPj4+Pj4g RnJvbTogVGhhcmEgR29waW5hdGggPHRoYXJhLmdvcGluYXRoQGxpbmFyby5vcmc+Cj4+Pj4+Cj4+ Pj4+IFRoaXMgcGF0Y2ggcmVwbGFjZXMgamlmZmllcyBiYXNlZCBhY2NvdW50aW5nIGZvciBydW50 aW1lX2FjdGl2ZV90aW1lCj4+Pj4+IGFuZCBydW50aW1lX3N1c3BlbmRlZF90aW1lIHdpdGgga3Rp bWUgYmFzZSBhY2NvdW50aW5nLiBUaGlzIG1ha2VzIHRoZQo+Pj4+PiBydW50aW1lIGRlYnVnIGNv dW50ZXJzIGlubGluZSB3aXRoIGdlbnBkIGFuZCBvdGhlciBwbSBzdWJzeXRlbXMgd2hpY2gKPj4+ Pj4gdXNlcyBrdGltZSBiYXNlZCBhY2NvdW50aW5nLgo+Pj4+Pgo+Pj4+PiB0aW1la2VlcGluZyBp cyBpbml0aWFsaXplZCBiZWZvcmUgcG1fcnVudGltZV9pbml0KCkgc28ga3RpbWVfZ2V0KCkgd2ls bAo+Pj4+PiBiZSByZWFkeSBiZWZvcmUgZmlyc3QgY2FsbC4gSW4gZmFjdCwgdGltZWtlZXBpbmdf aW5pdCgpIGlzIGNhbGxlZCBlYXJseQo+Pj4+PiBpbiBzdGFydF9rZXJuZWwoKSB3aGljaCBpcyB3 YXkgYmVmb3JlIGRyaXZlcl9pbml0KCkgKGFuZCB0aGF0J3Mgd2hlbgo+Pj4+PiBkZXZpY2VzIGNh biBzdGFydCB0byBiZSBpbml0aWFsaXplZCkgY2FsbGVkIGZyb20gcmVzdF9pbml0KCkgdmlhCj4+ Pj4+IGtlcm5lbF9pbml0X2ZyZWVhYmxlKCkgYW5kIGRvX2Jhc2ljX3NldHVwKCkuCj4+Pj4+Cj4+ Pj4gVGhpcyBpcyBub3QgKGFsd2F5cykgY29ycmVjdC4gTXkgcWVtdSAiY29sbGllIiBib290IHRl c3QgZmFpbHMgd2l0aCB0aGlzCj4+Pj4gcGF0Y2ggYXBwbGllZC4gUmV2ZXJ0aW5nIHRoZSBwYXRj aCBmaXhlcyB0aGUgcHJvYmxlbS4gQmlzZWN0IGxvZyBhdHRhY2hlZC4KPj4+Pgo+Pj4KPj4+IENh biB5b3UgdHJ5IHRoZSBwYXRjaCBiZWxvdyA/Cj4+PiBrdGltZV9nZXRfbW9ub19mYXN0X25zKCkg aGFzIHRoZSBhZHZhbnRhZ2Ugb2YgYmVpbmcgaW5pdCB3aXRoIGR1bW15IGNsb2NrIHNvCj4+PiBp dCBjYW4gYmUgdXNlZCBhdCBlYXJseV9pbml0Lgo+Pgo+PiBBbm90aGVyIHBvc3NpYmlsaXR5IHdv dWxkIGJlIGRlbGF5IHRoZSBpbml0IG9mIHRoZSBncGlvY2hpcAo+IAo+IFdlbGwsIHJpZ2h0Lgo+ IAo+IEluaXRpYWxpemluZyBkZXZpY2VzIGJlZm9yZSB0aW1la2VlcGluZyBkb2Vzbid0IGZlZWwg cGFydGljdWxhcmx5Cj4gcm9idXN0IGZyb20gdGhlIGRlc2lnbiBwZXJzcGVjdGl2ZS4KPiAKPiBI b3cgZXhhY3RseSBkb2VzIHRoYXQgaGFwcGVuPwo+IAoKV2l0aCBhbiBhZGRlZCAnaW5pdGlhbGl6 ZWQnIGZsYWcgYW5kIGJhY2t0cmFjZSBpbnRvIHRoZSB0aW1la2VlcGluZyBjb2RlLAp3aXRoIHRo ZSBjaGFuZ2Ugc3VnZ2VzdGVkIGVhcmxpZXIgYXBwbGllZDoKCi0tLS0tLS0tLS0tLVsgY3V0IGhl cmUgXS0tLS0tLS0tLS0tLQpXQVJOSU5HOiBDUFU6IDAgUElEOiAwIGF0IGtlcm5lbC90aW1lL3Rp bWVrZWVwaW5nLmM6NDUzIGt0aW1lX2dldF9tb25vX2Zhc3RfbnMrMHgxMTQvMHgxMmMKVGltZWtl ZXBpbmcgbm90IGluaXRpYWxpemVkCkNQVTogMCBQSUQ6IDAgQ29tbTogc3dhcHBlciBOb3QgdGFp bnRlZCA1LjAuMC1yYzItbmV4dC0yMDE5MDExNy1kaXJ0eSAjMgpIYXJkd2FyZSBuYW1lOiBTaGFy cC1Db2xsaWUKQmFja3RyYWNlOgpbPGMwMDBkY2U4Pl0gKGR1bXBfYmFja3RyYWNlKSBmcm9tIFs8 YzAwMGRmNzg+XSAoc2hvd19zdGFjaysweDE4LzB4MWMpCiAgcjc6MDAwMDAwMDkgcjY6MDAwMDAw MDAgcjU6YzA2NWJhOTAgcjQ6YzA2ZDNlNTQKWzxjMDAwZGY2MD5dIChzaG93X3N0YWNrKSBmcm9t IFs8YzA1ODg5MzA+XSAoZHVtcF9zdGFjaysweDIwLzB4MjgpCls8YzA1ODg5MTA+XSAoZHVtcF9z dGFjaykgZnJvbSBbPGMwMDE4YWU4Pl0gKF9fd2FybisweGNjLzB4ZjQpCls8YzAwMThhMWM+XSAo X193YXJuKSBmcm9tIFs8YzAwMThiNWM+XSAod2Fybl9zbG93cGF0aF9mbXQrMHg0Yy8weDZjKQog IHI4OmRmNDA3YjA4IHI3OjAwMDAwMDAwIHI2OmMwYzAxNTUwIHI1OmMwNjViYWQ4IHI0OmMwNmRk MDI4Cls8YzAwMThiMTQ+XSAod2Fybl9zbG93cGF0aF9mbXQpIGZyb20gWzxjMDA2OWUyYz5dIChr dGltZV9nZXRfbW9ub19mYXN0X25zKzB4MTE0LzB4MTJjKQogIHIzOjAwMDAwMDAwIHIyOmMwNjVi YWQ4CiAgcjU6MDAwMDAwMDAgcjQ6ZGY0MDdiMDgKWzxjMDA2OWQxOD5dIChrdGltZV9nZXRfbW9u b19mYXN0X25zKSBmcm9tIFs8YzAzYzc4MTA+XSAocG1fcnVudGltZV9pbml0KzB4MzgvMHhiOCkK ICByOTpjMDZjOWE1YyByODpkZjQwN2IwOCByNzowMDAwMDAwMCByNjpjMGMwMTU1MCByNTowMDAw MDAwMCByNDpkZjQwN2IwOApbPGMwM2M3N2Q4Pl0gKHBtX3J1bnRpbWVfaW5pdCkgZnJvbSBbPGMw M2I2YTM0Pl0gKGRldmljZV9pbml0aWFsaXplKzB4YjAvMHhlYykKICByNzowMDAwMDAwMCByNjpj MGMwMTU1MCByNTowMDAwMDAwMCByNDpkZjQwN2IwOApbPGMwM2I2OTg0Pl0gKGRldmljZV9pbml0 aWFsaXplKSBmcm9tIFs8YzAzNjZkMzA+XSAoZ3Bpb2NoaXBfYWRkX2RhdGFfd2l0aF9rZXkrMHg5 Yy8weDg4NCkKICByNzowMDAwMDAwMCByNjpjMDZmY2EzNCByNTowMDAwMDAwMCByNDowMDAwMDAw MApbPGMwMzY2Yzk0Pl0gKGdwaW9jaGlwX2FkZF9kYXRhX3dpdGhfa2V5KSBmcm9tIFs8YzA2Yjk3 MDg+XSAoc2ExMTAwX2luaXRfZ3BpbysweDQwLzB4OTgpCiAgcjEwOmRmZmZjZDYwIHI5OmMwNmM5 YTVjIHI4OmMwNmRkMDIwIHI3OmMwNmRkMDI4IHI2OmZmZmZmZmZmIHI1OjAwMDAwMDAwCiAgcjQ6 YzA2ZmNhMzQKWzxjMDZiOTZjOD5dIChzYTExMDBfaW5pdF9ncGlvKSBmcm9tIFs8YzA2YWU1OGM+ XSAoc2ExMTAwX2luaXRfaXJxKzB4MmMvMHgzYykKICByNzpjMDZkZDAyOCByNjpmZmZmZmZmZiBy NTpjMDcxMzMwMCByNDpjMDZlMTA3MApbPGMwNmFlNTYwPl0gKHNhMTEwMF9pbml0X2lycSkgZnJv bSBbPGMwNmFhYjFjPl0gKGluaXRfSVJRKzB4MjAvMHgyOCkKICByNTpjMDcxMzMwMCByNDowMDAw MDAwMApbPGMwNmFhYWZjPl0gKGluaXRfSVJRKSBmcm9tIFs8YzA2YTdjZDA+XSAoc3RhcnRfa2Vy bmVsKzB4MjU0LzB4NGNjKQpbPGMwNmE3YTdjPl0gKHN0YXJ0X2tlcm5lbCkgZnJvbSBbPDAwMDAw MDAwPl0gKCAgKG51bGwpKQogIHIxMDowMDAwNzE3ZiByOTo2OTAxYjExOSByODpjMDAwMDEwMCBy NzowMDAwMDA5MiByNjowMDAwMzEzZCByNTowMDAwMDA1MwogIHI0OmMwNmE3MzMwCi0tLVsgZW5k IHRyYWNlIDkxZTFiZDAwZGQ3Y2NlMzIgXS0tLQoKR3VlbnRlcgpfX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fXwpJbnRlbC1nZnggbWFpbGluZyBsaXN0CkludGVs LWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcv bWFpbG1hbi9saXN0aW5mby9pbnRlbC1nZngK