From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEE47C4338F for ; Mon, 9 Aug 2021 11:01:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8EB2461055 for ; Mon, 9 Aug 2021 11:01:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234949AbhHILBz (ORCPT ); Mon, 9 Aug 2021 07:01:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53958 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234948AbhHILBz (ORCPT ); Mon, 9 Aug 2021 07:01:55 -0400 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41AA1C0613D3 for ; Mon, 9 Aug 2021 04:01:35 -0700 (PDT) Received: from gallifrey.ext.pengutronix.de ([2001:67c:670:201:5054:ff:fe8d:eefb] helo=[IPv6:::1]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mD328-0003RL-TW; Mon, 09 Aug 2021 13:01:28 +0200 Message-ID: <8ea33d97fb3f7abb2d80b11db28cce8c01932a09.camel@pengutronix.de> Subject: Re: [PATCH v2 00/18] i.MX8MM GPC improvements and BLK_CTRL driver From: Lucas Stach To: Frieder Schrempf , Shawn Guo , Rob Herring Cc: NXP Linux Team , Adam Ford , Peng Fan , Marek Vasut , devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kernel@pengutronix.de, patchwork-lst@pengutronix.de Date: Mon, 09 Aug 2021 13:01:27 +0200 In-Reply-To: <8de1cd0a-4d91-60e2-61e6-9f903bbf546b@kontron.de> References: <20210716232916.3572966-1-l.stach@pengutronix.de> <20210721204703.1424034-1-l.stach@pengutronix.de> <818b52fe-8fa6-b47a-6dde-783ac378c603@kontron.de> <8de1cd0a-4d91-60e2-61e6-9f903bbf546b@kontron.de> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.3 (3.40.3-1.fc34) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 2001:67c:670:201:5054:ff:fe8d:eefb X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: devicetree@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Hi Frieder, Am Donnerstag, dem 05.08.2021 um 20:56 +0200 schrieb Frieder Schrempf: > On 05.08.21 12:18, Frieder Schrempf wrote: > > On 21.07.21 22:46, Lucas Stach wrote: > > > Hi all, > > > > > > second revision of the GPC improvements and BLK_CTRL driver to make use > > > of all the power-domains on the i.MX8MM. I'm not going to repeat the full > > > blurb from the v1 cover letter here, but if you are not familiar with > > > i.MX8MM power domains, it may be worth a read. > > > > > > This 2nd revision fixes the DT bindings to be valid yaml, some small > > > failure path issues and most importantly the interaction with system > > > suspend/resume. With the previous version some of the power domains > > > would not come up correctly after a suspend/resume cycle. > > > > > > Updated testing git trees here, disclaimer still applies: > > > https://git.pengutronix.de/cgit/lst/linux/log/?h=imx8m-power-domains > > > https://git.pengutronix.de/cgit/lst/linux/log/?h=imx8m-power-domains-testing > > > > I finally did some tests on my side using USB, GPU and DSI (no PCIe, VPU, CSI so far) and the results are promising. Thanks for the effort! > > > > I will try to run some more automated suspend/resume and reboot test cycles over the weekend and report the results here afterwards. > > > > Unfortunately I got some results sooner than I had hoped. I set up a simple loop to suspend/resume every few seconds and on the first run it took around 2-3 hours for the device to lock up on resume. On the second run it took less than half an hour. I had glmark2-es2-drm running in the background, but it looks like it crashed at some point before the lockup occurred. > > Of course this could also be unrelated and caused by some peripheral driver or something but the first suspicion is definitely the power domains. > > If you have any suggestions for which debug options to enable or where to add some printks, please let me know. If I do another run I would like to make sure that the resulting logs are helpful for debugging. > > And I would appreciate if someone else could try to reproduce this problem on his/her side. I use this simple script for testing: > > #!/bin/sh > > glmark2-es2-drm & > > while true; > do > echo +10 > /sys/class/rtc/rtc0/wakealarm > echo mem > /sys/power/state > sleep 5 > done; Hm, that's unfortunate. I'm back from a two week vacation, but it looks like I won't have much time available to look into this issue soon. It would be very helpful if you could try to pinpoint the hang a bit more. If you can reproduce the hang with no_console_suspend you might be able to extract a bit more info in which stage the hang happens (suspend, resume, TF-A, etc.) If the hang is in the kernel you might be able to add some prints to the suspend/resume paths to be able to track down the exact point of the hang. I'm happy to look into the issue once it's better known where to look, but I fear that I won't have time to do the above investigation myself short term. Frieder, is this something you could help with over the next few days? Regards, Lucas From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90E5AC4338F for ; Mon, 9 Aug 2021 11:03:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 36D8060295 for ; Mon, 9 Aug 2021 11:03:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 36D8060295 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=pengutronix.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:Cc:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Fj23kijc99FE+WyFbBhQmtE73PqxzsED2Vk6Dqa0Svs=; b=hYPEIVGK9sqRn3 vracIpNYdyePClx5wvVASApBxcGqh2LKtKhSuowVSQbG99Ow+OfrWRIUllO6XdyT3JShLWQ8l8ToL PL0ud7psrtnwqlzJejRNz0vBjJ65YKHLjhRui6tAZuPHh+GnZ/7orQlbw5v4f097uvN9sfcQWFddG cDrhed0JgxJEMTXl1nxwVz8aX0evz/O62kSuEfeuYO68bDzc7UEfzKrr7GDOF2d0ikno8Ky9zil0m GSe4iiXBNiSHLxyhV8t+aH7jssNbAWweo0tG5HOpg2KlHYRfzq0ku5edObVdOUbTp1cbmQJpEOHej fMHR31d+HBEryCqLBkIw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD32K-000HHp-0G; Mon, 09 Aug 2021 11:01:40 +0000 Received: from metis.ext.pengutronix.de ([2001:67c:670:201:290:27ff:fe1d:cc33]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD32F-000HFz-1z for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 11:01:36 +0000 Received: from gallifrey.ext.pengutronix.de ([2001:67c:670:201:5054:ff:fe8d:eefb] helo=[IPv6:::1]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mD328-0003RL-TW; Mon, 09 Aug 2021 13:01:28 +0200 Message-ID: <8ea33d97fb3f7abb2d80b11db28cce8c01932a09.camel@pengutronix.de> Subject: Re: [PATCH v2 00/18] i.MX8MM GPC improvements and BLK_CTRL driver From: Lucas Stach To: Frieder Schrempf , Shawn Guo , Rob Herring Cc: NXP Linux Team , Adam Ford , Peng Fan , Marek Vasut , devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kernel@pengutronix.de, patchwork-lst@pengutronix.de Date: Mon, 09 Aug 2021 13:01:27 +0200 In-Reply-To: <8de1cd0a-4d91-60e2-61e6-9f903bbf546b@kontron.de> References: <20210716232916.3572966-1-l.stach@pengutronix.de> <20210721204703.1424034-1-l.stach@pengutronix.de> <818b52fe-8fa6-b47a-6dde-783ac378c603@kontron.de> <8de1cd0a-4d91-60e2-61e6-9f903bbf546b@kontron.de> User-Agent: Evolution 3.40.3 (3.40.3-1.fc34) MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2001:67c:670:201:5054:ff:fe8d:eefb X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-arm-kernel@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_040135_151924_AD1151BB X-CRM114-Status: GOOD ( 35.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Frieder, Am Donnerstag, dem 05.08.2021 um 20:56 +0200 schrieb Frieder Schrempf: > On 05.08.21 12:18, Frieder Schrempf wrote: > > On 21.07.21 22:46, Lucas Stach wrote: > > > Hi all, > > > > > > second revision of the GPC improvements and BLK_CTRL driver to make use > > > of all the power-domains on the i.MX8MM. I'm not going to repeat the full > > > blurb from the v1 cover letter here, but if you are not familiar with > > > i.MX8MM power domains, it may be worth a read. > > > > > > This 2nd revision fixes the DT bindings to be valid yaml, some small > > > failure path issues and most importantly the interaction with system > > > suspend/resume. With the previous version some of the power domains > > > would not come up correctly after a suspend/resume cycle. > > > > > > Updated testing git trees here, disclaimer still applies: > > > https://git.pengutronix.de/cgit/lst/linux/log/?h=imx8m-power-domains > > > https://git.pengutronix.de/cgit/lst/linux/log/?h=imx8m-power-domains-testing > > > > I finally did some tests on my side using USB, GPU and DSI (no PCIe, VPU, CSI so far) and the results are promising. Thanks for the effort! > > > > I will try to run some more automated suspend/resume and reboot test cycles over the weekend and report the results here afterwards. > > > > Unfortunately I got some results sooner than I had hoped. I set up a simple loop to suspend/resume every few seconds and on the first run it took around 2-3 hours for the device to lock up on resume. On the second run it took less than half an hour. I had glmark2-es2-drm running in the background, but it looks like it crashed at some point before the lockup occurred. > > Of course this could also be unrelated and caused by some peripheral driver or something but the first suspicion is definitely the power domains. > > If you have any suggestions for which debug options to enable or where to add some printks, please let me know. If I do another run I would like to make sure that the resulting logs are helpful for debugging. > > And I would appreciate if someone else could try to reproduce this problem on his/her side. I use this simple script for testing: > > #!/bin/sh > > glmark2-es2-drm & > > while true; > do > echo +10 > /sys/class/rtc/rtc0/wakealarm > echo mem > /sys/power/state > sleep 5 > done; Hm, that's unfortunate. I'm back from a two week vacation, but it looks like I won't have much time available to look into this issue soon. It would be very helpful if you could try to pinpoint the hang a bit more. If you can reproduce the hang with no_console_suspend you might be able to extract a bit more info in which stage the hang happens (suspend, resume, TF-A, etc.) If the hang is in the kernel you might be able to add some prints to the suspend/resume paths to be able to track down the exact point of the hang. I'm happy to look into the issue once it's better known where to look, but I fear that I won't have time to do the above investigation myself short term. Frieder, is this something you could help with over the next few days? Regards, Lucas _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel