Re: [PATCH v14 29/39] soc/tegra: regulators: Prepare for suspend

From: Dmitry Osipenko <digetx@gmail.com>
To: Ulf Hansson <ulf.hansson@linaro.org>
Cc: "Thierry Reding" <thierry.reding@gmail.com>,
	"Jonathan Hunter" <jonathanh@nvidia.com>,
	"Viresh Kumar" <vireshk@kernel.org>,
	"Stephen Boyd" <sboyd@kernel.org>,
	"Peter De Schrijver" <pdeschrijver@nvidia.com>,
	"Mikko Perttunen" <mperttunen@nvidia.com>,
	"Lee Jones" <lee.jones@linaro.org>,
	"Uwe Kleine-König" <u.kleine-koenig@pengutronix.de>,
	"Nishanth Menon" <nm@ti.com>,
	"Adrian Hunter" <adrian.hunter@intel.com>,
	"Michael Turquette" <mturquette@baylibre.com>,
	linux-kernel@vger.kernel.org, linux-tegra@vger.kernel.org,
	linux-pm@vger.kernel.org, linux-pwm@vger.kernel.org,
	linux-mmc@vger.kernel.org, dri-devel@lists.freedesktop.org,
	linux-clk@vger.kernel.org, "David Heidelberg" <david@ixit.cz>
Subject: Re: [PATCH v14 29/39] soc/tegra: regulators: Prepare for suspend
Date: Wed, 27 Oct 2021 22:39:46 +0300	[thread overview]
Message-ID: <a8280b5b-7347-8995-c97b-10b798cdf057@gmail.com> (raw)
In-Reply-To: <CAPDyKFrQfACqtHtsnbk9fJpfaXWgD6-GEy2HFq8DxMTe4+zZmA@mail.gmail.com>

27.10.2021 18:47, Ulf Hansson пишет:
>> Depending on hardware version, Tegra SoC may require a higher voltages
>> during resume from system suspend, otherwise hardware will crash. Set
>> SoC voltages to a nominal levels during suspend.
>>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> I don't understand the reason why you need to use pm notifiers to
> manage these things. Those are invoked really early during the system
> suspend process and really late during the system resume process.

The suspend/resume time doesn't matter as long as venc genpd resumes
earlier than regulator is unprepared during late resume. Hence early
suspend and late resume suit well.

> In regards to this, you are mentioning the behaviour in genpd around
> system suspend/resume in a comment a few lines below, and that it's
> problematic for the venc domain. Can you perhaps share some more
> information, just to make sure we shouldn't fix the problem in genpd
> instead?

GENPD core force-resumes all domains early during system resume and this
causes odd problem on Tegra20 device in regards to resuming of video
encoder domain where SoC sometimes hangs after couple milliseconds since
the time of ungating the domain if SoC core voltage is low at that time.

Initially I was blaming WiFi driver because somehow this problem didn't
happen if WiFi chip was disabled [1]. I dived into debugging and found
that hang happens after ungating venc early during resume from suspend,
i.e. when genpd core resumes it.

[1] https://www.spinics.net/lists/linux-wireless/msg212116.html

Interestingly, this problem isn't reproducible when system is fully
resumed, i.e. venc can be freely gated/ungated at a low voltage without
any visible problems.

What's also interesting, it's impossible to reproduce hang on a second
resume from suspend if it didn't happen on the first resume. Need to
reboot and try again in that case.

In the end I found that bumping SoC core voltage 100% solves the trouble.

I knew that downstream kernel bumps voltage during suspend, but it
doesn't explain why. I replicated the suspicious behaviour of downstream
kernel and the problem has gone. Could be that this only masks the real
problem, but I don't have more information and the problem is solved.