linux-renesas-soc.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Geert Uytterhoeven <geert+renesas@glider.be>,
	Joerg Roedel <joro@8bytes.org>,
	Magnus Damm <damm+renesas@opensource.se>,
	Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>,
	Linux IOMMU <iommu@lists.linux-foundation.org>,
	Linux-Renesas <linux-renesas-soc@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 7/7] iommu/ipmmu-vmsa: Add suspend/resume support
Date: Wed, 20 Feb 2019 18:11:13 +0200	[thread overview]
Message-ID: <20190220161113.GK3516@pendragon.ideasonboard.com> (raw)
In-Reply-To: <CAMuHMdW2KR=rbtgxbnhW94VCzehX821TMxoTZGKh8uQZMY80cA@mail.gmail.com>

Hi Geert,

On Wed, Feb 20, 2019 at 05:05:49PM +0100, Geert Uytterhoeven wrote:
> On Wed, Feb 20, 2019 at 4:42 PM Laurent Pinchart wrote:
> > On Wed, Feb 20, 2019 at 04:05:31PM +0100, Geert Uytterhoeven wrote:
> >> During PSCI system suspend, R-Car Gen3 SoCs are powered down, and all
> >> IPMMU state is lost.  Hence after s2ram, devices wired behind an IPMMU,
> >> and configured to use it, will see their DMA operations hang.
> >>
> >> To fix this, restore all IPMMU contexts, and re-enable all active
> >> micro-TLBs during system resume.
> >>
> >> To avoid overhead on platforms not needing it, the resume code has a
> >> build time dependency on sleep and PSCI support, and a runtime
> >> dependency on PSCI.
> >>
> >> Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
> >> ---
> >> This patch takes a different approach than the BSP, which implements a
> >> bulk save/restore of all registers during system suspend/resume.
> >
> > I like this approach better too.
> 
> Thanks ;-)
> 
> >> --- a/drivers/iommu/ipmmu-vmsa.c
> >> +++ b/drivers/iommu/ipmmu-vmsa.c
> 
> >> @@ -58,6 +62,7 @@ struct ipmmu_vmsa_device {
> >>       spinlock_t lock;                        /* Protects ctx and domains[] */
> >>       DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
> >>       struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX];
> >> +     s8 utlb_ctx[IPMMU_UTLB_MAX];
> >
> > How about making this a bitmask instead to save memory ? I would also
> > rename it as utlb_ctx doesn't really carry the meaning of the field,
> > whose purpose is to store whether the µTLB is enabled or disabled.
> 
> This field isn't just a binary flag, but stores the context used for the
> uTLB, so we can map from micro-TLB to context.
> Given there can be 8 contexts, plus the need to indicate unused contexts,
> that means 4 bits/micro-TLB.  So the overhead is just 24 bytes per IPMMU
> instance.

My bad, I've overlooked that.

> I considered allocating the array dynamically (by having s8 utlb_ctx[]
> at the end of the structure), but didn't go that route, as the domains[]
> array already uses more memory.
> 
> >> @@ -1158,10 +1166,52 @@ static int ipmmu_remove(struct platform_device *pdev)
> >>       return 0;
> >>  }
> >>
> >> +#if defined(CONFIG_PM_SLEEP) && defined(CONFIG_ARM_PSCI_FW)
> >> +static int ipmmu_resume_noirq(struct device *dev)
> >> +{
> >> +     struct ipmmu_vmsa_device *mmu = dev_get_drvdata(dev);
> >> +     unsigned int i;
> >> +
> >> +     /* This is the best we can do to check for the presence of PSCI */
> >> +     if (!psci_ops.cpu_suspend)
> >> +             return 0;
> >
> > PSCI suspend disabling power to the SoC completely may be a common
> > behaviour on our development boards, but isn't mandated by the PSCI
> > specification if I'm not mistaken. Is there a way to instead detect that
> > power has been lost, perhaps by checking whether a register has been
> > reset to its default value ?
> 
> The approach here is the same as in the clk and pinctrl drivers.
> 
> I think we could check if the IMCTR registers for allocated domains in root
> IPMMUs are non-zero.  But that's about as expensive as doing the full
> restore, I think.

Would reading just one register be more expensive that full
reconfiguration ? Or is there no single register that could serve this
purpose ?

> And it may have to be done for each and every IPMMU instance, or do
> you trust caching for this?

If we can find a single register I think that reading it for every IPMMU
instance wouldn't be an issue.

> >> +
> >> +     /* Reset root MMU and restore contexts */
> >
> > I think the rest of the code adds a period at the end of sentences in
> > comments.
> 
> The balance seems to be just under 50% ;-)
> 
> >> +     if (ipmmu_is_root(mmu)) {
> >> +             ipmmu_device_reset(mmu);
> >> +
> >> +             for (i = 0; i < mmu->num_ctx; i++) {
> >> +                     if (!mmu->domains[i])
> >> +                             continue;
> >> +
> >> +                     ipmmu_context_init(mmu->domains[i]);
> >> +             }
> >> +     }
> >> +
> >> +     /* Re-enable active micro-TLBs */
> >> +     for (i = 0; i < mmu->features->num_utlbs; i++) {
> >> +             if (mmu->utlb_ctx[i] == IPMMU_CTX_INVALID)
> >> +                     continue;
> >> +
> >> +             ipmmu_utlb_enable(mmu->root->domains[mmu->utlb_ctx[i]], i);
> >> +     }
> >> +
> >> +     return 0;
> >> +}
> >> +
> >> +static const struct dev_pm_ops ipmmu_pm  = {
> >> +     SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(NULL, ipmmu_resume_noirq)
> >> +};
> >> +#define DEV_PM_OPS   &ipmmu_pm
> >> +#else
> >> +#define DEV_PM_OPS   NULL
> >> +#endif /* CONFIG_PM_SLEEP && CONFIG_ARM_PSCI_FW */
> >> +
> >>  static struct platform_driver ipmmu_driver = {
> >>       .driver = {
> >>               .name = "ipmmu-vmsa",
> >>               .of_match_table = of_match_ptr(ipmmu_of_ids),
> >> +             .pm = DEV_PM_OPS,
> >
> > I would have used conditional compilation here instead of using a
> > DEV_PM_OPS macro, as I think the macro decreases readability (and also
> > given how its generic name could later conflict with something else).
> 
> You mean
> 
>     #ifdef ...
>     .pm = &ipmmu_pm,
>     #endif
> 
> and marking ipmmu_pm __maybe_unused__?

Yes. Up to you.

-- 
Regards,

Laurent Pinchart

  reply	other threads:[~2019-02-20 16:11 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-20 15:05 [PATCH 0/7] iommu/ipmmu-vmsa: Suspend/resume support and assorted cleanups Geert Uytterhoeven
2019-02-20 15:05 ` [PATCH 1/7] iommu/ipmmu-vmsa: Link IOMMUs and devices in sysfs Geert Uytterhoeven
2019-02-20 15:25   ` Laurent Pinchart
2019-02-20 15:05 ` [PATCH 2/7] iommu/ipmmu-vmsa: Call ipmmu_ctx_write_root() instead of open coding Geert Uytterhoeven
2019-02-20 15:27   ` Laurent Pinchart
2019-02-20 15:05 ` [PATCH 3/7] iommu/ipmmu-vmsa: Prepare to handle 40-bit error addresses Geert Uytterhoeven
2019-02-20 15:30   ` Laurent Pinchart
2019-02-20 15:42     ` Geert Uytterhoeven
2019-02-20 15:05 ` [PATCH 4/7] iommu/ipmmu-vmsa: Make IPMMU_CTX_MAX unsigned Geert Uytterhoeven
2019-02-20 15:31   ` Laurent Pinchart
2019-02-20 15:05 ` [PATCH 5/7] iommu/ipmmu-vmsa: Move num_utlbs to SoC-specific features Geert Uytterhoeven
2019-02-20 15:42   ` Laurent Pinchart
2019-02-20 15:05 ` [PATCH 6/7] iommu/ipmmu-vmsa: Extract hardware context initialization Geert Uytterhoeven
2019-02-20 15:35   ` Laurent Pinchart
2019-02-20 15:43     ` Geert Uytterhoeven
2019-02-20 15:05 ` [PATCH 7/7] iommu/ipmmu-vmsa: Add suspend/resume support Geert Uytterhoeven
2019-02-20 15:42   ` Laurent Pinchart
2019-02-20 16:05     ` Geert Uytterhoeven
2019-02-20 16:11       ` Laurent Pinchart [this message]
2019-02-20 19:47         ` Geert Uytterhoeven
2019-02-22 14:29   ` Robin Murphy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190220161113.GK3516@pendragon.ideasonboard.com \
    --to=laurent.pinchart@ideasonboard.com \
    --cc=damm+renesas@opensource.se \
    --cc=geert+renesas@glider.be \
    --cc=geert@linux-m68k.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=joro@8bytes.org \
    --cc=laurent.pinchart+renesas@ideasonboard.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-renesas-soc@vger.kernel.org \
    --subject='Re: [PATCH 7/7] iommu/ipmmu-vmsa: Add suspend/resume support' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox