linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jarkko Sakkinen <jarkko@kernel.org>
To: Jerry Snitselaar <jsnitsel@redhat.com>,
	Jani Nikula <jani.nikula@linux.intel.com>
Cc: linux-kernel@vger.kernel.org, linux-integrity@vger.kernel.org,
	Jason Gunthorpe <jgg@ziepe.ca>, Peter Huewe <peterhuewe@gmx.de>,
	James Bottomley <James.Bottomley@hansenpartnership.com>,
	Matthew Garrett <mjg59@google.com>,
	Hans de Goede <hdegoede@redhat.com>
Subject: Re: [PATCH v2] tpm_tis: Disable interrupts if interrupt storm detected
Date: Wed, 2 Dec 2020 18:49:31 +0200	[thread overview]
Message-ID: <20201202164931.GA91318@kernel.org> (raw)
In-Reply-To: <878sahmh5w.fsf@redhat.com>

On Tue, Dec 01, 2020 at 12:59:23PM -0700, Jerry Snitselaar wrote:
> 
> Jerry Snitselaar @ 2020-11-30 20:26 MST:
> 
> > Jerry Snitselaar @ 2020-11-30 19:58 MST:
> >
> >> When enabling the interrupt code for the tpm_tis driver we have
> >> noticed some systems have a bios issue causing an interrupt storm to
> >> occur. The issue isn't limited to a single tpm or system manufacturer
> >> so keeping a denylist of systems with the issue isn't optimal. Instead
> >> try to detect the problem occurring, disable interrupts, and revert to
> >> polling when it happens.
> >>
> >> Cc: Jarkko Sakkinen <jarkko@kernel.org>
> >> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> >> Cc: Peter Huewe <peterhuewe@gmx.de>
> >> Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
> >> Cc: Matthew Garrett <mjg59@google.com>
> >> Cc: Hans de Goede <hdegoede@redhat.com>
> >> Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
> >> ---
> >> v2: drop tpm_tis specific workqueue and use just system_wq
> >>
> >> drivers/char/tpm/tpm_tis_core.c | 27 +++++++++++++++++++++++++++
> >>  drivers/char/tpm/tpm_tis_core.h |  2 ++
> >>  2 files changed, 29 insertions(+)
> >>
> >> diff --git a/drivers/char/tpm/tpm_tis_core.c b/drivers/char/tpm/tpm_tis_core.c
> >> index 23b60583928b..72cc8a5a152c 100644
> >> --- a/drivers/char/tpm/tpm_tis_core.c
> >> +++ b/drivers/char/tpm/tpm_tis_core.c
> >> @@ -24,6 +24,8 @@
> >>  #include <linux/wait.h>
> >>  #include <linux/acpi.h>
> >>  #include <linux/freezer.h>
> >> +#include <linux/workqueue.h>
> >> +#include <linux/kernel_stat.h>
> >>  #include "tpm.h"
> >>  #include "tpm_tis_core.h"
> >>  
> >> @@ -745,9 +747,23 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)
> >>  {
> >>  	struct tpm_chip *chip = dev_id;
> >>  	struct tpm_tis_data *priv = dev_get_drvdata(&chip->dev);
> >> +	static bool check_storm = true;
> >> +	static unsigned int check_start;
> >>  	u32 interrupt;
> >>  	int i, rc;
> >>  
> >> +	if (unlikely(check_storm)) {
> >> +		if (!check_start) {
> >> +			check_start = jiffies_to_msecs(jiffies);
> >> +		} else if ((kstat_irqs(priv->irq) > 1000) &&
> >> +			   (jiffies_to_msecs(jiffies) - check_start < 500)) {
> >> +			check_storm = false;
> >> +			schedule_work(&priv->storm_work);
> >> +		} else if (jiffies_to_msecs(jiffies) - check_start >= 500) {
> >> +			check_storm = false;
> >> +		}
> >> +	}
> >> +
> >>  	rc = tpm_tis_read32(priv, TPM_INT_STATUS(priv->locality), &interrupt);
> >>  	if (rc < 0)
> >>  		return IRQ_NONE;
> >> @@ -987,6 +1003,14 @@ static const struct tpm_class_ops tpm_tis = {
> >>  	.clk_enable = tpm_tis_clkrun_enable,
> >>  };
> >>  
> >> +static void tpm_tis_storm_work(struct work_struct *work)
> >> +{
> >> +	struct tpm_tis_data *priv = container_of(work, struct tpm_tis_data, storm_work);
> >> +
> >> +	disable_interrupts(priv->chip);
> >> +	dev_warn(&priv->chip->dev, "Interrupt storm detected, using polling.\n");
> >> +}
> >> +
> >>  int tpm_tis_core_init(struct device *dev, struct tpm_tis_data *priv, int irq,
> >>  		      const struct tpm_tis_phy_ops *phy_ops,
> >>  		      acpi_handle acpi_dev_handle)
> >> @@ -1003,6 +1027,9 @@ int tpm_tis_core_init(struct device *dev, struct tpm_tis_data *priv, int irq,
> >>  	if (IS_ERR(chip))
> >>  		return PTR_ERR(chip);
> >>  
> >> +	priv->chip = chip;
> >> +	INIT_WORK(&priv->storm_work, tpm_tis_storm_work);
> >> +
> >>  #ifdef CONFIG_ACPI
> >>  	chip->acpi_dev_handle = acpi_dev_handle;
> >>  #endif
> >> diff --git a/drivers/char/tpm/tpm_tis_core.h b/drivers/char/tpm/tpm_tis_core.h
> >> index edeb5dc61c95..5630f294dc0c 100644
> >> --- a/drivers/char/tpm/tpm_tis_core.h
> >> +++ b/drivers/char/tpm/tpm_tis_core.h
> >> @@ -95,6 +95,8 @@ struct tpm_tis_data {
> >>  	u16 clkrun_enabled;
> >>  	wait_queue_head_t int_queue;
> >>  	wait_queue_head_t read_queue;
> >> +	struct work_struct storm_work;
> >> +	struct tpm_chip *chip;
> >>  	const struct tpm_tis_phy_ops *phy_ops;
> >>  	unsigned short rng_quality;
> >>  };
> >
> > I've tested this with the Intel platform that has an Infineon chip that
> > I found the other week. It works, but isn't the complete fix. With this
> > on top of James' patchset I sometimes see the message "Lost Interrupt
> > waiting for TPM stat", so I guess there needs to be a check in
> > wait_for_tpm_stat and request_locality to see if interrupts were
> > disabled when the wait_event_interruptible_timeout call times out.
> 
> As kernel test robot pointed out. kstat_irqs isn't visible when tpm_tis
> builds as a module. It looks like it is only called by kstat_irq_usrs,
> and that is only by the fs/proc code. I have a patch to export it, but
> the i915 driver open codes their own version instead of using it. Is
> there any reason not to export it?

If you add a patch that exports it, then for coherency it'd be better to
also patch i915 driver. Jani?

/Jarkko

  reply	other threads:[~2020-12-02 16:50 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-30 23:23 [PATCH] tpm_tis: Disable interrupts if interrupt storm detected Jerry Snitselaar
2020-12-01  2:58 ` [PATCH v2] " Jerry Snitselaar
2020-12-01  3:26   ` Jerry Snitselaar
2020-12-01 19:59     ` Jerry Snitselaar
2020-12-02 16:49       ` Jarkko Sakkinen [this message]
2020-12-03  0:02         ` Jerry Snitselaar
2020-12-03  6:11           ` Jerry Snitselaar
2020-12-03 20:14             ` Jerry Snitselaar
2020-12-03 21:05               ` James Bottomley
2020-12-04 21:51                 ` Jerry Snitselaar
2020-12-05  0:06                   ` James Bottomley
2020-12-04  1:59             ` Jarkko Sakkinen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201202164931.GA91318@kernel.org \
    --to=jarkko@kernel.org \
    --cc=James.Bottomley@hansenpartnership.com \
    --cc=hdegoede@redhat.com \
    --cc=jani.nikula@linux.intel.com \
    --cc=jgg@ziepe.ca \
    --cc=jsnitsel@redhat.com \
    --cc=linux-integrity@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mjg59@google.com \
    --cc=peterhuewe@gmx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).