From: Christoph Hellwig <hch@lst.de>
To: "Cédric Le Goater" <clg@kaod.org>
Cc: "Christoph Hellwig" <hch@lst.de>,
iommu@lists.linux-foundation.org,
"Alexey Kardashevskiy" <aik@ozlabs.ru>,
"Björn Töpel" <bjorn.topel@gmail.com>,
"Daniel Borkmann" <daniel@iogearbox.net>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Joerg Roedel" <joro@8bytes.org>,
"Robin Murphy" <robin.murphy@arm.com>,
linux-kernel@vger.kernel.org,
"Jesper Dangaard Brouer" <brouer@redhat.com>,
linuxppc-dev@lists.ozlabs.org,
"Lu Baolu" <baolu.lu@linux.intel.com>,
"Oliver O'Halloran" <oohall@gmail.com>,
"Michael Ellerman" <mpe@ellerman.id.au>,
aacraid@microsemi.com
Subject: Re: [PATCH 5/5] powerpc: use the generic dma_ops_bypass mode
Date: Mon, 31 Aug 2020 08:40:38 +0200 [thread overview]
Message-ID: <20200831064038.GB27617@lst.de> (raw)
In-Reply-To: <505bcc1d-01a7-9655-88e1-ebddd0b94d56@kaod.org>
On Sun, Aug 30, 2020 at 11:04:21AM +0200, Cédric Le Goater wrote:
> Hello,
>
> On 7/8/20 5:24 PM, Christoph Hellwig wrote:
> > Use the DMA API bypass mechanism for direct window mappings. This uses
> > common code and speed up the direct mapping case by avoiding indirect
> > calls just when not using dma ops at all. It also fixes a problem where
> > the sync_* methods were using the bypass check for DMA allocations, but
> > those are part of the streaming ops.
> >
> > Note that this patch loses the DMA_ATTR_WEAK_ORDERING override, which
> > has never been well defined, as is only used by a few drivers, which
> > IIRC never showed up in the typical Cell blade setups that are affected
> > by the ordering workaround.
> >
> > Fixes: efd176a04bef ("powerpc/pseries/dma: Allow SWIOTLB")
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > ---
> > arch/powerpc/Kconfig | 1 +
> > arch/powerpc/include/asm/device.h | 5 --
> > arch/powerpc/kernel/dma-iommu.c | 90 ++++---------------------------
> > 3 files changed, 10 insertions(+), 86 deletions(-)
>
> I am seeing corruptions on a couple of POWER9 systems (boston) when
> stressed with IO. stress-ng gives some results but I have first seen
> it when compiling the kernel in a guest and this is still the best way
> to raise the issue.
>
> These systems have of a SAS Adaptec controller :
>
> 0003:01:00.0 Serial Attached SCSI controller: Adaptec Series 8 12G SAS/PCIe 3 (rev 01)
>
> When the failure occurs, the POWERPC EEH interrupt fires and dumps
> lowlevel PHB4 registers among which :
>
> [ 2179.251069490,3] PHB#0003[0:3]: phbErrorStatus = 0000028000000000
> [ 2179.251117476,3] PHB#0003[0:3]: phbFirstErrorStatus = 0000020000000000
>
> The bits raised identify a PPC 'TCE' error, which means it is related
> to DMAs. See below for more details.
>
>
> Reverting this patch "fixes" the issue but it is probably else where,
> in some other layers or in the aacraid driver. How should I proceed
> to get more information ?
The aacraid DMA masks look like a mess. Can you try the hack
below and see it it helps?
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 769af4ca9ca97e..79c6b744dbb66c 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -2228,18 +2228,6 @@ int aac_get_adapter_info(struct aac_dev* dev)
expose_physicals = 0;
}
- if (dev->dac_support) {
- if (!pci_set_dma_mask(dev->pdev, DMA_BIT_MASK(64))) {
- if (!dev->in_reset)
- dev_info(&dev->pdev->dev, "64 Bit DAC enabled\n");
- } else if (!pci_set_dma_mask(dev->pdev, DMA_BIT_MASK(32))) {
- dev_info(&dev->pdev->dev, "DMA mask set failed, 64 Bit DAC disabled\n");
- dev->dac_support = 0;
- } else {
- dev_info(&dev->pdev->dev, "No suitable DMA available\n");
- rcode = -ENOMEM;
- }
- }
/*
* Deal with configuring for the individualized limits of each packet
* interface.
diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c
index adbdc3b7c7a706..dbb23b351a4e7d 100644
--- a/drivers/scsi/aacraid/commsup.c
+++ b/drivers/scsi/aacraid/commsup.c
@@ -1479,7 +1479,6 @@ static int _aac_reset_adapter(struct aac_dev *aac, int forced, u8 reset_type)
struct Scsi_Host *host = aac->scsi_host_ptr;
int jafo = 0;
int bled;
- u64 dmamask;
int num_of_fibs = 0;
/*
@@ -1558,22 +1557,7 @@ static int _aac_reset_adapter(struct aac_dev *aac, int forced, u8 reset_type)
kfree(aac->fsa_dev);
aac->fsa_dev = NULL;
- dmamask = DMA_BIT_MASK(32);
quirks = aac_get_driver_ident(index)->quirks;
- if (quirks & AAC_QUIRK_31BIT)
- retval = pci_set_dma_mask(aac->pdev, dmamask);
- else if (!(quirks & AAC_QUIRK_SRC))
- retval = pci_set_dma_mask(aac->pdev, dmamask);
- else
- retval = pci_set_consistent_dma_mask(aac->pdev, dmamask);
-
- if (quirks & AAC_QUIRK_31BIT && !retval) {
- dmamask = DMA_BIT_MASK(31);
- retval = pci_set_consistent_dma_mask(aac->pdev, dmamask);
- }
-
- if (retval)
- goto out;
if ((retval = (*(aac_get_driver_ident(index)->init))(aac)))
goto out;
diff --git a/drivers/scsi/aacraid/linit.c b/drivers/scsi/aacraid/linit.c
index 8588da0a065551..d897a9d59e24a1 100644
--- a/drivers/scsi/aacraid/linit.c
+++ b/drivers/scsi/aacraid/linit.c
@@ -1634,8 +1634,6 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
struct list_head *insert = &aac_devices;
int error;
int unique_id = 0;
- u64 dmamask;
- int mask_bits = 0;
extern int aac_sync_mode;
/*
@@ -1658,33 +1656,6 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
if (error)
goto out;
- if (!(aac_drivers[index].quirks & AAC_QUIRK_SRC)) {
- error = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
- if (error) {
- dev_err(&pdev->dev, "PCI 32 BIT dma mask set failed");
- goto out_disable_pdev;
- }
- }
-
- /*
- * If the quirk31 bit is set, the adapter needs adapter
- * to driver communication memory to be allocated below 2gig
- */
- if (aac_drivers[index].quirks & AAC_QUIRK_31BIT) {
- dmamask = DMA_BIT_MASK(31);
- mask_bits = 31;
- } else {
- dmamask = DMA_BIT_MASK(32);
- mask_bits = 32;
- }
-
- error = pci_set_consistent_dma_mask(pdev, dmamask);
- if (error) {
- dev_err(&pdev->dev, "PCI %d B consistent dma mask set failed\n"
- , mask_bits);
- goto out_disable_pdev;
- }
-
pci_set_master(pdev);
shost = scsi_host_alloc(&aac_driver_template, sizeof(struct aac_dev));
next prev parent reply other threads:[~2020-08-31 6:40 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-08 15:24 generic DMA bypass flag v4 Christoph Hellwig
2020-07-08 15:24 ` [PATCH 1/5] dma-mapping: move the remaining DMA API calls out of line Christoph Hellwig
2020-07-08 15:24 ` [PATCH 2/5] dma-mapping: inline the fast path dma-direct calls Christoph Hellwig
2020-07-08 15:24 ` [PATCH 3/5] dma-mapping: make support for dma ops optional Christoph Hellwig
2020-07-18 17:17 ` Guenter Roeck
2020-07-20 6:20 ` Christoph Hellwig
2020-07-08 15:24 ` [PATCH 4/5] dma-mapping: add a dma_ops_bypass flag to struct device Christoph Hellwig
2020-07-13 4:59 ` Alexey Kardashevskiy
2020-07-14 7:07 ` Christoph Hellwig
2020-07-14 7:12 ` Alexey Kardashevskiy
2020-07-08 15:24 ` [PATCH 5/5] powerpc: use the generic dma_ops_bypass mode Christoph Hellwig
2020-08-30 9:04 ` Cédric Le Goater
2020-08-31 6:40 ` Christoph Hellwig [this message]
2020-08-31 7:19 ` Cédric Le Goater
2020-09-05 15:45 ` Alexey Kardashevskiy
2020-07-10 13:26 ` generic DMA bypass flag v4 Jesper Dangaard Brouer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200831064038.GB27617@lst.de \
--to=hch@lst.de \
--cc=aacraid@microsemi.com \
--cc=aik@ozlabs.ru \
--cc=baolu.lu@linux.intel.com \
--cc=bjorn.topel@gmail.com \
--cc=brouer@redhat.com \
--cc=clg@kaod.org \
--cc=daniel@iogearbox.net \
--cc=gregkh@linuxfoundation.org \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=oohall@gmail.com \
--cc=robin.murphy@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).