* Re: [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver
[not found] ` <1578980874.11996.3.camel@abdul.in.ibm.com>
@ 2020-01-16 17:44 ` Christoph Hellwig
2020-01-17 12:51 ` Abdul Haleem
0 siblings, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2020-01-16 17:44 UTC (permalink / raw)
To: Abdul Haleem
Cc: sachinp, Chaitra P B, linux-scsi, MPT-FusionLinux.pdl, manvanth,
Sathya Prakash, jcmvbkbc, iommu, linux-next, Oliver,
aneesh.kumar, Suganath Prabu Subramani, Brian King, linuxppc-dev
Hi Abdul,
I think the problem is that mpt3sas has some convoluted logic to do
some DMA allocations with a 32-bit coherent mask, and then switches
to a 63 or 64 bit mask, which is not supported by the DMA API.
Can you try the patch below?
---
From 0738b1704ed528497b41b0408325f6828a8e51f6 Mon Sep 17 00:00:00 2001
From: Christoph Hellwig <hch@lst.de>
Date: Thu, 16 Jan 2020 18:31:38 +0100
Subject: mpt3sas: don't change the dma coherent mask after allocations
The DMA layer does not allow changing the DMA coherent mask after
there are outstanding allocations. Stop doing that and always
use a 32-bit coherent DMA mask in mpt3sas.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/scsi/mpt3sas/mpt3sas_base.c | 67 ++++++++---------------------
drivers/scsi/mpt3sas/mpt3sas_base.h | 2 -
2 files changed, 19 insertions(+), 50 deletions(-)
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
index fea3cb6a090b..3b51bed05008 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
@@ -2706,58 +2706,38 @@ _base_build_sg_ieee(struct MPT3SAS_ADAPTER *ioc, void *psge,
static int
_base_config_dma_addressing(struct MPT3SAS_ADAPTER *ioc, struct pci_dev *pdev)
{
- u64 required_mask, coherent_mask;
struct sysinfo s;
- /* Set 63 bit DMA mask for all SAS3 and SAS35 controllers */
- int dma_mask = (ioc->hba_mpi_version_belonged > MPI2_VERSION) ? 63 : 64;
-
- if (ioc->is_mcpu_endpoint)
- goto try_32bit;
+ int dma_mask;
- required_mask = dma_get_required_mask(&pdev->dev);
- if (sizeof(dma_addr_t) == 4 || required_mask == 32)
- goto try_32bit;
-
- if (ioc->dma_mask)
- coherent_mask = DMA_BIT_MASK(dma_mask);
+ if (ioc->is_mcpu_endpoint ||
+ sizeof(dma_addr_t) == 4 ||
+ dma_get_required_mask(&pdev->dev) <= 32)
+ dma_mask = 32;
+ /* Set 63 bit DMA mask for all SAS3 and SAS35 controllers */
+ else if (ioc->hba_mpi_version_belonged > MPI2_VERSION)
+ dma_mask = 63;
else
- coherent_mask = DMA_BIT_MASK(32);
+ dma_mask = 64;
if (dma_set_mask(&pdev->dev, DMA_BIT_MASK(dma_mask)) ||
- dma_set_coherent_mask(&pdev->dev, coherent_mask))
- goto try_32bit;
-
- ioc->base_add_sg_single = &_base_add_sg_single_64;
- ioc->sge_size = sizeof(Mpi2SGESimple64_t);
- ioc->dma_mask = dma_mask;
- goto out;
-
- try_32bit:
- if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)))
+ dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32)))
return -ENODEV;
- ioc->base_add_sg_single = &_base_add_sg_single_32;
- ioc->sge_size = sizeof(Mpi2SGESimple32_t);
- ioc->dma_mask = 32;
- out:
+ if (dma_mask > 32) {
+ ioc->base_add_sg_single = &_base_add_sg_single_64;
+ ioc->sge_size = sizeof(Mpi2SGESimple64_t);
+ } else {
+ ioc->base_add_sg_single = &_base_add_sg_single_32;
+ ioc->sge_size = sizeof(Mpi2SGESimple32_t);
+ }
+
si_meminfo(&s);
ioc_info(ioc, "%d BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (%ld kB)\n",
- ioc->dma_mask, convert_to_kb(s.totalram));
+ dma_mask, convert_to_kb(s.totalram));
return 0;
}
-static int
-_base_change_consistent_dma_mask(struct MPT3SAS_ADAPTER *ioc,
- struct pci_dev *pdev)
-{
- if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(ioc->dma_mask))) {
- if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32)))
- return -ENODEV;
- }
- return 0;
-}
-
/**
* _base_check_enable_msix - checks MSIX capabable.
* @ioc: per adapter object
@@ -5030,14 +5010,6 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc)
total_sz += sz;
} while (ioc->rdpq_array_enable && (++i < ioc->reply_queue_count));
- if (ioc->dma_mask > 32) {
- if (_base_change_consistent_dma_mask(ioc, ioc->pdev) != 0) {
- ioc_warn(ioc, "no suitable consistent DMA mask for %s\n",
- pci_name(ioc->pdev));
- goto out;
- }
- }
-
ioc->scsiio_depth = ioc->hba_queue_depth -
ioc->hi_priority_depth - ioc->internal_depth;
@@ -6965,7 +6937,6 @@ mpt3sas_base_attach(struct MPT3SAS_ADAPTER *ioc)
ioc->smp_affinity_enable = smp_affinity_enable;
ioc->rdpq_array_enable_assigned = 0;
- ioc->dma_mask = 0;
if (ioc->is_aero_ioc)
ioc->base_readl = &_base_readl_aero;
else
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h
index faca0a5e71f8..e57cade1155c 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.h
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
@@ -1011,7 +1011,6 @@ typedef void (*MPT3SAS_FLUSH_RUNNING_CMDS)(struct MPT3SAS_ADAPTER *ioc);
* @ir_firmware: IR firmware present
* @bars: bitmask of BAR's that must be configured
* @mask_interrupts: ignore interrupt
- * @dma_mask: used to set the consistent dma mask
* @pci_access_mutex: Mutex to synchronize ioctl, sysfs show path and
* pci resource handling
* @fault_reset_work_q_name: fw fault work queue
@@ -1185,7 +1184,6 @@ struct MPT3SAS_ADAPTER {
u8 ir_firmware;
int bars;
u8 mask_interrupts;
- int dma_mask;
/* fw fault handler */
char fault_reset_work_q_name[20];
--
2.24.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver
2020-01-16 17:44 ` [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver Christoph Hellwig
@ 2020-01-17 12:51 ` Abdul Haleem
2020-02-25 6:20 ` Abdul Haleem
0 siblings, 1 reply; 5+ messages in thread
From: Abdul Haleem @ 2020-01-17 12:51 UTC (permalink / raw)
To: Christoph Hellwig
Cc: sachinp, Chaitra P B, linux-scsi, MPT-FusionLinux.pdl, manvanth,
Sathya Prakash, jcmvbkbc, iommu, linux-next, Oliver,
aneesh.kumar, Suganath Prabu Subramani, Brian King, linuxppc-dev
On Thu, 2020-01-16 at 09:44 -0800, Christoph Hellwig wrote:
> Hi Abdul,
>
> I think the problem is that mpt3sas has some convoluted logic to do
> some DMA allocations with a 32-bit coherent mask, and then switches
> to a 63 or 64 bit mask, which is not supported by the DMA API.
>
> Can you try the patch below?
Thank you Christoph, with the given patch applied the bug is not seen.
rmmod of mpt3sas driver is successful, no kernel Oops
Reported-and-tested-by: Abdul Haleem <abdhalee@linux.vnet.ibm.com>
>
> ---
> From 0738b1704ed528497b41b0408325f6828a8e51f6 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch@lst.de>
> Date: Thu, 16 Jan 2020 18:31:38 +0100
> Subject: mpt3sas: don't change the dma coherent mask after allocations
>
> The DMA layer does not allow changing the DMA coherent mask after
> there are outstanding allocations. Stop doing that and always
> use a 32-bit coherent DMA mask in mpt3sas.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> drivers/scsi/mpt3sas/mpt3sas_base.c | 67 ++++++++---------------------
> drivers/scsi/mpt3sas/mpt3sas_base.h | 2 -
> 2 files changed, 19 insertions(+), 50 deletions(-)
>
> diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
> index fea3cb6a090b..3b51bed05008 100644
> --- a/drivers/scsi/mpt3sas/mpt3sas_base.c
> +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
> @@ -2706,58 +2706,38 @@ _base_build_sg_ieee(struct MPT3SAS_ADAPTER *ioc, void *psge,
> static int
> _base_config_dma_addressing(struct MPT3SAS_ADAPTER *ioc, struct pci_dev *pdev)
> {
> - u64 required_mask, coherent_mask;
> struct sysinfo s;
> - /* Set 63 bit DMA mask for all SAS3 and SAS35 controllers */
> - int dma_mask = (ioc->hba_mpi_version_belonged > MPI2_VERSION) ? 63 : 64;
> -
> - if (ioc->is_mcpu_endpoint)
> - goto try_32bit;
> + int dma_mask;
>
> - required_mask = dma_get_required_mask(&pdev->dev);
> - if (sizeof(dma_addr_t) == 4 || required_mask == 32)
> - goto try_32bit;
> -
> - if (ioc->dma_mask)
> - coherent_mask = DMA_BIT_MASK(dma_mask);
> + if (ioc->is_mcpu_endpoint ||
> + sizeof(dma_addr_t) == 4 ||
> + dma_get_required_mask(&pdev->dev) <= 32)
> + dma_mask = 32;
> + /* Set 63 bit DMA mask for all SAS3 and SAS35 controllers */
> + else if (ioc->hba_mpi_version_belonged > MPI2_VERSION)
> + dma_mask = 63;
> else
> - coherent_mask = DMA_BIT_MASK(32);
> + dma_mask = 64;
>
> if (dma_set_mask(&pdev->dev, DMA_BIT_MASK(dma_mask)) ||
> - dma_set_coherent_mask(&pdev->dev, coherent_mask))
> - goto try_32bit;
> -
> - ioc->base_add_sg_single = &_base_add_sg_single_64;
> - ioc->sge_size = sizeof(Mpi2SGESimple64_t);
> - ioc->dma_mask = dma_mask;
> - goto out;
> -
> - try_32bit:
> - if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)))
> + dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32)))
> return -ENODEV;
>
> - ioc->base_add_sg_single = &_base_add_sg_single_32;
> - ioc->sge_size = sizeof(Mpi2SGESimple32_t);
> - ioc->dma_mask = 32;
> - out:
> + if (dma_mask > 32) {
> + ioc->base_add_sg_single = &_base_add_sg_single_64;
> + ioc->sge_size = sizeof(Mpi2SGESimple64_t);
> + } else {
> + ioc->base_add_sg_single = &_base_add_sg_single_32;
> + ioc->sge_size = sizeof(Mpi2SGESimple32_t);
> + }
> +
> si_meminfo(&s);
> ioc_info(ioc, "%d BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (%ld kB)\n",
> - ioc->dma_mask, convert_to_kb(s.totalram));
> + dma_mask, convert_to_kb(s.totalram));
>
> return 0;
> }
>
> -static int
> -_base_change_consistent_dma_mask(struct MPT3SAS_ADAPTER *ioc,
> - struct pci_dev *pdev)
> -{
> - if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(ioc->dma_mask))) {
> - if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32)))
> - return -ENODEV;
> - }
> - return 0;
> -}
> -
> /**
> * _base_check_enable_msix - checks MSIX capabable.
> * @ioc: per adapter object
> @@ -5030,14 +5010,6 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc)
> total_sz += sz;
> } while (ioc->rdpq_array_enable && (++i < ioc->reply_queue_count));
>
> - if (ioc->dma_mask > 32) {
> - if (_base_change_consistent_dma_mask(ioc, ioc->pdev) != 0) {
> - ioc_warn(ioc, "no suitable consistent DMA mask for %s\n",
> - pci_name(ioc->pdev));
> - goto out;
> - }
> - }
> -
> ioc->scsiio_depth = ioc->hba_queue_depth -
> ioc->hi_priority_depth - ioc->internal_depth;
>
> @@ -6965,7 +6937,6 @@ mpt3sas_base_attach(struct MPT3SAS_ADAPTER *ioc)
> ioc->smp_affinity_enable = smp_affinity_enable;
>
> ioc->rdpq_array_enable_assigned = 0;
> - ioc->dma_mask = 0;
> if (ioc->is_aero_ioc)
> ioc->base_readl = &_base_readl_aero;
> else
> diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h
> index faca0a5e71f8..e57cade1155c 100644
> --- a/drivers/scsi/mpt3sas/mpt3sas_base.h
> +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
> @@ -1011,7 +1011,6 @@ typedef void (*MPT3SAS_FLUSH_RUNNING_CMDS)(struct MPT3SAS_ADAPTER *ioc);
> * @ir_firmware: IR firmware present
> * @bars: bitmask of BAR's that must be configured
> * @mask_interrupts: ignore interrupt
> - * @dma_mask: used to set the consistent dma mask
> * @pci_access_mutex: Mutex to synchronize ioctl, sysfs show path and
> * pci resource handling
> * @fault_reset_work_q_name: fw fault work queue
> @@ -1185,7 +1184,6 @@ struct MPT3SAS_ADAPTER {
> u8 ir_firmware;
> int bars;
> u8 mask_interrupts;
> - int dma_mask;
>
> /* fw fault handler */
> char fault_reset_work_q_name[20];
--
Regard's
Abdul Haleem
IBM Linux Technology Centre
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver
2020-01-17 12:51 ` Abdul Haleem
@ 2020-02-25 6:20 ` Abdul Haleem
2020-02-25 6:53 ` Sreekanth Reddy via iommu
0 siblings, 1 reply; 5+ messages in thread
From: Abdul Haleem @ 2020-02-25 6:20 UTC (permalink / raw)
To: Christoph Hellwig
Cc: sachinp, Chaitra P B, linux-scsi, MPT-FusionLinux.pdl, manvanth,
Sathya Prakash, jcmvbkbc, iommu, linux-next, Oliver,
aneesh.kumar, Suganath Prabu Subramani, Brian King, linuxppc-dev
On Fri, 2020-01-17 at 18:21 +0530, Abdul Haleem wrote:
> On Thu, 2020-01-16 at 09:44 -0800, Christoph Hellwig wrote:
> > Hi Abdul,
> >
> > I think the problem is that mpt3sas has some convoluted logic to do
> > some DMA allocations with a 32-bit coherent mask, and then switches
> > to a 63 or 64 bit mask, which is not supported by the DMA API.
> >
> > Can you try the patch below?
>
> Thank you Christoph, with the given patch applied the bug is not seen.
>
> rmmod of mpt3sas driver is successful, no kernel Oops
>
> Reported-and-tested-by: Abdul Haleem <abdhalee@linux.vnet.ibm.com>
Hi Christoph,
I see the patch is under discussion, will this be merged upstream any
time soon ? as boot is broken on our machines with out your patch.
--
Regard's
Abdul Haleem
IBM Linux Technology Centre
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver
2020-02-25 6:20 ` Abdul Haleem
@ 2020-02-25 6:53 ` Sreekanth Reddy via iommu
2020-03-20 9:59 ` Abdul Haleem
0 siblings, 1 reply; 5+ messages in thread
From: Sreekanth Reddy via iommu @ 2020-02-25 6:53 UTC (permalink / raw)
To: Abdul Haleem
Cc: sachinp, Chaitra P B, linux-scsi, PDL-MPT-FUSIONLINUX, manvanth,
Sathya Prakash, jcmvbkbc, iommu, linux-next, Oliver,
aneesh.kumar, Suganath Prabu Subramani, Brian King, linuxppc-dev
On Tue, Feb 25, 2020 at 11:51 AM Abdul Haleem
<abdhalee@linux.vnet.ibm.com> wrote:
>
> On Fri, 2020-01-17 at 18:21 +0530, Abdul Haleem wrote:
> > On Thu, 2020-01-16 at 09:44 -0800, Christoph Hellwig wrote:
> > > Hi Abdul,
> > >
> > > I think the problem is that mpt3sas has some convoluted logic to do
> > > some DMA allocations with a 32-bit coherent mask, and then switches
> > > to a 63 or 64 bit mask, which is not supported by the DMA API.
> > >
> > > Can you try the patch below?
> >
> > Thank you Christoph, with the given patch applied the bug is not seen.
> >
> > rmmod of mpt3sas driver is successful, no kernel Oops
> >
> > Reported-and-tested-by: Abdul Haleem <abdhalee@linux.vnet.ibm.com>
>
> Hi Christoph,
>
> I see the patch is under discussion, will this be merged upstream any
> time soon ? as boot is broken on our machines with out your patch.
>
Hi Abdul,
We have posted a new set of patches to fix this issue. This patch set
won't change the DMA Mask on the fly and also won't hardcode the DMA
mask to 32 bit.
[PATCH 0/5] mpt3sas: Fix changing coherent mask after allocation.
This patchset will have below patches, Please review and try with this
patch set.
Suganath Prabu S (5):
mpt3sas: Don't change the dma coherent mask after allocations
mpt3sas: Rename function name is_MSB_are_same
mpt3sas: Code Refactoring.
mpt3sas: Handle RDPQ DMA allocation in same 4g region
mpt3sas: Update version to 33.101.00.00
Regards,
Sreekanth
> --
> Regard's
>
> Abdul Haleem
> IBM Linux Technology Centre
>
>
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver
2020-02-25 6:53 ` Sreekanth Reddy via iommu
@ 2020-03-20 9:59 ` Abdul Haleem
0 siblings, 0 replies; 5+ messages in thread
From: Abdul Haleem @ 2020-03-20 9:59 UTC (permalink / raw)
To: Sreekanth Reddy
Cc: sachinp, Chaitra P B, linux-scsi, PDL-MPT-FUSIONLINUX, manvanth,
Sathya Prakash, jcmvbkbc, iommu, linux-next, Oliver,
aneesh.kumar, Suganath Prabu Subramani, Brian King, linuxppc-dev
On Tue, 2020-02-25 at 12:23 +0530, Sreekanth Reddy wrote:
> On Tue, Feb 25, 2020 at 11:51 AM Abdul Haleem
> <abdhalee@linux.vnet.ibm.com> wrote:
> >
> > On Fri, 2020-01-17 at 18:21 +0530, Abdul Haleem wrote:
> > > On Thu, 2020-01-16 at 09:44 -0800, Christoph Hellwig wrote:
> > > > Hi Abdul,
> > > >
> > > > I think the problem is that mpt3sas has some convoluted logic to do
> > > > some DMA allocations with a 32-bit coherent mask, and then switches
> > > > to a 63 or 64 bit mask, which is not supported by the DMA API.
> > > >
> > > > Can you try the patch below?
> > >
> > > Thank you Christoph, with the given patch applied the bug is not seen.
> > >
> > > rmmod of mpt3sas driver is successful, no kernel Oops
> > >
> > > Reported-and-tested-by: Abdul Haleem <abdhalee@linux.vnet.ibm.com>
> >
> > Hi Christoph,
> >
> > I see the patch is under discussion, will this be merged upstream any
> > time soon ? as boot is broken on our machines with out your patch.
> >
>
> Hi Abdul,
>
> We have posted a new set of patches to fix this issue. This patch set
> won't change the DMA Mask on the fly and also won't hardcode the DMA
> mask to 32 bit.
>
> [PATCH 0/5] mpt3sas: Fix changing coherent mask after allocation.
>
> This patchset will have below patches, Please review and try with this
> patch set.
>
> Suganath Prabu S (5):
> mpt3sas: Don't change the dma coherent mask after allocations
> mpt3sas: Rename function name is_MSB_are_same
> mpt3sas: Code Refactoring.
> mpt3sas: Handle RDPQ DMA allocation in same 4g region
> mpt3sas: Update version to 33.101.00.00
Hi Suganath,
The above patch fixes the issue, driver is loading and unloading with no
kernel oops.
Reported-and-tested-by: Abdul Haleem <abdhalee@linux.vnet.ibm.com>
--
Regard's
Abdul Haleem
IBM Linux Technology Centre
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-03-20 10:00 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1578489498.29952.11.camel@abdul>
[not found] ` <1578560245.30409.0.camel@abdul.in.ibm.com>
[not found] ` <20200109142218.GA16477@infradead.org>
[not found] ` <1578980874.11996.3.camel@abdul.in.ibm.com>
2020-01-16 17:44 ` [linux-next/mainline][bisected 3acac06][ppc] Oops when unloading mpt3sas driver Christoph Hellwig
2020-01-17 12:51 ` Abdul Haleem
2020-02-25 6:20 ` Abdul Haleem
2020-02-25 6:53 ` Sreekanth Reddy via iommu
2020-03-20 9:59 ` Abdul Haleem
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).