aacraid: kernel: AAC: Host adapter dead -1 (bisected)

aacraid: kernel: AAC: Host adapter dead -1 (bisected)

[Arkadiusz Miskiewicz]

Hi.

There is a bug with handling of adaptec raid cards (in my case it is Adaptec 
3405) where kernel logs hundreds of "AAC: Host adapter dead -1" messages.

Bug was reported previously on lkml but there was no progres in solving it.

There is also bugzilla entry:
https://bugzilla.kernel.org/show_bug.cgi?id=151661

I've bisected that to commit bellow and indeed, reverting it from kernel 4.9.3 
makes messages go away.

Could anyone at microsemi look at this regression?

Thanks

commit 78cbccd3bd683c295a44af8050797dc4a41376ff
Author: Raghava Aditya Renukunta <RaghavaAditya.Renukunta@microsemi.com>
Date:   Mon Apr 25 23:32:37 2016 -0700

    aacraid: Fix for KDUMP driver hang

    When KDUMP is triggered the driver first talks to the firmware in INTX
    mode, but the adapter firmware is still in MSIX mode. Therefore the first
    driver command hangs since the driver is waiting for an INTX response and
    firmware gives a MSIX response. If when the OS is installed on a RAID
    drive created by the adapter KDUMP will hang since the driver does not
    receive a response in sync mode.

    Fixed by: Change the firmware to INTX mode if it is in MSIX mode before
    sending the first sync command.

    Cc: stable@vger.kernel.org
    Signed-off-by: Raghava Aditya Renukunta 
<RaghavaAditya.Renukunta@microsemi.com>
    Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
    Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

my hardware:
02:0e.0 RAID bus controller [0104]: Adaptec AAC-RAID [9005:0285]
        Subsystem: Adaptec 3405 [9005:02bb]
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping+ SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
<TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 32 (250ns min, 250ns max), Cache Line Size: 4 bytes
        Interrupt: pin A routed to IRQ 18
        Region 0: Memory at fba00000 (64-bit, non-prefetchable) [size=2M]
        [virtual] Expansion ROM at fbc00000 [disabled] [size=256K]
        Capabilities: [c0] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA 
PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [d0] MSI: Enable- Count=1/2 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [e0] PCI-X non-bridge device
                Command: DPERE- ERO- RBC=512 OST=4
                Status: Dev=02:0e.0 64bit+ 133MHz+ SCD- USC- DC=bridge 
DMMRBC=1024 DMOST=4 DMCRS=16 RSCEM- 266MHz- 533MHz-
        Kernel driver in use: aacraid
        Kernel modules: aacraid

[    1.956009] Adaptec aacraid driver 1.2-1[41066]-ms
[    2.164584] AAC0: kernel 5.2-0[17342] Aug  4 2010
[    2.164633] AAC0: monitor 5.2-0[17342]
[    2.164676] AAC0: bios 5.2-0[17342]
[    2.164719] AAC0: serial 7C46114103A
[    2.164761] AAC0: Non-DASD support enabled.
[    2.164804] AAC0: 64bit support enabled.
[    2.164846] AAC0: 64 Bit DAC enabled
[    2.177929] scsi host6: aacraid


-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

Re: aacraid: kernel: AAC: Host adapter dead -1 (bisected)

[Arkadiusz Miskiewicz]

On Tuesday 17 of January 2017, Dave Carroll wrote:
> > Hi.
> > 
> > There is a bug with handling of adaptec raid cards (in my case it is
> > Adaptec 3405) where kernel logs hundreds of "AAC: Host adapter dead -1"
> > messages.
> > 
> > Bug was reported previously on lkml but there was no progres in solving
> > it.
> > 
> > There is also bugzilla entry:
> > https://bugzilla.kernel.org/show_bug.cgi?id=151661
> > 
> > I've bisected that to commit bellow and indeed, reverting it from kernel
> > 4.9.3 makes messages go away.
> > 
> > Could anyone at microsemi look at this regression?
> > 
> > Thanks
> 
> Hi Arkadiusz,
> 
> Thanks for your effort in determining the cause of the issue. It makes
> sense now that the patch should have been included in controller specific
> code, rather than common code.
> 
> I will prepare a patch for this, and if you are willing to test it, that
> would be great!

Great!

I have dedicated machine for testing this, so yes - I'll test.

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

Re: aacraid: kernel: AAC: Host adapter dead -1 (bisected)

[Greg Kroah-Hartman]

On Fri, Feb 10, 2017 at 01:45:06PM +0300, Andrey Melnikov wrote:
> Cc: linux-scsi@vger.kernel.org
> 
> 2017-02-10 13:24 GMT+03:00 Greg Kroah-Hartman <gregkh@linuxfoundation.org>:
> > On Fri, Feb 10, 2017 at 02:25:26AM +0300, Andrey Jr. Melnikov wrote:
> >> In article <201701151205.37563.a.miskiewicz@gmail.com> you wrote:
> >> > Newsgroups: gmane.linux.kernel
> >>
> >>
> >> > Hi.
> >>
> >> > There is a bug with handling of adaptec raid cards (in my case it is Adaptec
> >> > 3405) where kernel logs hundreds of "AAC: Host adapter dead -1" messages.
> >>
> >> > Bug was reported previously on lkml but there was no progres in solving it.
> >>
> >> > There is also bugzilla entry:
> >> > https://bugzilla.kernel.org/show_bug.cgi?id=151661
> >>
> >> > I've bisected that to commit bellow and indeed, reverting it from kernel 4.9.3
> >> > makes messages go away.
> >>
> >>
> >> Don't try to switch Adaptec 3405/3805 RAID cards to MSI-X interrupt mode.
> >> Fix https://bugzilla.kernel.org/show_bug.cgi?id=151661
> >>
> >> Signed-off-by: Andrey Jr. Melnikov <temnota.am@gmail.com>
> >>
> >> ---
> >>
> >> diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h
> >> index 969c312de1be..2ad8403dea40 100644
> >> --- a/drivers/scsi/aacraid/aacraid.h
> >> +++ b/drivers/scsi/aacraid/aacraid.h
> >
> > <snip>
> >
> > Why are you sending this to me and not the scsi developers who can
> > actually do something with this patch?
> 
> Bug in bugzilla open half year ago, microsemi maintainer slowly read
> his fine docs about his hardware, broken driver fills our log with
> useless messages every 10 seconds.
> So, make decision - apply this patch to stable 4.9.x/4.4.x tree or
> revert commit 78cbccd3bd683c295a44af8050797dc4a41376ff from it.

I don't understand, that's not how the stable kernels work, please read
Documentation/stable_kernel_rules.txt for how the process works.  Please
get a patch accepted into Linus's tree and then we will be glad to apply
it to the stable kernel trees.

thanks,

greg k-h

Re: aacraid: kernel: AAC: Host adapter dead -1 (bisected)

[Andrey Melnikov]

Cc: linux-scsi@vger.kernel.org

2017-02-10 13:24 GMT+03:00 Greg Kroah-Hartman <gregkh@linuxfoundation.org>:
> On Fri, Feb 10, 2017 at 02:25:26AM +0300, Andrey Jr. Melnikov wrote:
>> In article <201701151205.37563.a.miskiewicz@gmail.com> you wrote:
>> > Newsgroups: gmane.linux.kernel
>>
>>
>> > Hi.
>>
>> > There is a bug with handling of adaptec raid cards (in my case it is Adaptec
>> > 3405) where kernel logs hundreds of "AAC: Host adapter dead -1" messages.
>>
>> > Bug was reported previously on lkml but there was no progres in solving it.
>>
>> > There is also bugzilla entry:
>> > https://bugzilla.kernel.org/show_bug.cgi?id=151661
>>
>> > I've bisected that to commit bellow and indeed, reverting it from kernel 4.9.3
>> > makes messages go away.
>>
>>
>> Don't try to switch Adaptec 3405/3805 RAID cards to MSI-X interrupt mode.
>> Fix https://bugzilla.kernel.org/show_bug.cgi?id=151661
>>
>> Signed-off-by: Andrey Jr. Melnikov <temnota.am@gmail.com>
>>
>> ---
>>
>> diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h
>> index 969c312de1be..2ad8403dea40 100644
>> --- a/drivers/scsi/aacraid/aacraid.h
>> +++ b/drivers/scsi/aacraid/aacraid.h
>
> <snip>
>
> Why are you sending this to me and not the scsi developers who can
> actually do something with this patch?

Bug in bugzilla open half year ago, microsemi maintainer slowly read
his fine docs about his hardware, broken driver fills our log with
useless messages every 10 seconds.
So, make decision - apply this patch to stable 4.9.x/4.4.x tree or
revert commit 78cbccd3bd683c295a44af8050797dc4a41376ff from it.

Re: aacraid: kernel: AAC: Host adapter dead -1 (bisected)

[Greg Kroah-Hartman]

On Fri, Feb 10, 2017 at 02:25:26AM +0300, Andrey Jr. Melnikov wrote:
> In article <201701151205.37563.a.miskiewicz@gmail.com> you wrote:
> > Newsgroups: gmane.linux.kernel
> 
> 
> > Hi.
> 
> > There is a bug with handling of adaptec raid cards (in my case it is Adaptec 
> > 3405) where kernel logs hundreds of "AAC: Host adapter dead -1" messages.
> 
> > Bug was reported previously on lkml but there was no progres in solving it.
> 
> > There is also bugzilla entry:
> > https://bugzilla.kernel.org/show_bug.cgi?id=151661
> 
> > I've bisected that to commit bellow and indeed, reverting it from kernel 4.9.3 
> > makes messages go away.
> 
> 
> Don't try to switch Adaptec 3405/3805 RAID cards to MSI-X interrupt mode.
> Fix https://bugzilla.kernel.org/show_bug.cgi?id=151661
> 
> Signed-off-by: Andrey Jr. Melnikov <temnota.am@gmail.com>
> 
> ---
> 
> diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h
> index 969c312de1be..2ad8403dea40 100644
> --- a/drivers/scsi/aacraid/aacraid.h
> +++ b/drivers/scsi/aacraid/aacraid.h

<snip>

Why are you sending this to me and not the scsi developers who can
actually do something with this patch?

Please fix up and resend.

thanks,

greg k-h

Re: aacraid: kernel: AAC: Host adapter dead -1 (bisected)

[Andrey Jr. Melnikov]

In article <201701151205.37563.a.miskiewicz@gmail.com> you wrote:
> Newsgroups: gmane.linux.kernel


> Hi.

> There is a bug with handling of adaptec raid cards (in my case it is Adaptec 
> 3405) where kernel logs hundreds of "AAC: Host adapter dead -1" messages.

> Bug was reported previously on lkml but there was no progres in solving it.

> There is also bugzilla entry:
> https://bugzilla.kernel.org/show_bug.cgi?id=151661

> I've bisected that to commit bellow and indeed, reverting it from kernel 4.9.3 
> makes messages go away.


Don't try to switch Adaptec 3405/3805 RAID cards to MSI-X interrupt mode.
Fix https://bugzilla.kernel.org/show_bug.cgi?id=151661

Signed-off-by: Andrey Jr. Melnikov <temnota.am@gmail.com>

---

diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h
index 969c312de1be..2ad8403dea40 100644
--- a/drivers/scsi/aacraid/aacraid.h
+++ b/drivers/scsi/aacraid/aacraid.h
@@ -12,6 +12,9 @@
  *              D E F I N E S
  *----------------------------------------------------------------------------*/
 
+#define AAC_SUBID_3805		0x02bc
+#define AAC_SUBID_3405		0x02bb
+
 #define AAC_MAX_MSIX		32	/* vectors */
 #define AAC_PCI_MSI_ENABLE	0x8000
 
diff --git a/drivers/scsi/aacraid/comminit.c b/drivers/scsi/aacraid/comminit.c
index 341ea327ae79..a61138504927 100644
--- a/drivers/scsi/aacraid/comminit.c
+++ b/drivers/scsi/aacraid/comminit.c
@@ -52,6 +52,11 @@ static inline int aac_is_msix_mode(struct aac_dev *dev)
 {
 	u32 status;
 
+	/* Don't allow switch 3405/3805 cards to MSI-X interrupt mode */
+	if (dev->pdev->subsystem_device == AAC_SUBID_3405 ||
+	    dev->pdev->subsystem_device == AAC_SUBID_3405)
+		return 0;
+
 	status = src_readl(dev, MUnit.OMR);
 	return (status & AAC_INT_MODE_MSIX);
 }