interrupt coalescing in LSI HBA controller

interrupt coalescing in LSI HBA controller

[Zheng Da]

Hello,

First, I'm not very sure if this mailing list allows to ask technical
questions. I apologize if it's not a right place to ask questions.

I use one LSI SAS 9207-8e controllers to attach 8 SSDs to my server.
They together can deliver over 300,000 IO per second. I use mpt2sas
driver and MPI-X is enabled. In order to handle so many IOs, I assign
different IRQs to different CPU cores.

I see very high CPU consumption, so I hope to enable interrupt
coalescing to reduce CPU consumption. I monitored interrupts by using
systemtap to count the number of invocations of handle_irq, and found
the number of interrupts delivered by the LSI HBA is more or less the
same as the number of IOs performed on the SSDs. I assume interrupt
coalescing doesn't work in my case.

If so, how do I enable interrupt coalescing in the LSI HBA. I checked
all mpt2sas parameters and none of them are used to control it. The
tool lsiutil doesn't work for this controller. I'm pretty sure the LSI
HBA supports interrupt coalescing. Do I have to use some special
configuration (the Linux kernel or hardware) in order to make it work?

Thanks,
Da

Re: interrupt coalescing in LSI HBA controller

[Baruch Even]

On Fri, Jun 14, 2013 at 7:44 PM, Zheng Da <zhengda1936@gmail.com> wrote:
> Hello,
>
> First, I'm not very sure if this mailing list allows to ask technical
> questions. I apologize if it's not a right place to ask questions.
>
> I use one LSI SAS 9207-8e controllers to attach 8 SSDs to my server.
> They together can deliver over 300,000 IO per second. I use mpt2sas
> driver and MPI-X is enabled. In order to handle so many IOs, I assign
> different IRQs to different CPU cores.
>
> I see very high CPU consumption, so I hope to enable interrupt
> coalescing to reduce CPU consumption. I monitored interrupts by using
> systemtap to count the number of invocations of handle_irq, and found
> the number of interrupts delivered by the LSI HBA is more or less the
> same as the number of IOs performed on the SSDs. I assume interrupt
> coalescing doesn't work in my case.
>
> If so, how do I enable interrupt coalescing in the LSI HBA. I checked
> all mpt2sas parameters and none of them are used to control it. The
> tool lsiutil doesn't work for this controller. I'm pretty sure the LSI
> HBA supports interrupt coalescing. Do I have to use some special
> configuration (the Linux kernel or hardware) in order to make it work?

lsiutil will enable you to read and control the interrupt coalescing on the hba.
It should be possible to find it on lsi's website.

Baruch

Re: interrupt coalescing in LSI HBA controller

[Zheng Da]

Hello,

I have tried lsiutil and it cannot recognize the LSI controllers. I
have talked to the LSI tech support and they told me that lsiutil
doesn't support this type of HBA controllers.
I found lsiutil in some other place. I couldn't find it in on LSI's
website. Do you know where I can download it from LSI's website?

Thanks,
Da

On Mon, Jun 17, 2013 at 2:44 PM, Baruch Even <baruch@ev-en.org> wrote:
> On Fri, Jun 14, 2013 at 7:44 PM, Zheng Da <zhengda1936@gmail.com> wrote:
>> Hello,
>>
>> First, I'm not very sure if this mailing list allows to ask technical
>> questions. I apologize if it's not a right place to ask questions.
>>
>> I use one LSI SAS 9207-8e controllers to attach 8 SSDs to my server.
>> They together can deliver over 300,000 IO per second. I use mpt2sas
>> driver and MPI-X is enabled. In order to handle so many IOs, I assign
>> different IRQs to different CPU cores.
>>
>> I see very high CPU consumption, so I hope to enable interrupt
>> coalescing to reduce CPU consumption. I monitored interrupts by using
>> systemtap to count the number of invocations of handle_irq, and found
>> the number of interrupts delivered by the LSI HBA is more or less the
>> same as the number of IOs performed on the SSDs. I assume interrupt
>> coalescing doesn't work in my case.
>>
>> If so, how do I enable interrupt coalescing in the LSI HBA. I checked
>> all mpt2sas parameters and none of them are used to control it. The
>> tool lsiutil doesn't work for this controller. I'm pretty sure the LSI
>> HBA supports interrupt coalescing. Do I have to use some special
>> configuration (the Linux kernel or hardware) in order to make it work?
>
> lsiutil will enable you to read and control the interrupt coalescing on the hba.
> It should be possible to find it on lsi's website.
>
> Baruch

Re: interrupt coalescing in LSI HBA controller

[Baruch Even]

On Mon, Jun 17, 2013 at 11:34 PM, Zheng Da <zhengda1936@gmail.com> wrote:
> Hello,
>
> I have tried lsiutil and it cannot recognize the LSI controllers. I
> have talked to the LSI tech support and they told me that lsiutil
> doesn't support this type of HBA controllers.
> I found lsiutil in some other place. I couldn't find it in on LSI's
> website. Do you know where I can download it from LSI's website?

I didn't try to find it at lsi, there are other places but if it
didn't work it wouldn't help much to find it in another place.

At work I wrote a utility based on the LSI documentation that works on
similar HBAs but I don't think I'd be able to release that. The
relevant headers are in the Linux kernel and you can try to ask LSI
support for the documentation directly, I obviously can't provide it.

Baruch