Help: MSI DWC connecting with a MSIx Endpoint

Help: MSI DWC connecting with a MSIx Endpoint

[Joao Pinto]

Hi Mark,

I am facing a challenge in a setup I must bring up that is composed by a
Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox.
The endpoint probe fails because it is not capable of enabling msix.

In order to get more info about it, I added some debug messages to the msix
enable process and got the following:

MSIX DBG[mlx5_enable_msix]: Num. EQs: 256
MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4
MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4
MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1
MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3
MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0
MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1
MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2
MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3
MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range
MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0]
MSIX DBG[__pci_enable_msix]: NVec: 4
MSIX DBG[__pci_enable_msix]: Num. Entries: 64
MSIX DBG[__pci_enable_msix]: Checking invalid entries
MSIX DBG[msix_capability_init]: Ensure MSI is disable
MSIX DBG[msix_capability_init]: Getting MSIX Capability
MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region
MSIX DBG[msix_capability_init]: Setup Entries
MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0
MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1
MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2
MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3
MSIX DBG[msix_capability_init]: Setup IRQS
MSIX DBG[pci_msi_setup_msi_irqs]: Inside!
MSIX DBG[arch_setup_msi_irqs]: Inside!
MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs
MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22
mlx5_core 0000:01:00.0: enable msix failed

The conclusion I came up was that the msix setup in the endpoint fails because
in the pcie-designware-host.c we are rejecting msix as you can see here:

/* MSI-X interrupts are not supported */
if (type == PCI_CAP_ID_MSIX)
        return -EINVAL;

Did you ever came across this problem?

Thanks,
Joao

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Marc Zyngier]

Hi Joao,

On 20/03/17 17:30, Joao Pinto wrote:
> Hi Mark,
> 
> I am facing a challenge in a setup I must bring up that is composed by a
> Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox.
> The endpoint probe fails because it is not capable of enabling msix.
> 
> In order to get more info about it, I added some debug messages to the msix
> enable process and got the following:
> 
> MSIX DBG[mlx5_enable_msix]: Num. EQs: 256
> MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4
> MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4
> MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1
> MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3
> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0
> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1
> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2
> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3
> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range
> MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0]
> MSIX DBG[__pci_enable_msix]: NVec: 4
> MSIX DBG[__pci_enable_msix]: Num. Entries: 64
> MSIX DBG[__pci_enable_msix]: Checking invalid entries
> MSIX DBG[msix_capability_init]: Ensure MSI is disable
> MSIX DBG[msix_capability_init]: Getting MSIX Capability
> MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region
> MSIX DBG[msix_capability_init]: Setup Entries
> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0
> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1
> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2
> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3
> MSIX DBG[msix_capability_init]: Setup IRQS
> MSIX DBG[pci_msi_setup_msi_irqs]: Inside!
> MSIX DBG[arch_setup_msi_irqs]: Inside!
> MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs
> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22
> mlx5_core 0000:01:00.0: enable msix failed
> 
> The conclusion I came up was that the msix setup in the endpoint fails because
> in the pcie-designware-host.c we are rejecting msix as you can see here:
> 
> /* MSI-X interrupts are not supported */
> if (type == PCI_CAP_ID_MSIX)
>         return -EINVAL;
> 
> Did you ever came across this problem?

I don't have access to any system using this IP, unfortunately. This
looks like an arbitrary limitation of the host controller driver, as
there isn't much difference between MSI and MSI-X at the MSI controller
level, other than the reduced number of interrupts.

Unfortunately, this driver seems to rely on rather old APIs, and could
use some care and attention. I'm pretty sure that converting this driver
to the generic MSI API would solve this problem entirely, but I'm afraid
I don't have the bandwidth to do so.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Joao Pinto]

Hi Mark,

Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu:
> Hi Joao,
> 
> On 20/03/17 17:30, Joao Pinto wrote:
>> Hi Mark,
>>
>> I am facing a challenge in a setup I must bring up that is composed by a
>> Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox.
>> The endpoint probe fails because it is not capable of enabling msix.
>>
>> In order to get more info about it, I added some debug messages to the msix
>> enable process and got the following:
>>
>> MSIX DBG[mlx5_enable_msix]: Num. EQs: 256
>> MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4
>> MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4
>> MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1
>> MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3
>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0
>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1
>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2
>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3
>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range
>> MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0]
>> MSIX DBG[__pci_enable_msix]: NVec: 4
>> MSIX DBG[__pci_enable_msix]: Num. Entries: 64
>> MSIX DBG[__pci_enable_msix]: Checking invalid entries
>> MSIX DBG[msix_capability_init]: Ensure MSI is disable
>> MSIX DBG[msix_capability_init]: Getting MSIX Capability
>> MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region
>> MSIX DBG[msix_capability_init]: Setup Entries
>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0
>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1
>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2
>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3
>> MSIX DBG[msix_capability_init]: Setup IRQS
>> MSIX DBG[pci_msi_setup_msi_irqs]: Inside!
>> MSIX DBG[arch_setup_msi_irqs]: Inside!
>> MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs
>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22
>> mlx5_core 0000:01:00.0: enable msix failed
>>
>> The conclusion I came up was that the msix setup in the endpoint fails because
>> in the pcie-designware-host.c we are rejecting msix as you can see here:
>>
>> /* MSI-X interrupts are not supported */
>> if (type == PCI_CAP_ID_MSIX)
>>         return -EINVAL;
>>
>> Did you ever came across this problem?
> 
> I don't have access to any system using this IP, unfortunately. This
> looks like an arbitrary limitation of the host controller driver, as
> there isn't much difference between MSI and MSI-X at the MSI controller
> level, other than the reduced number of interrupts.
> 
> Unfortunately, this driver seems to rely on rather old APIs, and could
> use some care and attention. I'm pretty sure that converting this driver
> to the generic MSI API would solve this problem entirely, but I'm afraid
> I don't have the bandwidth to do so.

I volunteer to do it, no problem, but I will need some guidance and examples.
Could you give me some hints to get started?

> 
> Thanks,
> 
> 	M.
> 

Thanks.

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Marc Zyngier]

On 21/03/17 10:27, Joao Pinto wrote:
> 
> Hi Mark,
> 
> Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu:
>> Hi Joao,
>>
>> On 20/03/17 17:30, Joao Pinto wrote:
>>> Hi Mark,
>>>
>>> I am facing a challenge in a setup I must bring up that is composed by a
>>> Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox.
>>> The endpoint probe fails because it is not capable of enabling msix.
>>>
>>> In order to get more info about it, I added some debug messages to the msix
>>> enable process and got the following:
>>>
>>> MSIX DBG[mlx5_enable_msix]: Num. EQs: 256
>>> MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4
>>> MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4
>>> MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1
>>> MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3
>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0
>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1
>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2
>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3
>>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range
>>> MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0]
>>> MSIX DBG[__pci_enable_msix]: NVec: 4
>>> MSIX DBG[__pci_enable_msix]: Num. Entries: 64
>>> MSIX DBG[__pci_enable_msix]: Checking invalid entries
>>> MSIX DBG[msix_capability_init]: Ensure MSI is disable
>>> MSIX DBG[msix_capability_init]: Getting MSIX Capability
>>> MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region
>>> MSIX DBG[msix_capability_init]: Setup Entries
>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0
>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1
>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2
>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3
>>> MSIX DBG[msix_capability_init]: Setup IRQS
>>> MSIX DBG[pci_msi_setup_msi_irqs]: Inside!
>>> MSIX DBG[arch_setup_msi_irqs]: Inside!
>>> MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs
>>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22
>>> mlx5_core 0000:01:00.0: enable msix failed
>>>
>>> The conclusion I came up was that the msix setup in the endpoint fails because
>>> in the pcie-designware-host.c we are rejecting msix as you can see here:
>>>
>>> /* MSI-X interrupts are not supported */
>>> if (type == PCI_CAP_ID_MSIX)
>>>         return -EINVAL;
>>>
>>> Did you ever came across this problem?
>>
>> I don't have access to any system using this IP, unfortunately. This
>> looks like an arbitrary limitation of the host controller driver, as
>> there isn't much difference between MSI and MSI-X at the MSI controller
>> level, other than the reduced number of interrupts.
>>
>> Unfortunately, this driver seems to rely on rather old APIs, and could
>> use some care and attention. I'm pretty sure that converting this driver
>> to the generic MSI API would solve this problem entirely, but I'm afraid
>> I don't have the bandwidth to do so.
> 
> I volunteer to do it, no problem, but I will need some guidance and examples.
> Could you give me some hints to get started?

A very simple example using the current API is pcie-altera-msi.c (some
others are much more intricate), and should give you a good picture of
how PCI/MSI, generic MSI and MSI controllers interact.

Start with understanding how the various layers interact via their own
IRQ domains, and implement their own IRQ chip. Happy to answer questions
(I'm sure you'll have some).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Joao Pinto]

Às 10:41 AM de 3/21/2017, Marc Zyngier escreveu:
> On 21/03/17 10:27, Joao Pinto wrote:
>>
>> Hi Mark,
>>
>> Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu:
>>> Hi Joao,
>>>
>>> On 20/03/17 17:30, Joao Pinto wrote:
>>>> Hi Mark,
>>>>
>>>> I am facing a challenge in a setup I must bring up that is composed by a
>>>> Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox.
>>>> The endpoint probe fails because it is not capable of enabling msix.
>>>>
>>>> In order to get more info about it, I added some debug messages to the msix
>>>> enable process and got the following:
>>>>
>>>> MSIX DBG[mlx5_enable_msix]: Num. EQs: 256
>>>> MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4
>>>> MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4
>>>> MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1
>>>> MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3
>>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0
>>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1
>>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2
>>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3
>>>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range
>>>> MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0]
>>>> MSIX DBG[__pci_enable_msix]: NVec: 4
>>>> MSIX DBG[__pci_enable_msix]: Num. Entries: 64
>>>> MSIX DBG[__pci_enable_msix]: Checking invalid entries
>>>> MSIX DBG[msix_capability_init]: Ensure MSI is disable
>>>> MSIX DBG[msix_capability_init]: Getting MSIX Capability
>>>> MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region
>>>> MSIX DBG[msix_capability_init]: Setup Entries
>>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0
>>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1
>>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2
>>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3
>>>> MSIX DBG[msix_capability_init]: Setup IRQS
>>>> MSIX DBG[pci_msi_setup_msi_irqs]: Inside!
>>>> MSIX DBG[arch_setup_msi_irqs]: Inside!
>>>> MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs
>>>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22
>>>> mlx5_core 0000:01:00.0: enable msix failed
>>>>
>>>> The conclusion I came up was that the msix setup in the endpoint fails because
>>>> in the pcie-designware-host.c we are rejecting msix as you can see here:
>>>>
>>>> /* MSI-X interrupts are not supported */
>>>> if (type == PCI_CAP_ID_MSIX)
>>>>         return -EINVAL;
>>>>
>>>> Did you ever came across this problem?
>>>
>>> I don't have access to any system using this IP, unfortunately. This
>>> looks like an arbitrary limitation of the host controller driver, as
>>> there isn't much difference between MSI and MSI-X at the MSI controller
>>> level, other than the reduced number of interrupts.
>>>
>>> Unfortunately, this driver seems to rely on rather old APIs, and could
>>> use some care and attention. I'm pretty sure that converting this driver
>>> to the generic MSI API would solve this problem entirely, but I'm afraid
>>> I don't have the bandwidth to do so.
>>
>> I volunteer to do it, no problem, but I will need some guidance and examples.
>> Could you give me some hints to get started?
> 
> A very simple example using the current API is pcie-altera-msi.c (some
> others are much more intricate), and should give you a good picture of
> how PCI/MSI, generic MSI and MSI controllers interact.
> 
> Start with understanding how the various layers interact via their own
> IRQ domains, and implement their own IRQ chip. Happy to answer questions
> (I'm sure you'll have some).
> 
> Thanks,
> 
> 	M.
> 

Thanks for the guidance Marc!

Joao

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Joao Pinto]

Hi Mark,

Às 10:41 AM de 3/21/2017, Marc Zyngier escreveu:
> On 21/03/17 10:27, Joao Pinto wrote:
>>
>> Hi Mark,
>>
>> Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu:
>>> Hi Joao,
>>>
>>> On 20/03/17 17:30, Joao Pinto wrote:
>>>> Hi Mark,
>>>>
> A very simple example using the current API is pcie-altera-msi.c (some
> others are much more intricate), and should give you a good picture of
> how PCI/MSI, generic MSI and MSI controllers interact.
> 
> Start with understanding how the various layers interact via their own
> IRQ domains, and implement their own IRQ chip. Happy to answer questions
> (I'm sure you'll have some).
> 
> Thanks,
> 
> 	M.

I have one quick question... What address should I configure in vector_phy?
is this needed in pcie-designware?

Thanks

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Marc Zyngier]

On 23/03/17 15:22, Joao Pinto wrote:
> Hi Mark,
> 
> Às 10:41 AM de 3/21/2017, Marc Zyngier escreveu:
>> On 21/03/17 10:27, Joao Pinto wrote:
>>>
>>> Hi Mark,
>>>
>>> Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu:
>>>> Hi Joao,
>>>>
>>>> On 20/03/17 17:30, Joao Pinto wrote:
>>>>> Hi Mark,
>>>>>
>> A very simple example using the current API is pcie-altera-msi.c (some
>> others are much more intricate), and should give you a good picture of
>> how PCI/MSI, generic MSI and MSI controllers interact.
>>
>> Start with understanding how the various layers interact via their own
>> IRQ domains, and implement their own IRQ chip. Happy to answer questions
>> (I'm sure you'll have some).
>>
>> Thanks,
>>
>> 	M.
> 
> I have one quick question... What address should I configure in vector_phy?
> is this needed in pcie-designware?

You may be in a better place than me to answer that question! ;-)

If I remember well, the Altera RC needs to be programmed with the
address of the MSI doorbell so that it can detect the end-point writes
to it, and it can be anything you want. In order not to waste any memory
space, we use the base address of the RC.

Both very nice and dead ugly, if you want my opinion.

If we look at this:

void dw_pcie_msi_init(struct pcie_port *pp)
{
        u64 msi_target;

        pp->msi_data = __get_free_pages(GFP_KERNEL, 0);
        msi_target = virt_to_phys((void *)pp->msi_data);

        /* program the msi_data */
        dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,
                            (u32)(msi_target & 0xffffffff));
        dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_HI, 4,
                            (u32)(msi_target >> 32 & 0xffffffff));
}

We can infer the Designware IP has a similar behaviour, except that it
wastes a page of RAM for no obvious reason. It'd be good to know if the
RC actually propagates the write to the memory. If it really does, then
that's the only way to do it. If it doesn't, then the Altera method is
much better...

Hope this helps,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: Help: MSI DWC connecting with a MSIx Endpoint

[Joao Pinto]

Às 3:52 PM de 3/23/2017, Marc Zyngier escreveu:
> On 23/03/17 15:22, Joao Pinto wrote:
>> Hi Mark,
>>
>> Às 10:41 AM de 3/21/2017, Marc Zyngier escreveu:
>>> On 21/03/17 10:27, Joao Pinto wrote:
>>>>
>>>> Hi Mark,
>>>>
>>>> Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu:
>>>>> Hi Joao,
>>>>>
>>>>> On 20/03/17 17:30, Joao Pinto wrote:
>>>>>> Hi Mark,
>>>>>>
>>> A very simple example using the current API is pcie-altera-msi.c (some
>>> others are much more intricate), and should give you a good picture of
>>> how PCI/MSI, generic MSI and MSI controllers interact.
>>>
>>> Start with understanding how the various layers interact via their own
>>> IRQ domains, and implement their own IRQ chip. Happy to answer questions
>>> (I'm sure you'll have some).
>>>
>>> Thanks,
>>>
>>> 	M.
>>
>> I have one quick question... What address should I configure in vector_phy?
>> is this needed in pcie-designware?
> 
> You may be in a better place than me to answer that question! ;-)
> 
> If I remember well, the Altera RC needs to be programmed with the
> address of the MSI doorbell so that it can detect the end-point writes
> to it, and it can be anything you want. In order not to waste any memory
> space, we use the base address of the RC.
> 
> Both very nice and dead ugly, if you want my opinion.
> 
> If we look at this:
> 
> void dw_pcie_msi_init(struct pcie_port *pp)
> {
>         u64 msi_target;
> 
>         pp->msi_data = __get_free_pages(GFP_KERNEL, 0);
>         msi_target = virt_to_phys((void *)pp->msi_data);
> 
>         /* program the msi_data */
>         dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,
>                             (u32)(msi_target & 0xffffffff));
>         dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_HI, 4,
>                             (u32)(msi_target >> 32 & 0xffffffff));
> }
> 
> We can infer the Designware IP has a similar behaviour, except that it
> wastes a page of RAM for no obvious reason. It'd be good to know if the
> RC actually propagates the write to the memory. If it really does, then
> that's the only way to do it. If it doesn't, then the Altera method is
> much better...

Thanks for the help, I am not very familar with the MSI mechanism, so still
giving baby steps :) Yes, for now I am using "pp->msi_data =
__get_free_pages(GFP_KERNEL, 0);". Thanks for the tip!

> 
> Hope this helps,
> 
> 	M.
>