RE: [patch V2 00/40] genirq, irqchip: Convert ARM MSI handling to per device MSI domains

[Frank Li <Frank.Li@nxp.com>]

From: Frank Li <frank.li@nxp.com>

On Mon, Nov 21, 2022 at 03:39:28PM +0100, Thomas Gleixner wrote:
> The series is also available from git:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-v2-arm
>
> The work was done to validate that the design decisions for per device MSI
> domains and the related data structures are up to the task to handle the
> ARM oddities. The design held up, it was just necessary to extend some of
> the data structures to handle the ARM specifics in a sane way, but there
> was no fundamental shortcoming.

I based on branch devmsi-arm, and run at ls1028a platform. And found 
Integrated Endpoint Root Complex and enetc can't work.

The reason is that pci_msi_domain_get_msi_rid can't get correct rid.

[    5.153596]  pci_msi_domain_get_msi_rid+0x94/0x110
[    5.158399]  its_pci_msi_prepare+0xac/0x180
[    5.162594]  __msi_domain_alloc_irqs+0x78/0x464
[    5.167138]  __msi_domain_alloc_locked+0x148/0x190
[    5.171942]  msi_domain_alloc_irqs_all_locked+0x54/0xb0
[    5.177184]  pci_msi_setup_msi_irqs+0x2c/0x4c
[    5.181551]  __pci_enable_msix_range+0x374/0x600
[    5.186182]  pci_alloc_irq_vectors_affinity+0xbc/0x13c
[    5.191336]  pci_alloc_irq_vectors+0x14/0x20
[    5.195618]  enetc_alloc_msix+0x40/0x2cc
[    5.199550]  enetc_pf_probe+0x788/0xea0
[    5.203395]  local_pci_probe+0x40/0x9c
[    5.207151]  pci_device_probe+0xac/0x20c


u32 pci_msi_domain_get_msi_rid(struct irq_domain *domain, struct pci_dev *pdev)
{
        struct device_node *of_node;
        u32 rid = pci_dev_id(pdev);

        pci_for_each_dma_alias(pdev, get_msi_id_cb, &rid);

        of_node = irq_domain_get_of_node(domain);
        rid = of_node ? of_msi_map_id(&pdev->dev, of_node, rid) :
                        iort_msi_map_id(&pdev->dev, rid);

}

supposed of_msi_map_id get map informaiton from device tree
msi-map = <0 &its 0x17 0xe>;


but when msi_create_device_irq_domain pre device msi domain, 
fwnode is NOT PCI host controller's device node.

bool msi_create_device_irq_domain()
{
	...
	if (bundle->info.flags & MSI_FLAG_USE_DEV_FWNODE)
                fwnode = dev->fwnode;
        else
                fwnode = fwnalloced = irq_domain_alloc_named_fwnode(bundle->name);
	...
}



[    4.601732]  msi_create_device_irq_domain+0x1bc/0x20c
[    4.606800]  pci_setup_msix_device_domain+0x98/0xd4
[    4.611691]  __pci_enable_msix_range+0x21c/0x600
[    4.616322]  pci_alloc_irq_vectors_affinity+0xbc/0x13c
[    4.621477]  pci_alloc_irq_vectors+0x14/0x20
[    4.625760]  enetc_alloc_msix+0x40/0x2cc
[    4.629693]  enetc_pf_probe+0x788/0xea0
[    4.633537]  local_pci_probe+0x40/0x9c
[    4.637294]  pci_device_probe+0xac/0x20c


I don't know how to correct fix this problem. Can you give some suggestion?

Frank