On Tue, Nov 19, 2019 at 11:05:55AM +0800, chenxiang (M) wrote:
> Hi Ming,
> 
> 在 2019/11/19 9:42, Ming Lei 写道:
> > On Tue, Nov 19, 2019 at 09:25:30AM +0800, chenxiang (M) wrote:
> > > Hi,
> > > 
> > > There are 128 cpus and 16 irqs for SAS controller in my system, and there
> > > are 4 Nodes, every 32 cpus are for one node (cpu0-31 for node0, cpu32-63 for
> > > node1, cpu64-95 for node2, cpu96-127 for node3).
> > > We use function pci_alloc_irq_vectors_affinity() to set the affinity of
> > > irqs.
> > > 
> > > I find that  before the patch (Fixes: b1a5a73e64e9 ("genirq/affinity: Spread
> > > vectors on node according to nr_cpu ratio")), the relationship between irqs
> > > and cpus is: irq0 bind to cpu0-7, irq1 bind to cpu8-15,
> > > irq2 bind to cpu16-23, irq3 bind to cpu24-31,irq4 bind to cpu32-39... irq15
> > > bind to cpu120-127. But after the patch, the relationship is changed: irq0
> > > bind to cpu32-39,
> > > irq1 bind to cpu40-47, ..., irq11 bind to cpu120-127, irq12 bind to cpu0-7,
> > > irq13 bind to cpu8-15, irq14 bind to cpu16-23, irq15 bind to cpu24-31.
> > > 
> > > I notice that before calling the sort() in function alloc_nodes_vectors(),
> > > the id of array node_vectors[] is from 0,1,2,3. But after function sort(),
> > > the index of array node_vectors[] is 1,2,3,0.
> > > But i think it sorts according to the numbers of cpus in those nodes, so it
> > > should be the same as before calling sort() as the numbers of cpus in every
> > > node are 32.
> > Maybe there are more non-present CPUs covered by node 0.
> > 
> > Could you provide the following log?
> > 
> > 1) lscpu
> > 
> > 2) ./dump-io-irq-affinity $PCI_ID_SAS
> > 
> > 	http://people.redhat.com/minlei/tests/tools/dump-io-irq-affinity
> > 
> > You need to figure out the PCI ID(the 1st column of lspci output) of the SAS
> > controller via lspci.
> 
> Sorry, I can't access the link you provide, but i can provide those irqs'
> affinity in the attachment.
> I also write a small testcase, and find id is 1, 2, 3, 0 after calling
> sort() .

Runtime log from /proc/interrupts isn't useful for investigating
affinity allocation issue, please use the attached script for collecting
log.


Thanks,
Ming