* RPS arp processing
@ 2019-12-11 15:26 Aleksei Zakharov
2019-12-11 18:59 ` Willem de Bruijn
0 siblings, 1 reply; 3+ messages in thread
From: Aleksei Zakharov @ 2019-12-11 15:26 UTC (permalink / raw)
To: netdev
Hi, everyone
Is it possible to balance ARP across CPUs with RPS?
I don't clearly understand how hash is calulated for ARP packets, but it seems that it should consider source and target IPs.
In our current setup we have one l2 segment between external hardware routers and namespaces on linux server.
When router sends ARP request, it is passed through server's physical port, then via openvswitch bridge it is copied to every namespace.
We've found that all ARPs (for different destination ips and few source ips) are processed on one CPU inside namespaces. We use RPS, and most packets are balanced between all CPUs.
Kernel 4.15.0-65 from ubuntu 18.04.
Might this issue be related to namespaces somehow?
--
Regards,
Aleksei Zakharov
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: RPS arp processing
2019-12-11 15:26 RPS arp processing Aleksei Zakharov
@ 2019-12-11 18:59 ` Willem de Bruijn
2019-12-11 22:08 ` Aleksei Zakharov
0 siblings, 1 reply; 3+ messages in thread
From: Willem de Bruijn @ 2019-12-11 18:59 UTC (permalink / raw)
To: Aleksei Zakharov; +Cc: Network Development
On Wed, Dec 11, 2019 at 10:34 AM Aleksei Zakharov
<zakharov.a.g@yandex.ru> wrote:
>
> Hi, everyone
> Is it possible to balance ARP across CPUs with RPS?
> I don't clearly understand how hash is calulated for ARP packets, but it seems that it should consider source and target IPs.
The hash is derived by flow dissection:
get_rps_cpus
___skb_get_hash
skb_flow_dissect_flow_keys
This calls __skb_flow_dissector with the flow_keys_dissector
dissection program, which is initialized in
init_default_flow_dissectors from flow_keys_dissector_keys.
That program incorporates IPV4_ADDRS and IPV6_ADDRS. But that does not
apply to ARP packets. Contrast case ETH_P_IPV6 with case ETH_P_ARP in
__skb_flow_dissect.
The flow dissector calls __skb_flow_dissect_arp() for deeper
dissection, from which you could extract entropy for RPS. But the
flow_keys_dissector program does not have FLOW_DISSECTOR_KEY_ARP
enabled.
> In our current setup we have one l2 segment between external hardware routers and namespaces on linux server.
> When router sends ARP request, it is passed through server's physical port, then via openvswitch bridge it is copied to every namespace.
> We've found that all ARPs (for different destination ips and few source ips) are processed on one CPU inside namespaces. We use RPS, and most packets are balanced between all CPUs.
I suggest looking at the newer BPF flow dissector, which allows tuning
dissection to specific use cases, like yours.
> Kernel 4.15.0-65 from ubuntu 18.04.
>
> Might this issue be related to namespaces somehow?
>
> --
> Regards,
> Aleksei Zakharov
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: RPS arp processing
2019-12-11 18:59 ` Willem de Bruijn
@ 2019-12-11 22:08 ` Aleksei Zakharov
0 siblings, 0 replies; 3+ messages in thread
From: Aleksei Zakharov @ 2019-12-11 22:08 UTC (permalink / raw)
To: Willem de Bruijn; +Cc: Network Development
11.12.2019, 22:00, "Willem de Bruijn" <willemdebruijn.kernel@gmail.com>:
> On Wed, Dec 11, 2019 at 10:34 AM Aleksei Zakharov
> <zakharov.a.g@yandex.ru> wrote:
>> Hi, everyone
>> Is it possible to balance ARP across CPUs with RPS?
>> I don't clearly understand how hash is calulated for ARP packets, but it seems that it should consider source and target IPs.
>
> The hash is derived by flow dissection:
>
> get_rps_cpus
> ___skb_get_hash
> skb_flow_dissect_flow_keys
>
> This calls __skb_flow_dissector with the flow_keys_dissector
> dissection program, which is initialized in
> init_default_flow_dissectors from flow_keys_dissector_keys.
>
> That program incorporates IPV4_ADDRS and IPV6_ADDRS. But that does not
> apply to ARP packets. Contrast case ETH_P_IPV6 with case ETH_P_ARP in
> __skb_flow_dissect.
>
> The flow dissector calls __skb_flow_dissect_arp() for deeper
> dissection, from which you could extract entropy for RPS. But the
> flow_keys_dissector program does not have FLOW_DISSECTOR_KEY_ARP
> enabled.
Thank you very much for the explanation!
>
>> In our current setup we have one l2 segment between external hardware routers and namespaces on linux server.
>> When router sends ARP request, it is passed through server's physical port, then via openvswitch bridge it is copied to every namespace.
>> We've found that all ARPs (for different destination ips and few source ips) are processed on one CPU inside namespaces. We use RPS, and most packets are balanced between all CPUs.
>
> I suggest looking at the newer BPF flow dissector, which allows tuning
> dissection to specific use cases, like yours.
Thanks, I'll take a look at bpf dissector.
>
>> Kernel 4.15.0-65 from ubuntu 18.04.
>>
>> Might this issue be related to namespaces somehow?
>>
>> --
>> Regards,
>> Aleksei Zakharov
--
Regards,
Aleksei Zakharov
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-12-11 22:08 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-11 15:26 RPS arp processing Aleksei Zakharov
2019-12-11 18:59 ` Willem de Bruijn
2019-12-11 22:08 ` Aleksei Zakharov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).