WireGuard Archive on lore.kernel.org
 help / color / Atom feed
* Speed on Raspberry Pi 4
@ 2019-06-29 10:38 Christopher Bachner
  2019-07-17 20:50 ` Roman Mamedov
  0 siblings, 1 reply; 6+ messages in thread
From: Christopher Bachner @ 2019-06-29 10:38 UTC (permalink / raw)
  To: WireGuard mailing list

[-- Attachment #1.1.1: Type: text/plain, Size: 457 bytes --]

Hello,

I got a Raspberry Pi 4 with 4GB Ram.

I ran some benchmarks. With pure iperf3 I get 950 Mbit/s.

With wireguard in the same network I can only get max 750 Mbit/s (which in
itself is already great).

In htop I can see that one of the 4 cores is running at 99%. So I assume
that is the bottleneck.

[image: image.png]

Is there a way to improve this? I assume it does not matter which side is
the server and which is the client?

Thanks,

Christopher

[-- Attachment #1.1.2: Type: text/html, Size: 739 bytes --]

<div dir="ltr">Hello,<div><br></div><div>I got a Raspberry Pi 4 with 4GB Ram. </div><div><br></div><div>I ran some benchmarks. With pure iperf3 I get 950 Mbit/s.</div><div><br></div><div>With wireguard in the same network I can only get max 750 Mbit/s (which in itself is already great).</div><div><br></div><div>In htop I can see that one of the 4 cores is running at 99%. So I assume that is the bottleneck.</div><div><br></div><div><div><img src="cid:ii_jxhe4ain0" alt="image.png" width="531" height="141"><br></div></div><div><br></div><div>Is there a way to improve this? I assume it does not matter which side is the server and which is the client?</div><div><br></div><div>Thanks,</div><div><br></div><div>Christopher</div></div>

[-- Attachment #1.2: image.png --]
[-- Type: image/png, Size: 9933 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Speed on Raspberry Pi 4
  2019-06-29 10:38 Speed on Raspberry Pi 4 Christopher Bachner
@ 2019-07-17 20:50 ` Roman Mamedov
  2019-07-18  6:38   ` Janne Johansson
  0 siblings, 1 reply; 6+ messages in thread
From: Roman Mamedov @ 2019-07-17 20:50 UTC (permalink / raw)
  To: Christopher Bachner; +Cc: WireGuard mailing list

On Sat, 29 Jun 2019 12:38:01 +0200
Christopher Bachner <hello@chrisbox.org> wrote:

> In htop I can see that one of the 4 cores is running at 99%. So I assume
> that is the bottleneck.
> 
> Is there a way to improve this? I assume it does not matter which side is
> the server and which is the client?

You can see that the load from WireGuard encryption is about 42-43% per each
core. But the thing is, one of them (the 1st) also gets to process interrupt
load from the NIC, and that consumes the rest of it, causing the bottleneck. In
theory, if you could limit WG to run encryption on all cores EXCEPT the first
one, then maaaaybe...

Another way would be to use a NIC which is capable of splitting interrupt load
across multiple CPU cores. But I believe the RPi one can't do that, and no USB
NICs can.

-- 
With respect,
Roman
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Speed on Raspberry Pi 4
  2019-07-17 20:50 ` Roman Mamedov
@ 2019-07-18  6:38   ` Janne Johansson
  2019-07-18  6:56     ` Roman Mamedov
  0 siblings, 1 reply; 6+ messages in thread
From: Janne Johansson @ 2019-07-18  6:38 UTC (permalink / raw)
  To: Roman Mamedov; +Cc: Christopher Bachner, WireGuard mailing list

[-- Attachment #1.1: Type: text/plain, Size: 884 bytes --]

Den ons 17 juli 2019 kl 22:53 skrev Roman Mamedov <rm@romanrm.net>:

> On Sat, 29 Jun 2019 12:38:01 +0200
> Christopher Bachner <hello@chrisbox.org> wrote:
> > In htop I can see that one of the 4 cores is running at 99%. So I assume
> > that is the bottleneck.
> > Is there a way to improve this? I assume it does not matter which side is
> > the server and which is the client?
>
> You can see that the load from WireGuard encryption is about 42-43% per
> each
> core. But the thing is, one of them (the 1st) also gets to process
> interrupt
> load from the NIC, and that consumes the rest of it, causing the
> bottleneck. In
> theory, if you could limit WG to run encryption on all cores EXCEPT the
> first
> one, then maaaaybe...
>
>
With taskset you should be able to:
https://www.howtoforge.com/linux-taskset-command/


-- 
May the most significant bit of your life be positive.

[-- Attachment #1.2: Type: text/html, Size: 1422 bytes --]

<div dir="ltr"><div dir="ltr">Den ons 17 juli 2019 kl 22:53 skrev Roman Mamedov &lt;<a href="mailto:rm@romanrm.net">rm@romanrm.net</a>&gt;:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sat, 29 Jun 2019 12:38:01 +0200<br>
Christopher Bachner &lt;<a href="mailto:hello@chrisbox.org" target="_blank">hello@chrisbox.org</a>&gt; wrote:<br>
&gt; In htop I can see that one of the 4 cores is running at 99%. So I assume<br>
&gt; that is the bottleneck.<br>
&gt; Is there a way to improve this? I assume it does not matter which side is<br>
&gt; the server and which is the client?<br>
<br>
You can see that the load from WireGuard encryption is about 42-43% per each<br>
core. But the thing is, one of them (the 1st) also gets to process interrupt<br>
load from the NIC, and that consumes the rest of it, causing the bottleneck. In<br>
theory, if you could limit WG to run encryption on all cores EXCEPT the first<br>
one, then maaaaybe...<br>
<br></blockquote><div><br></div><div>With taskset you should be able to:</div><div><a href="https://www.howtoforge.com/linux-taskset-command/">https://www.howtoforge.com/linux-taskset-command/</a><br></div><div><br></div></div><div><br></div>-- <br><div dir="ltr" class="gmail_signature">May the most significant bit of your life be positive.<br></div></div>

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Speed on Raspberry Pi 4
  2019-07-18  6:38   ` Janne Johansson
@ 2019-07-18  6:56     ` Roman Mamedov
  2019-07-18  8:01       ` Janne Johansson
  0 siblings, 1 reply; 6+ messages in thread
From: Roman Mamedov @ 2019-07-18  6:56 UTC (permalink / raw)
  To: Janne Johansson; +Cc: Christopher Bachner, WireGuard mailing list

On Thu, 18 Jul 2019 08:38:54 +0200
Janne Johansson <icepic.dz@gmail.com> wrote:

> With taskset you should be able to:
> https://www.howtoforge.com/linux-taskset-command/

It appears "taskset" only works on regular programs, not kernel threads:

# taskset -p -c 1 2128
pid 2128's current affinity list: 0
taskset: failed to set pid 2128's affinity: Invalid argument

# taskset -p -c 1 2129
pid 2129's current affinity list: 1
taskset: failed to set pid 2129's affinity: Invalid argument

As the above shows, WG threads are already bound to a particular CPU and this
can't be changed.

-- 
With respect,
Roman
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Speed on Raspberry Pi 4
  2019-07-18  6:56     ` Roman Mamedov
@ 2019-07-18  8:01       ` Janne Johansson
  2019-07-18  8:15         ` Matthias Urlichs
  0 siblings, 1 reply; 6+ messages in thread
From: Janne Johansson @ 2019-07-18  8:01 UTC (permalink / raw)
  To: Roman Mamedov; +Cc: Christopher Bachner, WireGuard mailing list

[-- Attachment #1.1: Type: text/plain, Size: 765 bytes --]

Den tors 18 juli 2019 kl 08:56 skrev Roman Mamedov <rm@romanrm.net>:

> On Thu, 18 Jul 2019 08:38:54 +0200
> Janne Johansson <icepic.dz@gmail.com> wrote:
>
> > With taskset you should be able to:
> > https://www.howtoforge.com/linux-taskset-command/
>
> It appears "taskset" only works on regular programs, not kernel threads:
>
> # taskset -p -c 1 2128
> pid 2128's current affinity list: 0
> taskset: failed to set pid 2128's affinity: Invalid argument
>
> # taskset -p -c 1 2129
> pid 2129's current affinity list: 1
> taskset: failed to set pid 2129's affinity: Invalid argument
>
> As the above shows, WG threads are already bound to a particular CPU and
> this
> can't be changed.
>

Right, my bad.

-- 
May the most significant bit of your life be positive.

[-- Attachment #1.2: Type: text/html, Size: 1406 bytes --]

<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Den tors 18 juli 2019 kl 08:56 skrev Roman Mamedov &lt;<a href="mailto:rm@romanrm.net">rm@romanrm.net</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Thu, 18 Jul 2019 08:38:54 +0200<br>
Janne Johansson &lt;<a href="mailto:icepic.dz@gmail.com" target="_blank">icepic.dz@gmail.com</a>&gt; wrote:<br>
<br>
&gt; With taskset you should be able to:<br>
&gt; <a href="https://www.howtoforge.com/linux-taskset-command/" rel="noreferrer" target="_blank">https://www.howtoforge.com/linux-taskset-command/</a><br>
<br>
It appears &quot;taskset&quot; only works on regular programs, not kernel threads:<br>
<br>
# taskset -p -c 1 2128<br>
pid 2128&#39;s current affinity list: 0<br>
taskset: failed to set pid 2128&#39;s affinity: Invalid argument<br>
<br>
# taskset -p -c 1 2129<br>
pid 2129&#39;s current affinity list: 1<br>
taskset: failed to set pid 2129&#39;s affinity: Invalid argument<br>
<br>
As the above shows, WG threads are already bound to a particular CPU and this<br>
can&#39;t be changed.<br></blockquote><div><br></div><div>Right, my bad. </div></div><div><br></div>-- <br><div dir="ltr" class="gmail_signature">May the most significant bit of your life be positive.<br></div></div>

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Speed on Raspberry Pi 4
  2019-07-18  8:01       ` Janne Johansson
@ 2019-07-18  8:15         ` Matthias Urlichs
  0 siblings, 0 replies; 6+ messages in thread
From: Matthias Urlichs @ 2019-07-18  8:15 UTC (permalink / raw)
  To: wireguard

[-- Attachment #1.1: Type: text/plain, Size: 728 bytes --]

Hi,
>
>     As the above shows, WG threads are already bound to a particular
>     CPU and this
>     can't be changed.
>
>
> Right, my bad. 

OK. So we have N kernel threads (one per CPU) and one CPU that really
shouldn't do anything but interrupt processing.

That looks like we need an option to limit wireguard to a specific set
of CPUs. That'd be a good option to have in any case, because we don't
want the poor Raspberry Pi (or any other semi-underpowered machine) to
starve everything else when it gets flooded with more wireguard work
than it can handle.

We could then set the network interface's IRQ affinity to one of the
"free" CPUs, and we'd be all set.

-- 
-- mit freundlichen Grüßen
-- 
-- Matthias Urlichs


[-- Attachment #1.2: Type: text/html, Size: 1378 bytes --]

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">Hi,<br>
    </div>
    <blockquote type="cite"
cite="mid:CAA6-MF_yzAj5-+D6K_H9GaQRaZLLW0D2a88FQnMmwO66+=O67g@mail.gmail.com">
      <blockquote class="gmail_quote" style="margin:0px 0px 0px
        0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
        As the above shows, WG threads are already bound to a particular
        CPU and this<br>
        can't be changed.<br>
      </blockquote>
      <div><br>
      </div>
      <div>Right, my bad. </div>
    </blockquote>
    <p>OK. So we have N kernel threads (one per CPU) and one CPU that
      really shouldn't do anything but interrupt processing.</p>
    <p>That looks like we need an option to limit wireguard to a
      specific set of CPUs. That'd be a good option to have in any case,
      because we don't want the poor Raspberry Pi (or any other
      semi-underpowered machine) to starve everything else when it gets
      flooded with more wireguard work than it can handle.</p>
    <p>We could then set the network interface's IRQ affinity to one of
      the "free" CPUs, and we'd be all set.</p>
    <pre class="moz-signature" cols="72">-- 
-- mit freundlichen Grüßen
-- 
-- Matthias Urlichs</pre>
  </body>
</html>

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, back to index

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-29 10:38 Speed on Raspberry Pi 4 Christopher Bachner
2019-07-17 20:50 ` Roman Mamedov
2019-07-18  6:38   ` Janne Johansson
2019-07-18  6:56     ` Roman Mamedov
2019-07-18  8:01       ` Janne Johansson
2019-07-18  8:15         ` Matthias Urlichs

WireGuard Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/wireguard/0 wireguard/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 wireguard wireguard/ https://lore.kernel.org/wireguard \
		wireguard@lists.zx2c4.com zx2c4-wireguard@archiver.kernel.org
	public-inbox-index wireguard


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/com.zx2c4.lists.wireguard


AGPL code for this site: git clone https://public-inbox.org/ public-inbox