[WireGuard] Client changes endpoint port, why?

[WireGuard] Client changes endpoint port, why?

[Jan De Landtsheer]

[-- Attachment #1: Type: text/plain, Size: 2333 bytes --]

Testing out this little thingie ; looks Grrrr-eat!
I dont understand why the client setup changes it’s port after a while:

[delandtj@rt01 ~]$ sudo wg
interface: wg0
  public key: fppppppppppppppppppppppppppppppppp=
  private key: IiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiE=
  listening port: 23123

peer: RnPeeerrreeeerrrk=
  endpoint: xxx.xxx.xxx.126:51820
  allowed ips: 192.168.251.1/32
  latest handshake: 2 minutes, 14 seconds ago
  bandwidth: 658 B received, 944 B sent
[delandtj@rt01 ~]$ ping 192.168.251.1
PING 192.168.251.1 (192.168.251.1) 56(84) bytes of data.
64 bytes from 192.168.251.1: icmp_seq=1 ttl=64 time=10.5 ms
^C
--- 192.168.251.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 10.544/10.544/10.544/0.000 ms

Yay! now wait a few minutes

[delandtj@rt01 ~]$ ping 192.168.251.1
PING 192.168.251.1 (192.168.251.1) 56(84) bytes of data.
^C
--- 192.168.251.1 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4009ms

[delandtj@rt01 ~]$ sudo wg
interface: wg0
  public key: ppppppppppppppppppppppppppppppppppppc=
  private key: Iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii3my0E=
  listening port: 23123

peer: RPeeerrrreeerrrrrpppppppppppppppk=
  endpoint: xxx.xxx.xxx.126:17409
  allowed ips: 192.168.251.1/32
  latest handshake: 13 minutes, 55 seconds ago
  bandwidth: 4.23 KiB received, 6.42 KiB sent

Ugh… endpoint port changed ? why ?
Let’s set it back

[delandtj@rt01 ~]$ sudo wg setconf wg0 wg/conf
[delandtj@rt01 ~]$ sudo wg
interface: wg0
  public key: frILpppppppppppppppppppppppppppppppppppppppppppp/Qc=
  private key: IJiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii=
  listening port: 23123

peer: RnOKNCpBs2Wf2dQP9Gedd+jCVkcOJOQ0m5FK+3WFGmk=
  endpoint: xxx.xxx.xxx.126:51820
  allowed ips: 192.168.251.1/32
[delandtj@rt01 ~]$ ping 192.168.251.1
PING 192.168.251.1 (192.168.251.1) 56(84) bytes of data.
64 bytes from 192.168.251.1: icmp_seq=1 ttl=64 time=25.0 ms
^C
--- 192.168.251.1 ping statistics ---
2 packets transmitted, 1 received, 50% packet loss, time 1000ms
rtt min/avg/max/mdev = 25.066/25.066/25.066/0.000 ms

So… am I missing something ?

Jan
​

[-- Attachment #2: Type: text/html, Size: 8793 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Jason A. Donenfeld]

Hi Jan,

That's very strange. Are you sure there aren't other wireguard peers
running thare using the same private key?

Does it always change to the *same* wrong port?

Jason

Re: [WireGuard] Client changes endpoint port, why?

[Jan De Landtsheer]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

  - about changing ports:
hmmm. can't really say...
What I noticed: I could ping yesterday, without doing anything, I couldn't
this morning. that's when I saw the difference.
I had something like it yesterday, and thinking I did something wrong, I
set it in stone in a config file. applied it, had my ping, kept the
terminal session on the server open (had also an openvpn to the remote).
This morning, from the remote , there was no ping. Verified why. And then I
sent this mail ;-)

Note: it's properly up since, so I don't know...
I'll keep it as it is, will let you know if something switches again.
Note2: No, no different peers, there is only one client, one server, so
there wouldn't be any overlap.

running arch linux, latest & geatest

  - about something else:
are these pure ip  tunnels, or could I envision to add the interfaces to an
OpenVSwitch bridge and use them as tunnel ports?

Thx
Jan


On Thu, Jul 7, 2016 at 1:29 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:

> Hi Jan,
>
> That's very strange. Are you sure there aren't other wireguard peers
> running thare using the same private key?
>
> Does it always change to the *same* wrong port?
>
> Jason
>

[-- Attachment #2: Type: text/html, Size: 1688 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 2105 bytes --]

On Thu, Jul 07, 2016 at 12:53:24PM +0000, Jan De Landtsheer wrote:
>   - about changing ports:
> hmmm. can't really say...
> What I noticed: I could ping yesterday, without doing anything, I couldn't
> this morning. that's when I saw the difference.
> I had something like it yesterday, and thinking I did something wrong, I
> set it in stone in a config file. applied it, had my ping, kept the
> terminal session on the server open (had also an openvpn to the remote).
> This morning, from the remote , there was no ping. Verified why. And then I
> sent this mail ;-)

Could there be a NAT or stateful firewall on your network, messing up the
UDP source port of packets received from the server?

If you manage to reproduce, it would be helpful to have a packet capture
before your wireguard client changes endpoint, with something like:

  client# tcpdump -w wireguard.pcap -i eth0 -s 64 'udp and host xxx.xxx.xxx.126'

Change the interface if needed, and xxx.xxx.xxx.126 is the public IP of
your server.  The packet trace will only contain the packet headers and
a small bit of encrypted data, but you can send it privately (to me and/or
Jason).

> Note: it's properly up since, so I don't know...
> I'll keep it as it is, will let you know if something switches again.
> Note2: No, no different peers, there is only one client, one server, so
> there wouldn't be any overlap.
> 
> running arch linux, latest & geatest
> 
>   - about something else:
> are these pure ip  tunnels, or could I envision to add the interfaces to an
> OpenVSwitch bridge and use them as tunnel ports?
> 
> Thx
> Jan
> 
> 
> On Thu, Jul 7, 2016 at 1:29 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> 
> > Hi Jan,
> >
> > That's very strange. Are you sure there aren't other wireguard peers
> > running thare using the same private key?
> >
> > Does it always change to the *same* wrong port?
> >
> > Jason
> >

> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Jan De Landtsheer]

[-- Attachment #1: Type: text/plain, Size: 2520 bytes --]

On Thu, Jul 7, 2016 at 3:13 PM Baptiste Jonglez <baptiste@bitsofnetworks.org>
wrote:

> On Thu, Jul 07, 2016 at 12:53:24PM +0000, Jan De Landtsheer wrote:
> >   - about changing ports:
> > hmmm. can't really say...
> > What I noticed: I could ping yesterday, without doing anything, I
> couldn't
> > this morning. that's when I saw the difference.
> > I had something like it yesterday, and thinking I did something wrong, I
> > set it in stone in a config file. applied it, had my ping, kept the
> > terminal session on the server open (had also an openvpn to the remote).
> > This morning, from the remote , there was no ping. Verified why. And
> then I
> > sent this mail ;-)
>
> Could there be a NAT or stateful firewall on your network, messing up the
> UDP source port of packets received from the server?
>

nope, Start with basics, use pub ip to pub ip
BTW, can a client run behind NAT ? (I assume not, as AFAICT both need to
listen on a port)

But like I said, I'll see if it happens again... went through my history
log still thinking it might be me, but it doesn't seem so.



> If you manage to reproduce, it would be helpful to have a packet capture
> before your wireguard client changes endpoint, with something like:
>
>   client# tcpdump -w wireguard.pcap -i eth0 -s 64 'udp and host
> xxx.xxx.xxx.126'
>
> Change the interface if needed, and xxx.xxx.xxx.126 is the public IP of
> your server.  The packet trace will only contain the packet headers and
> a small bit of encrypted data, but you can send it privately (to me and/or
> Jason).
>
> > Note: it's properly up since, so I don't know...
> > I'll keep it as it is, will let you know if something switches again.
> > Note2: No, no different peers, there is only one client, one server, so
> > there wouldn't be any overlap.
> >
> > running arch linux, latest & geatest
> >
> >   - about something else:
> > are these pure ip  tunnels, or could I envision to add the interfaces to
> an
> > OpenVSwitch bridge and use them as tunnel ports?
> >
> > Thx
> > Jan
> >
> >
> > On Thu, Jul 7, 2016 at 1:29 PM Jason A. Donenfeld <Jason@zx2c4.com>
> wrote:
> >
> > > Hi Jan,
> > >
> > > That's very strange. Are you sure there aren't other wireguard peers
> > > running thare using the same private key?
> > >
> > > Does it always change to the *same* wrong port?
> > >
> > > Jason
> > >
>
> > _______________________________________________
> > WireGuard mailing list
> > WireGuard@lists.zx2c4.com
> > http://lists.zx2c4.com/mailman/listinfo/wireguard
>
>

[-- Attachment #2: Type: text/html, Size: 3562 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Bruno Wolff III]

On Thu, Jul 07, 2016 at 14:45:22 +0000,
  Jan De Landtsheer <jan@incubaid.com> wrote:
>
>nope, Start with basics, use pub ip to pub ip
>BTW, can a client run behind NAT ? (I assume not, as AFAICT both need to
>listen on a port)

The one behind nat can hold the tunnel open so the other end can always 
reach it.

Re: [WireGuard] Client changes endpoint port, why?

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1420 bytes --]

On Thu, Jul 07, 2016 at 02:45:22PM +0000, Jan De Landtsheer wrote:
> On Thu, Jul 7, 2016 at 3:13 PM Baptiste Jonglez <baptiste@bitsofnetworks.org>
> wrote:
> 
> > On Thu, Jul 07, 2016 at 12:53:24PM +0000, Jan De Landtsheer wrote:
> > >   - about changing ports:
> > > hmmm. can't really say...
> > > What I noticed: I could ping yesterday, without doing anything, I
> > couldn't
> > > this morning. that's when I saw the difference.
> > > I had something like it yesterday, and thinking I did something wrong, I
> > > set it in stone in a config file. applied it, had my ping, kept the
> > > terminal session on the server open (had also an openvpn to the remote).
> > > This morning, from the remote , there was no ping. Verified why. And
> > then I
> > > sent this mail ;-)
> >
> > Could there be a NAT or stateful firewall on your network, messing up the
> > UDP source port of packets received from the server?
> >
> 
> nope, Start with basics, use pub ip to pub ip

Hmm, that's really strange then.  Any weird firewall rules on any of the
hosts?

> BTW, can a client run behind NAT ? (I assume not, as AFAICT both need to
> listen on a port)

Yes, you can run behind a NAT (well, maybe not if *both* peers are behind
a NAT).  Wireguard uses its local "listening port" as source UDP port when
sending packets, so this will create a mapping in a NAT or stateful
firewall.

Baptiste

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[WireGuard] Wireguard behind NAT (Was: Client changes endpoint port, why?)

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Thu, Jul 07, 2016 at 05:06:21PM +0200, Baptiste Jonglez wrote:
> On Thu, Jul 07, 2016 at 02:45:22PM +0000, Jan De Landtsheer wrote:
> > BTW, can a client run behind NAT ? (I assume not, as AFAICT both need to
> > listen on a port)
> 
> Yes, you can run behind a NAT (well, maybe not if *both* peers are behind
> a NAT).  Wireguard uses its local "listening port" as source UDP port when
> sending packets, so this will create a mapping in a NAT or stateful
> firewall.

Well, thinking about it, you should be able to make wireguard work even if
both peers are behind a stateful firewall or NAT :)

You just have to specify endpoints on *both* peers to point at the public
IP address of the other peer.

For instance:

- Peer A listens on port 4444 and is behind a NAT with public IP address X
- Peer B listens on port 5555 and is behind a NAT with public IP address Y
- Peer A does this: wg set wg0 peer B endpoint Y:5555
- Peer B does this: wg set wg0 peer A endpoint X:4444

Once both peers have sent messages, this trick should create a mapping in
both NATs.

It might not work for NAT that also rewrite the source port, though
(I never understood the terminology, but this might be called a symmetric
NAT?).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Wireguard behind NAT (Was: Client changes endpoint port, why?)

[Jason A. Donenfeld]

It might be possible to use existing hole punching tricks like STUN
with WireGuard, but there's no way I'm baking something like that into
the kernel protocol itself. This is something neat that might
piggyback on an upper layer.

However, check out the other thread for another more pressing NAT
traversal issue.

Regards,
Jason

Re: [WireGuard] Client changes endpoint port, why?

[Jason A. Donenfeld]

On Thu, Jul 7, 2016 at 5:00 PM, Bruno Wolff III <bruno@wolff.to> wrote:
> On Thu, Jul 07, 2016 at 14:45:22 +0000,
>  Jan De Landtsheer <jan@incubaid.com> wrote:
>>
>>
>> nope, Start with basics, use pub ip to pub ip
>> BTW, can a client run behind NAT ? (I assume not, as AFAICT both need to
>> listen on a port)
>
>
> The one behind nat can hold the tunnel open so the other end can always
> reach it.

This is the thrust of the issue -- holding the tunnel open when
there's no traffic. This needs to be addressed. Started new thread to
discuss this.

Re: [WireGuard] Client changes endpoint port, why?

[Jan De Landtsheer]

[-- Attachment #1: Type: text/plain, Size: 2339 bytes --]

happened again, link was up a few moments ago, and then no ping …

[delandtj@rt01 ~]$ sudo wg
interface: wg0
  public key: Stillthesame=
  private key: Stillthesame=
  listening port: 23123

peer: Stillthesame=
  endpoint: xxx.xxx.xxx.126:17059    #### changed port
  allowed ips: 192.168.251.1/32
  latest handshake: 1 hour, 58 minutes, 4 seconds ago
  bandwidth: 161.04 MiB received, 5.38 MiB sent

Then, with

[delandtj@rt01 ~]$ sudo wg setconf wg0 wg/
conf        Dockerfile  priv        pub
[delandtj@rt01 ~]$ sudo wg setconf wg0 wg/conf
[delandtj@rt01 ~]$ sudo wg
interface: wg0
  public key: REDACTED=
  private key: REDACTED=
  listening port: 23123

peer: REDACTED=
  endpoint: xxx.xxx.xxx.126:51820
  allowed ips: 192.168.251.1/32
[delandtj@rt01 ~]$ ping -c1 192.168.251.1
PING 192.168.251.1 (192.168.251.1) 56(84) bytes of data.
64 bytes from 192.168.251.1: icmp_seq=1 ttl=64 time=27.3 ms

--- 192.168.251.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 27.333/27.333/27.333/0.000 ms

and ping was back

it took 24-ish hours to happen, but not having touched the tunnel, nor the
set-up, I can definitely confirm this happening…

[delandtj@rt01 ~]$ cat wg/conf
[Interface]
PrivateKey = REDACTED=
ListenPort = 23123

[Peer]
PublicKey = REDACTED=
EndPoint = xxx.xxx.xxx.126:51820
AllowedIPs =  192.168.251.1/32

### and server :
[root@Firewall001 ~]# cat /etc/zcomp/wireguard/wg.conf
[Interface]
ListenPort = 51820
PrivateKey = REDACTED=

[Peer]
PublicKey = REDACTED=
AllowedIPs =  192.168.251.2/32, 192.168.64.0/24

Jan
​

On Thu, Jul 7, 2016 at 6:38 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:

> On Thu, Jul 7, 2016 at 5:00 PM, Bruno Wolff III <bruno@wolff.to> wrote:
> > On Thu, Jul 07, 2016 at 14:45:22 +0000,
> >  Jan De Landtsheer <jan@incubaid.com> wrote:
> >>
> >>
> >> nope, Start with basics, use pub ip to pub ip
> >> BTW, can a client run behind NAT ? (I assume not, as AFAICT both need to
> >> listen on a port)
> >
> >
> > The one behind nat can hold the tunnel open so the other end can always
> > reach it.
>
> This is the thrust of the issue -- holding the tunnel open when
> there's no traffic. This needs to be addressed. Started new thread to
> discuss this.
>

[-- Attachment #2: Type: text/html, Size: 8548 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Jason A. Donenfeld]

Hi Jan,

Have you tried this with the latest snapshot I published today? It
should fix your problem. For the boxes that are behind NAT, set
'PersistentKeepAlive = 25'. See the new manpage and the /quickstart/
blurb on the website. Alternatively, check out the [ANNOUNCE] release
notes on the mailing list from today.

Regards,
Jason

Re: [WireGuard] Client changes endpoint port, why?

[Jan De Landtsheer]

[-- Attachment #1: Type: text/plain, Size: 491 bytes --]

I'll check it out. I'm traveling now, will check tomorrow.
Jan

On Fri, Jul 8, 2016, 7:47 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:

> Hi Jan,
>
> Have you tried this with the latest snapshot I published today? It
> should fix your problem. For the boxes that are behind NAT, set
> 'PersistentKeepAlive = 25'. See the new manpage and the /quickstart/
> blurb on the website. Alternatively, check out the [ANNOUNCE] release
> notes on the mailing list from today.
>
> Regards,
> Jason
>

[-- Attachment #2: Type: text/html, Size: 777 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Jason A. Donenfeld]

On Sat, Jul 9, 2016 at 4:36 PM, Jan De Landtsheer <jan@incubaid.com> wrote:
> I'll check it out. I'm traveling now, will check tomorrow.
> Jan

No problem. There are further improvements to that right now in the
authenticated-persistent-keepalive branch. If you wait til next week,
I'll have an even better snapshot then.

Re: [WireGuard] Client changes endpoint port, why?

[Jan De Landtsheer]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

\o/ A LOT better now :-)
Great!

I'll dig in deeper... Is there a way that the kernel module logs
connection/disconnection/ip migration ?

Jan

On Sat, Jul 9, 2016 at 4:47 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:

> On Sat, Jul 9, 2016 at 4:36 PM, Jan De Landtsheer <jan@incubaid.com>
> wrote:
> > I'll check it out. I'm traveling now, will check tomorrow.
> > Jan
>
> No problem. There are further improvements to that right now in the
> authenticated-persistent-keepalive branch. If you wait til next week,
> I'll have an even better snapshot then.
>

[-- Attachment #2: Type: text/html, Size: 972 bytes --]

Re: [WireGuard] Client changes endpoint port, why?

[Jason A. Donenfeld]

On Mon, Jul 11, 2016 at 2:16 PM, Jan De Landtsheer <jan@incubaid.com> wrote:
> \o/ A LOT better now :-)
> Great!

Good to hear. I assume you tested with the snapshot I published this morning?


>
> I'll dig in deeper... Is there a way that the kernel module logs
> connection/disconnection/ip migration ?

You can get lots of junk in your dmesg if you compile with `make debug`.

Re: [WireGuard] Client changes endpoint port, why?

[Quan Zhou]

[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]

The logs didn't show up immediately after `make debug; sudo make install'.
Is there easier way to reload wg than rebooting, or deleting the interface?

On Mon, Jul 11, 2016 at 11:40 PM, Jason A. Donenfeld <Jason@zx2c4.com>
wrote:

> On Mon, Jul 11, 2016 at 2:16 PM, Jan De Landtsheer <jan@incubaid.com>
> wrote:
> > \o/ A LOT better now :-)
> > Great!
>
> Good to hear. I assume you tested with the snapshot I published this
> morning?
>
>
> >
> > I'll dig in deeper... Is there a way that the kernel module logs
> > connection/disconnection/ip migration ?
>
> You can get lots of junk in your dmesg if you compile with `make debug`.
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard
>



-- 
Regards,

Quan Zhou
+------------------------+
|pub [expires 2019-05-04]|
|2C0C 4D88 E631 4C73 4C44|
|CDE0 C0E 5470 1D2D 3F3EE|
+------------------------+
|pub [revoked 2016-04-16]|
|44D2 0307 1643 E80F 2E31|
|F081 FAFA 6643 7F9F D46F|
+------------------------+
|quanzhou822@gmail.com   |
|https://keybase.io/qzhou|
+------------------------+

[-- Attachment #2: Type: text/html, Size: 2527 bytes --]