Setup:

Client - Raspberry PI, running Wireguard native
Server - FreeBSD box, running Wireguard-go

Note, all of the computers involved in the test are running inside my local LAN, so there are no (active) firewalls involved at the moment, so any/all traffic is allowed between hosts.

I setup a proof of concept using a FreeBSD VM, and monitored the entire process, and it worked fine (sort of, but that's a topic for another email).

When I switched to a physical box (same OS version, etc..), things didn't work so well.  But, occassionally, it would start working for reasons that weren't obvious, when I finally figured out what was going on.

On the FreeBSD box (server), I have the em0 interface which is the local ethernet.  It also has the wg0 interface, which was created by wireguard-go.

Server Configuration file:
---- cut here -----
[Interface]
ListenPort = 1194
PrivateKey = ...

[Peer]
PublicKey = ...
PresharedKey = ...
AllowedIPs = 10.8.0.2/32
PersistentKeepalive = 120
---- cut here -----
Pretty straight-foward (no Endpoint since the client provides it)

On the RPI, it uses wireless, so wlan0, and the wg0 interface.
---- cut here -----
[Interface]
PrivateKey = ...

[Peer]
Endpoint = server.yogotech.com:1194
PublicKey = ...
PresharedKey = ...
AllowedIPs = 10.8.0.1/32
PersistentKeepalive = 120
---- cut here -----
Again, no ListenPort since it has to connect to the server and the port doesn't matter.

If I sniff on the physical on the FreeBSD box, I can see packets from the PI
# tcpdump -ni em0 port 1194
14:53:41.454233 IP 172.30.77.45.40788 > 172.30.77.1.1194: UDP, length 148
...
Unfortunately, there is no connectivity.  The FreeBSD box doesn't do anything with the packets.  It will stay that way all day without actually making a connection.

However, if I do the following
# tcpdump -ni wg0

As soon as this is done, wireguard starts working.
The kernel message that is created when this occurs is:
wg0: promiscuous mode enabled
wg0: promiscuous mode disabled

This is very repeatable.  The link will stay active until the link is refreshed (stopped/restarted) at the server end, at which point it will not reconnect UNTIL I put the wg0 interface in promiscous mode (my guess) using tcpdump.

Note, if I don't refresh the link on the server, the client can reboot/restart the connection at will without issue.

I'm trying a simple post-configuration script to fix the issue with
#!/bin/sh
/usr/sbin/tcpdump -ni wg0 > /dev/null 2>&1 &
pid=$!
sleep 5
kill $pid

As I don't know anything else that sticks the interface into promiscous mode.  However, this is *REALLY* ugly.

Ideas?


Nate