[WireGuard] Troubleshooting with WireGuard

[WireGuard] Troubleshooting with WireGuard

[Quan Zhou]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

Hello,

I'm new to WireGuard and it look very much different from the tools I used
before, one thing that is, there aren't much log in /var/log/syslog
(Debian). There was something went wrong on my network and now I have no
idea where to look at. Did I missed something?

-- 
Regards,

Quan Zhou
+------------------------+
|pub [expires 2019-05-04]|
|2C0C 4D88 E631 4C73 4C44|
|CDE0 C0E 5470 1D2D 3F3EE|
+------------------------+
|pub [revoked 2016-04-16]|
|44D2 0307 1643 E80F 2E31|
|F081 FAFA 6643 7F9F D46F|
+------------------------+
|quanzhou822@gmail.com   |
|https://keybase.io/qzhou|
+------------------------+

[-- Attachment #2: Type: text/html, Size: 1412 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

Hello Quan,

You can build the module with "make debug", and then you'll get lots
of helpful debugging information in your dmesg.

What problem are you facing specifically?

Regards,
Jason

Re: [WireGuard] Troubleshooting with WireGuard

[Daniel Kahn Gillmor]

Hi Jason--

On Mon 2016-07-11 12:18:43 +0200, Jason A. Donenfeld wrote:

> You can build the module with "make debug", and then you'll get lots
> of helpful debugging information in your dmesg.

Would it be possible to log more when the kernel is in a more vebose
mode, or to have a module parameter that increases the logging?  Not
having to rebuild make it simpler for people experimenting with
pre-packaged (or dkms-built) modules to just "enable debugging", which
might produce more verbose problem reports.

Or does anyone know of the simplest way to encourage dkms to build a
specific module in debug mode?  If so, i could try to ship a
wireguard-dkms-debug package (which Conflicts: wireguard-dkms and
Provides: wireguard-dkms) as an alternate option for people who don't
want to do the build themselves.

        --dkg

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

On Mon, Jul 11, 2016 at 5:21 PM, Daniel Kahn Gillmor
<dkg@fifthhorseman.net> wrote:
> Would it be possible to log more when the kernel is in a more vebose
> mode, or to have a module parameter that increases the logging?  Not
> having to rebuild make it simpler for people experimenting with
> pre-packaged (or dkms-built) modules to just "enable debugging", which
> might produce more verbose problem reports.
>
> Or does anyone know of the simplest way to encourage dkms to build a
> specific module in debug mode?  If so, i could try to ship a
> wireguard-dkms-debug package (which Conflicts: wireguard-dkms and
> Provides: wireguard-dkms) as an alternate option for people who don't
> want to do the build themselves.

I'd prefer that early experimenters not get used to having too much
log output from WireGuard. It's supposed to appear "stateless" to the
user, and I'm afraid that this sort of thing might encourage bad
patterns with regards to it.

I'd rather learn precisely what issues the user is facing, and see if
we can debug it from there. Maybe that approach will prove impossible,
and in the end we'll need some logging, but I think it's worth at
least trying without logs.

That said, if you intend to help with the /development/ of wireguard,
the debug logs are quite helpful, but in that situation you'll already
be building the module yourself.

Re: [WireGuard] Troubleshooting with WireGuard

[Quan Zhou]

[-- Attachment #1: Type: text/plain, Size: 1464 bytes --]

Thank you, Jason. This is very helpful.

I was bit confused with the term AllowedIPs, and connecting to a IPv6
Endpoint somehow didn't work, so I wanted to see the logs, and find out
what I did wrong in conf.

Exerpts from configuration:

/*  ==== Begin Exerpts from Server HND06 ==== */
[Interface]
// PublicKey is 3o0...3E8=
// Has IP Address 2400:...:8993
ListenPort = 41414

[Peer]
PublicKey = K1O...4xU=
AllowedIPs = 2600:....:1487
Endpoint = [2600:...:1487]:41414

/* ==== End of HND06 ==== */

/*  ==== Begin Exerpts from Server FMT03 */
[Interface]
// PublicKey is K1O...4xU=
// Has IP address 2600:..1487
ListenPort = 41414

[Peer]
PublicKey = 3o0...3E8=
AllowedIPs = 2400:..:8993
Endpoint = [2400:...:8993]:41414

/* ==== End of FMT03 ==== */


====

Somehow the connection was never established.

Warm Regards,
Quan


On Mon, Jul 11, 2016 at 6:18 PM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:

> Hello Quan,
>
> You can build the module with "make debug", and then you'll get lots
> of helpful debugging information in your dmesg.
>
> What problem are you facing specifically?
>
> Regards,
> Jason
>



-- 
Regards,

Quan Zhou
+------------------------+
|pub [expires 2019-05-04]|
|2C0C 4D88 E631 4C73 4C44|
|CDE0 C0E 5470 1D2D 3F3EE|
+------------------------+
|pub [revoked 2016-04-16]|
|44D2 0307 1643 E80F 2E31|
|F081 FAFA 6643 7F9F D46F|
+------------------------+
|quanzhou822@gmail.com   |
|https://keybase.io/qzhou|
+------------------------+

[-- Attachment #2: Type: text/html, Size: 3021 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

Hi Quan,

WireGuard is a layer 3 tunnel device, so AllowedIPs should probably be
a different IP than endpoint unless you've set up some policy based
routing. Also, you'll need to configure the devices with ip-addr too
-- what were your commands there?

Regards,
Jason

Re: [WireGuard] Troubleshooting with WireGuard

[Quan Zhou]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]

I'm trying to connect to [2600:3c01:..:1487]:41414 from [2400:6180:...:1]

I have added the specific IPv6 /128 address to the AllowedIPs on one side,
and
on the other side I simply put ::0/0.

The address I've set was:

srv1 # ip addr add 10.240.51.2/24 dev wg0
srv2 # ip addr add 10.240.65.2/24 dev wg0

I will deal with routing later, with OSPF or simply a static route. Right
now the
problem is:

Jul 12 15:32:41 debian kernel: [1081190.595578] Invalid MAC of handshake,
dropping packet from [2400:...:aec:1]:41414/0%0

Having no idea about handshake, what should I do next.

On Tue, Jul 12, 2016 at 5:05 PM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:

> Hi Quan,
>
> WireGuard is a layer 3 tunnel device, so AllowedIPs should probably be
> a different IP than endpoint unless you've set up some policy based
> routing. Also, you'll need to configure the devices with ip-addr too
> -- what were your commands there?
>
> Regards,
> Jason
>



-- 
Regards,

Quan Zhou
+------------------------+
|pub [expires 2019-05-04]|
|2C0C 4D88 E631 4C73 4C44|
|CDE0 C0E 5470 1D2D 3F3EE|
+------------------------+
|pub [revoked 2016-04-16]|
|44D2 0307 1643 E80F 2E31|
|F081 FAFA 6643 7F9F D46F|
+------------------------+
|quanzhou822@gmail.com   |
|https://keybase.io/qzhou|
+------------------------+

[-- Attachment #2: Type: text/html, Size: 2615 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

Hi Quan,

You've goofed up your keys.

srv1.Interface.PrivateKey should be `wg genkey`
srv2.Interface.PrivateKey should be `wg genkey`

srv1.Peer.PublicKey should be `echo srv2.Interface.PrivateKey | wg pubkey`
srv2.Peer.PublicKey should be `echo srv1.Interface.PrivateKey | wg pubkey`

Here's an example:

srv1:
[Interface]
PrivateKey  = qOmcJSeA6Ewp/PYunF1k2LfMMJRhJQO8L1mMAJBwPnE=
[Peer]
PublicKey = /0I75wp3pTBPj2WOU2olEc+MdZfv/yMkEIfWrQlx5hE=

srv2:
[Interface]
PrivateKey = UHBCSeBaz2ydkuObf5itzGT1iT66hEQW3VDYEssk6GY=
[Peer]
PublicKey = PRrLsRYLyCto5CMTqnR71hg722/4Uhbd4Xev0QyW+m4=

You can confirm yourself here that srv1.Peer.PublicKey = `echo
srv2.Interface.PrivateKey | wg pubkey` and that srv2.Peer.PublicKey =
`echo srv1.Interface.PrivateKey | wg pubkey`.

This should fix your problem.

Jason

Re: [WireGuard] Troubleshooting with WireGuard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

On Tue, Jul 12, 2016 at 11:42:28PM +0800, Quan Zhou wrote:
> I'm trying to connect to [2600:3c01:..:1487]:41414 from [2400:6180:...:1]
> 
> I have added the specific IPv6 /128 address to the AllowedIPs on one side,
> and
> on the other side I simply put ::0/0.
> 
> The address I've set was:
> 
> srv1 # ip addr add 10.240.51.2/24 dev wg0
> srv2 # ip addr add 10.240.65.2/24 dev wg0

I think you are confusing IP addresses used on the public Internet and IP
addresses used "inside" the wireguard VPN.  AllowedIPs refers to IP
addresses *inside* the VPN.

From what I gathered, your hosts have public IPv6 addresses, and you are
using this to make the hosts communicate with each other over the public
Internet.  You are then trying to use IPv4 addresses inside the VPN.

If that is the case, then AllowedIPs should refer to the IPv4 addresses
(10.24.XX.XX in your example).

> I will deal with routing later, with OSPF or simply a static route. Right
> now the
> problem is:
> 
> Jul 12 15:32:41 debian kernel: [1081190.595578] Invalid MAC of handshake,
> dropping packet from [2400:...:aec:1]:41414/0%0
> 
> Having no idea about handshake, what should I do next.
> 
> On Tue, Jul 12, 2016 at 5:05 PM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> 
> > Hi Quan,
> >
> > WireGuard is a layer 3 tunnel device, so AllowedIPs should probably be
> > a different IP than endpoint unless you've set up some policy based
> > routing. Also, you'll need to configure the devices with ip-addr too
> > -- what were your commands there?
> >
> > Regards,
> > Jason
> >

> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Daniel Kahn Gillmor]

On Tue 2016-07-12 17:58:58 +0200, Baptiste Jonglez wrote:
> On Tue, Jul 12, 2016 at 11:42:28PM +0800, Quan Zhou wrote:
>> I'm trying to connect to [2600:3c01:..:1487]:41414 from [2400:6180:...:1]
>> 
>> I have added the specific IPv6 /128 address to the AllowedIPs on one side,
>> and
>> on the other side I simply put ::0/0.
>> 
>> The address I've set was:
>> 
>> srv1 # ip addr add 10.240.51.2/24 dev wg0
>> srv2 # ip addr add 10.240.65.2/24 dev wg0
>
> I think you are confusing IP addresses used on the public Internet and IP
> addresses used "inside" the wireguard VPN.  AllowedIPs refers to IP
> addresses *inside* the VPN.

This isn't the first confusion of this type with wireguard.  It too me a
while to figure out and understand the distinction myself.  This
suggests that the documentation could be improved or (maybe and?) that
the choice of configuration names might be suboptimal.

While we're still in the experimental stages is a good time to propose
improvements in naming.  Is there a better name to give either Endpoint
or AllowedIPs (or both)?

   --dkg

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

On Tue, Jul 12, 2016 at 7:46 PM, Daniel Kahn Gillmor
<dkg@fifthhorseman.net> wrote:
> While we're still in the experimental stages is a good time to propose
> improvements in naming.  Is there a better name to give either Endpoint
> or AllowedIPs (or both)?

Endpoint is a good name.
AllowedIPs is a horrible name. But I'm not sure what else to call it.
I'm open to all suggestions.

Re: [WireGuard] Troubleshooting with WireGuard

[Daniel Kahn Gillmor]

On Tue 2016-07-12 19:55:50 +0200, Jason A. Donenfeld wrote:
> Endpoint is a good name.
> AllowedIPs is a horrible name. But I'm not sure what else to call it.
> I'm open to all suggestions.

AllowedTunnelledIPs ?
TunnelledCIDRs ?

               --dkg

Re: [WireGuard] Troubleshooting with WireGuard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 370 bytes --]

On Tue, Jul 12, 2016 at 10:20:50PM +0200, Daniel Kahn Gillmor wrote:
> On Tue 2016-07-12 19:55:50 +0200, Jason A. Donenfeld wrote:
> > Endpoint is a good name.
> > AllowedIPs is a horrible name. But I'm not sure what else to call it.
> > I'm open to all suggestions.
> 
> AllowedTunnelledIPs ?
> TunnelledCIDRs ?

VPNSubnets ?
PeerVPNSubnets ?
InternalIPs ?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Maykel Moya]

On 12/07/16 22:20, Daniel Kahn Gillmor wrote:

Hi, all

First of all, I'd like to thank Jason for wireguard and those packagers
who are making wireguard more easy to install.

Easy and solid crypto for all is a good thing.

> On Tue 2016-07-12 19:55:50 +0200, JRason A. Donenfeld wrote:
>> Endpoint is a good name.
>> AllowedIPs is a horrible name. But I'm not sure what else to call it.
>> I'm open to all suggestions.
> 
> AllowedTunnelledIPs ?
> TunnelledCIDRs ?

Let's bikeshed.

I vote for 'AllowedTunnelledIPs' because:

* It's near to what we have now (AllowedIPs).
* It's simple (not technicisms in the name like 'vpn', 'cidr').
* It's reasonable concise (not like AllowedTunnelledSourceIPs or
AllowedIncomingSourceIPs or whatever).
* The name represents exactly what is under the hood. This value
represents those ips allowed to pop up from the wg iface and not
necessarily the subnets of the peer.

I don't subscribe Baptiste suggestions (VPNSubnets, PeerVPNSubnets,
InternalIPs) because considering the case when you're routing all ip4 or
ip6 through the tunnel, in the 'client' side you will have to allow
0.0.0.0/0 and ::/0 and those are neither internal ips nor subnets of the
peers.

mmoya

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

TunnelIPs ?

Re: [WireGuard] Troubleshooting with WireGuard

[Alex Xu]

On Wed, 13 Jul 2016 10:28:17 +0200
"Jason A. Donenfeld" <Jason@zx2c4.com> wrote:

> TunnelIPs ?
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard

I vote for ReceiveSubnets. I also support PeerSubnets. "CIDR" is also
not too complex because you must know what CIDR is to set the option
properly anyways.

IMO the use of the term "VPN" should be avoided to cases where an
actual private network is being used. Here, it is simply referring to
the subnets on the other side.

I also considered "RemoteSubnets", but that would seem to imply that
that also affects the subnets that are *sent* in the wg tunnel, which
AIUI is actually controlled by routing tables.

For the record as well, the SWAN family of IPsec implementations calls
a similar configuration option "rightsubnet", which is not terrible
but applies poorly in this case, since (AIUI) that one actually
configures the xfrm tables.

Re: [WireGuard] Troubleshooting with WireGuard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

On Wed, Jul 13, 2016 at 08:36:33AM -0400, Alex Xu wrote:
> On Wed, 13 Jul 2016 10:28:17 +0200
> "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
> 
> > TunnelIPs ?
> > _______________________________________________
> > WireGuard mailing list
> > WireGuard@lists.zx2c4.com
> > http://lists.zx2c4.com/mailman/listinfo/wireguard
> 
> I vote for ReceiveSubnets. I also support PeerSubnets. "CIDR" is also
> not too complex because you must know what CIDR is to set the option
> properly anyways.
> 
> IMO the use of the term "VPN" should be avoided to cases where an
> actual private network is being used. Here, it is simply referring to
> the subnets on the other side.

Right, please mentally replace "VPN" by "Tunnel" in the propositions then.

> I also considered "RemoteSubnets", but that would seem to imply that
> that also affects the subnets that are *sent* in the wg tunnel, which
> AIUI is actually controlled by routing tables.

Actually, it does !  This dual usage brings more confusion.  Despite the
name, "AllowedIPs" controls both:

1) packets that are *received* from a peer (by looking at the source IP
   address after decrypting an incoming packet, and only allowing the
   packet if it matches an AllowedIPs rule for this peer)

2) packets that are *sent* through a wireguard interface, where the right
   peer is found by looking for a matching AllowedIPs entry (using the
   destination IP address of the packet, this time).  That's the
   "cryptokey routing" part.

So, the name should reflect this dual usage, which is difficult.

> For the record as well, the SWAN family of IPsec implementations calls
> a similar configuration option "rightsubnet", which is not terrible
> but applies poorly in this case, since (AIUI) that one actually
> configures the xfrm tables.
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Daniel Kahn Gillmor]

On Wed 2016-07-13 18:57:45 +0200, Baptiste Jonglez wrote:
> Actually, it does !  This dual usage brings more confusion.  Despite the
> name, "AllowedIPs" controls both:
>
> 1) packets that are *received* from a peer (by looking at the source IP
>    address after decrypting an incoming packet, and only allowing the
>    packet if it matches an AllowedIPs rule for this peer)
>
> 2) packets that are *sent* through a wireguard interface, where the right
>    peer is found by looking for a matching AllowedIPs entry (using the
>    destination IP address of the packet, this time).  That's the
>    "cryptokey routing" part.

so if a given interface has two peers, their AllowedIPs (or whatever we
end up calling it) are not permitted to overlap?  That constraint should
probably be better documented as well.  It makes sense now that you
describe it, but it wasn't obvious from the current docs.

         --dkg

Re: [WireGuard] Troubleshooting with WireGuard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

On Wed, Jul 13, 2016 at 07:39:21PM +0200, Daniel Kahn Gillmor wrote:
> On Wed 2016-07-13 18:57:45 +0200, Baptiste Jonglez wrote:
> > Actually, it does !  This dual usage brings more confusion.  Despite the
> > name, "AllowedIPs" controls both:
> >
> > 1) packets that are *received* from a peer (by looking at the source IP
> >    address after decrypting an incoming packet, and only allowing the
> >    packet if it matches an AllowedIPs rule for this peer)
> >
> > 2) packets that are *sent* through a wireguard interface, where the right
> >    peer is found by looking for a matching AllowedIPs entry (using the
> >    destination IP address of the packet, this time).  That's the
> >    "cryptokey routing" part.
> 
> so if a given interface has two peers, their AllowedIPs (or whatever we
> end up calling it) are not permitted to overlap?

It's a longest-prefix-match on the destination IP address.  So, the
"AllowedIPs" for two peers can overlap, but then the most specific prefix
will win.  If two peers have the exact same AllowedIPs entry, then I'm not
sure what happens.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

> On Wed 2016-07-13 18:57:45 +0200, Baptiste Jonglez wrote:
>> Actually, it does !  This dual usage brings more confusion.  Despite the
>> name, "AllowedIPs" controls both:
>>
>> 1) packets that are *received* from a peer (by looking at the source IP
>>    address after decrypting an incoming packet, and only allowing the
>>    packet if it matches an AllowedIPs rule for this peer)
>>
>> 2) packets that are *sent* through a wireguard interface, where the right
>>    peer is found by looking for a matching AllowedIPs entry (using the
>>    destination IP address of the packet, this time).  That's the
>>    "cryptokey routing" part.
>

Baptiste is exactly right here. It's a routing table on the way out
and an ACL on the way in.
Now, theoretically, these don't actually have to be the same list.
About a year ago I was experimenting with keeping these separate, but
opted to keep them the same because... which leads me to Daniel's
remark.


On Wed, Jul 13, 2016 at 7:39 PM, Daniel Kahn Gillmor
<dkg@fifthhorseman.net> wrote:
> so if a given interface has two peers, their AllowedIPs (or whatever we
> end up calling it) are not permitted to overlap?  That constraint should
> probably be better documented as well.  It makes sense now that you
> describe it, but it wasn't obvious from the current docs.

If the AllowedIPs overlap, on the way out, it functions in the exact
same way as a routing table: the most specific route is the one that's
chosen.
On the way in, for checking the ACL, the way it works is NOT by (A)
checking to see if that peer has an AllowedIPs that matches the
incoming packet's src. RATHER, (B) it consults the entire cryptokey
routing table, globally, to see to whom this packet would be routed on
the way out, and if that peer matches the receiving peer, then it's
allowed. By making AllowedIPs cover both, we get this nice property of
(B). Namely, with (B) we have a stronger variant of reverse-path
filtering that ties packets to keys. (B) then enables userspace to be
completely sure that the peer they're talking to is necessarily the
one single peer who has a particular public key and specific
AllowedIPs entry. The stronger crypto reverse-path filtering (B) is
actually something that mitigates real vulnerabilities with certain
misconfgured IPsec boxes I've owned (as part of my day job) in the
wild.

You're right -- I should document this. There's a lot of thought that
went into all these little components, and at the time of
documentation-writing, it was hard to enumerate what they all were.
It's good that things are surfacing organically now.

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

On Wed, Jul 13, 2016 at 7:51 PM, Baptiste Jonglez
<baptiste@bitsofnetworks.org> wrote:
> will win.  If two peers have the exact same AllowedIPs entry, then I'm not
> sure what happens.

Fear not, WireGuard has no undefined behavior! The routing table
implementation does not allow for nodes to be duplicated like that.
Only one peer may be assigned to the same node (IP/CIDR). If you run
these commands:

# wg set wg0 peer ABCD allowed-ips 1.2.3.4/32
# wg set wg0 peer EFGH allowed-ips 1.2.3.4/32

Then afterward if you run:

# wg show wg0

You'll see that ABCD has an allowed-ips of "(none)" and EFGH has an
allowed-ips of "1.2.3.4/32". In other words, setting will "move" an
existing entry to the new peer.

Re: [WireGuard] Troubleshooting with WireGuard

[Jason A. Donenfeld]

On Thu, Jul 14, 2016 at 5:08 PM, Maykel Moya <mmoya@mmoya.org> wrote:
> On 13/07/16 10:28, Jason A. Donenfeld wrote:
>
>> TunnelIPs ?
>
> Consider 'TunnelIPs' (suggesting something *related to* the tunnel
> itself) vs. 'TunnelledIPs' (suggesting something *going through* the
> tunnel).

Good point. That's a subtle distinction. I didn't see it quite that way though

TunnelIPs -- IPs that are part of the tunnel, thus, in the tunnel
TunneledIPs -- IPs that are tunneled

So I think it amounts to the same thing....


>
> Happy bikeshedding!

Being a Gentoo dev, theoretically I'm supposed to really thrive during
bike shedding opportunities... blurg.