Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT

We are living in a age of multi levels of NAT and dynamic IP address at times.

Yes I like how wireguard is design.   Current design over looks
dealing with the case of where client and server cannot ip connect to
each other or that the wireguard server does not have a fixed IP
address so changes IP address while clients are off-line or changes IP
address due to being on a dynamic IP so unable to inform clients.
Dynamic DNS updates can be slow where STUN/TURN connections can be
very quick to get update information out there.

**The client configuration contains an initial endpoint of its single
peer (the server), so that it knows where to send encrypted data
before it has received encrypted data.**

Its basically this bit that is the mistake.

**If the server itself changes its own endpoint, and sends data to the
clients, the clients will discover the new server endpoint and update
the configuration just the same.**

If a client is not connected at the time server changes it address
information how is the client going to find it now.   The answer is
with current design client is not going to be able to.   This has been
existing bug bear of existing VPN solutions.

Now if the client config includes fall back STUN/TURN configurations
the server can have moved it IP address the client configuration can
be old and the vpn client able to find it way back to server as long
as the STUN/TURN servers have not disappeared as well.

So a more options to get Endpoint information will make the protocol a
lot more durable.   If client loses connection to server and endpoint
address appears incorrect there need to be a server discovery process.
  Server endpoint discovery on clients could be calling a user-space
program that goes though stun/turn/IRC/xmmp.... options in the
configuration file searching for a current valid endpoint information.
  This is to reduce amount of code kernel side and allow discovery
options to be updated without requiring kernel changes.

There is the possible case that endpoint address is current but not
workable because client has changed to a internet connection where
direction connection between wireguard server and client is not
possible with current design roaming fails  this why TURN need to be
in the mix.   If sending data to a turn server to get it relayed will
not work you are fairly much stuffed.   Adding relay by turn server
could cost kernel side code.

Really we need wireguard vpn clients to be able to start up
connections without valid endpoint information in configuration file
just with valid discovery information list  so able to go use like
STUN/TURN/IRC/xmmp.... and request current endpoint information.   Of
course this means wireguard server need to run a service to answer
endpoint information requests.   Items answering and processing end
point information request could be quite variable and not requiring
lots of different privileges and be shared between user-space and
kernel space implementations it not like you should be running end
point discovery all the time client side.

https://nice.freedesktop.org/wiki/

None of what I am suggesting is required starting from scratch at
first.  libnice includes stun and turn.   Its just the weakness in the
design I see.   Of course some users will want to keep discovery
options to a min other will have no choice but to use them.


Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Jason A. Donenfeld]

Hi Peter,

That's a pretty bombastic and rambling way to put a simple point. I
believe you're conflating two different things. Let's break it down:

1. Dynamic IPs.
2. Both peers behind NAT.

With regard to dynamic IPs, the solution is to use a dynamic DNS
service, and simply use the DNS name as the endpoint. When you set up
the device after being disconnected, userspace will do the DNS lookup
to determine a current IP. When you're already connected to that IP,
WireGuard's roaming capabilities will automatically keep track of IP
updates.

With regard to NAT, this is where you want your STUN-like solution.
However, this does _not_ belong inside of WireGuard. This is where you
simply need some basic userspace utility to handle punching holes in
NAT/firewall, and then bringing up the WireGuard interface with the
discovered address/port. It's not that complicated. In fact, there was
already a substantial discussion about this on the list (did you check
the archives?), and I wound up writing a simple PoC utility for it:
https://git.zx2c4.com/WireGuard/tree/contrib/examples/nat-hole-punching/README
It works well, though it of course needs work before it's
production-ready. Perhaps you could rewrite it to use your glib-based
libnice? Send an email here when you've got a github repo for
something like this, as I'm sure a few others on the list might
benefit from it too.

Jason

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

On Tue, Jan 3, 2017 at 12:18 AM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> Hi Peter,
>
> That's a pretty bombastic and rambling way to put a simple point. I
> believe you're conflating two different things. Let's break it down:
>
> 1. Dynamic IPs.
> 2. Both peers behind NAT.
That misses one completely
3. Server and Peers both behind NAT.   Yes there is a usage case for this one.
>
> With regard to dynamic IPs, the solution is to use a dynamic DNS
> service, and simply use the DNS name as the endpoint. When you set up
> the device after being disconnected, userspace will do the DNS lookup
> to determine a current IP. When you're already connected to that IP,
> WireGuard's roaming capabilities will automatically keep track of IP
> updates.

Dynamic DNS has it weaknesses.   Transparent DNS caching and DNS
access restrictions in some networks mess with the solution you
describe.

https://tools.ietf.org/html/rfc3489

For Voip STUN was developed for many reasons three key reasons.
1. you can username and password protect a STUN server so restricting
the users who can find out about the service.
2 . It does support TLS so encrypted.
3. Information on a STUN server is not replaced to other servers like
lots of dynamic DNS are so in case of attack there are limited sources
of information.

Dynamic DNS option really using a hack that was not suitable for voip
yet for some reason people think it suitable to use for VPN.   Dynamic
DNS is not designed for punching though NAT solutions for the server
address like STUN is or designed to limit access to the address
information like STUN is.

>
> With regard to NAT, this is where you want your STUN-like solution.
> However, this does _not_ belong inside of WireGuard. This is where you
> simply need some basic userspace utility to handle punching holes in
> NAT/firewall, and then bringing up the WireGuard interface with the
> discovered address/port. It's not that complicated. In fact, there was
> already a substantial discussion about this on the list (did you check
> the archives?), and I wound up writing a simple PoC utility for it:
> https://git.zx2c4.com/WireGuard/tree/contrib/examples/nat-hole-punching/README
> It works well, though it of course needs work before it's
> production-ready.
This example the WireGuard server has a public IP address.   The case
I am mentioning Wireguard server may not have a public IP address.

Now STUN will attempt hole punching in the case you don't have a
public IP address for the WireGuard server if the NAT in use are
cooperative.   Of course if you read STUN rfc they state the case
where STUN cannot be used to create a link between server and client
both behind NATs as long as the STUN server is in the open.

TURN relay protocol used in SIP and other usage cases is for when you
don't have a public IP address of your own and you are behind NAT
solutions where you cannot form a direct link between client and
server.
https://tools.ietf.org/html/rfc5766

TURN is not without is overhead.   Now if you are hosting a TURN
server somewhere it does not have to contain any important encryption
information as it just a Relay point.

> Perhaps you could rewrite it to use your glib-based
> libnice? Send an email here when you've got a github repo for
> something like this, as I'm sure a few others on the list might
> benefit from it too.
>

Jason the issue here to be able to use STUN/TURN combination in all
cases I need Wireguard to be able to be directed to use TURN.

The difference is a turn relay server is not holding critical secrets.

Current model is
[Client]-[Server]
What is needed is
[Client]-[Relay]-[Server] with [Client]-[Server] to cover all usage
cases.   Of course the relay being something standard and common
reduces the number of servers that have to be publicly deployed and
maintained.

This is a fault in a lot of VPN designs no relay option because is
always presumed that you can get a public IP the clients can access
for the server and the location where you can get the public IP
address is suitable to house the encryption keys that is not always
the case.   With a TURN server relay option packets passing though the
TURN relay server will remain encoded at the relay server this does
solve the issue where the encryption keys due to privacy acts have to
remain inside a particular country or department.    The only part
that requires a public address when using a TURN is the TURN relay
server.    This is just not reinventing the wheel protocol that is
fairly widely deployed being TURN can provide Relay option.    A VPN
that can run with TURN relays means servers and peers containing
encryption keys don't have to be on the internet exposed IP addresses.

Jason I don't code that much I work on deployments.    I have run into
issues where I don't have a public IP address to use.   I have used
solutions containing libnice  to get me around problems.  Libnice is
not my project.

Please do note needing TURN/Relay comes important even when attempt to
link up clients by IRC, Xmmp or other resolve methods where it may
turn out that both ends are sitting behind NAT where direct
connectivity is impossible.

Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Jason A. Donenfeld]

Hi Peter,

On Thu, Jan 5, 2017 at 12:08 PM, Peter Dolding <oiaohm@gmail.com> wrote:
>> 1. Dynamic IPs.
>> 2. Both peers behind NAT.
> That misses one completely
> 3. Server and Peers both behind NAT.   Yes there is a usage case for this one.

WireGuard has no concept of client/server. There are only peers. So,
when I wrote "both peers behind NAT", I most certainly had in mind
what you refer to as "server and peer". Please reread my answer in
light of this new understanding.

> Dynamic DNS has it weaknesses.   Transparent DNS caching and DNS
> access restrictions in some networks mess with the solution you
> describe.
>
> https://tools.ietf.org/html/rfc3489
>
> For Voip STUN was developed for many reasons three key reasons.
> 1. you can username and password protect a STUN server so restricting
> the users who can find out about the service.
> 2 . It does support TLS so encrypted.
> 3. Information on a STUN server is not replaced to other servers like
> lots of dynamic DNS are so in case of attack there are limited sources
> of information.
>
> Dynamic DNS option really using a hack that was not suitable for voip
> yet for some reason people think it suitable to use for VPN.   Dynamic
> DNS is not designed for punching though NAT solutions for the server
> address like STUN is or designed to limit access to the address
> information like STUN is.

So, as I already wrote in my previous email, implement a STUN tool for
setting up initial WireGuard communications. The example code I linked
to already provides a framework on how this might be done. Just
replace my homebaked hooks with whatever STUN library tickles your
fancy.

> This example the WireGuard server has a public IP address.   The case
> I am mentioning Wireguard server may not have a public IP address.

Um, no. Did you even read the example? Both WireGuard peers have
private IP addresses. Only the NAT-helper server has a public IP
address. This is the same model as STUN. Spend some time actually
reading and studying the work already done on this before wasting more
time with long emails.

> Now STUN will attempt hole punching in the case you don't have a
> public IP address for the WireGuard server if the NAT in use are
> cooperative.   Of course if you read STUN rfc they state the case
> where STUN cannot be used to create a link between server and client
> both behind NATs as long as the STUN server is in the open.

The example code I linked to presents the same model.

> Jason the issue here to be able to use STUN/TURN combination in all
> cases I need Wireguard to be able to be directed to use TURN.

Great. I already outlined how this could be done, and I provided
example code. Plug your STUN or TURN library into that, and you'll be
all set.

No, I'm not going to write it for you. You got a NAT-punching example
from me. You can get a STUN/TURN library from somebody else. All
that's left for you is putting them together.

> Current model is
> [Client]-[Server]
> What is needed is
> [Client]-[Relay]-[Server] with [Client]-[Server] to cover all usage
> cases.   Of course the relay being something standard and common
> reduces the number of servers that have to be publicly deployed and
> maintained.

Yes, I'm aware. And this is exactly what the example code demonstrates
is possible.

> This is a fault in a lot of VPN designs no relay option because is
> always presumed that you can get a public IP the clients can access
> for the server and the location where you can get the public IP
> address is suitable to house the encryption keys that is not always
> the case.   With a TURN server relay option packets passing though the
> TURN relay server will remain encoded at the relay server this does
> solve the issue where the encryption keys due to privacy acts have to
> remain inside a particular country or department.    The only part
> that requires a public address when using a TURN is the TURN relay
> server.    This is just not reinventing the wheel protocol that is
> fairly widely deployed being TURN can provide Relay option.    A VPN
> that can run with TURN relays means servers and peers containing
> encryption keys don't have to be on the internet exposed IP addresses.

Great, so plug your TURN library into some resembling the example I
wrote, and you'll be all set.

> Jason I don't code that much I work on deployments.    I have run into
> issues where I don't have a public IP address to use.   I have used
> solutions containing libnice  to get me around problems.  Libnice is
> not my project.

Then integrate libnice into my example code, and you'll have what you want.

> Please do note needing TURN/Relay comes important even when attempt to
> link up clients by IRC, Xmmp or other resolve methods where it may
> turn out that both ends are sitting behind NAT where direct
> connectivity is impossible.

Yes, I'm aware. Again, see example code. Replace the homebrewed
structures with a STUN/TURN library.

Please do not write again until you've read the example code and taken
the time to understand what it does.

Regards,
Jason

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

On Fri, Jan 6, 2017 at 6:33 AM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> Hi Peter,
>
> On Thu, Jan 5, 2017 at 12:08 PM, Peter Dolding <oiaohm@gmail.com> wrote:
>>> 1. Dynamic IPs.
>>> 2. Both peers behind NAT.
>> That misses one completely
>> 3. Server and Peers both behind NAT.   Yes there is a usage case for this one.
>
> WireGuard has no concept of client/server. There are only peers. So,
> when I wrote "both peers behind NAT", I most certainly had in mind
> what you refer to as "server and peer". Please reread my answer in
> light of this new understanding.
>
>> Dynamic DNS has it weaknesses.   Transparent DNS caching and DNS
>> access restrictions in some networks mess with the solution you
>> describe.
>>
>> https://tools.ietf.org/html/rfc3489
>>
>> For Voip STUN was developed for many reasons three key reasons.
>> 1. you can username and password protect a STUN server so restricting
>> the users who can find out about the service.
>> 2 . It does support TLS so encrypted.
>> 3. Information on a STUN server is not replaced to other servers like
>> lots of dynamic DNS are so in case of attack there are limited sources
>> of information.
>>
>> Dynamic DNS option really using a hack that was not suitable for voip
>> yet for some reason people think it suitable to use for VPN.   Dynamic
>> DNS is not designed for punching though NAT solutions for the server
>> address like STUN is or designed to limit access to the address
>> information like STUN is.
>
> So, as I already wrote in my previous email, implement a STUN tool for
> setting up initial WireGuard communications. The example code I linked
> to already provides a framework on how this might be done. Just
> replace my homebaked hooks with whatever STUN library tickles your
> fancy.
>
>> This example the WireGuard server has a public IP address.   The case
>> I am mentioning Wireguard server may not have a public IP address.
>
> Um, no. Did you even read the example? Both WireGuard peers have
> private IP addresses. Only the NAT-helper server has a public IP
> address. This is the same model as STUN. Spend some time actually
> reading and studying the work already done on this before wasting more
> time with long emails.
>
>> Now STUN will attempt hole punching in the case you don't have a
>> public IP address for the WireGuard server if the NAT in use are
>> cooperative.   Of course if you read STUN rfc they state the case
>> where STUN cannot be used to create a link between server and client
>> both behind NATs as long as the STUN server is in the open.
>
> The example code I linked to presents the same model.
>
It is not the same model some critical is missing.

You example gets you a connection.   You example does not cope with IP
change as that happens in NAT environments.


You need a particular pattern of operations.

NAT Hole punch/dynamic DNS resolve.
Start VPN.
VPN detects connection lost triggers resolve again to check if the IP
address/port it is using is still current and correct.

This way VPN does not fall over and die.   Implementing STUN
completely I need to way of connecting an application to Wireguard
that Wireguard will resort to when connection fails keep alive message
and before informing applications or users VPN is lost.

So I need how to connect resolve to Wireguard so it can be done as
part of maintaining VPN connection.   This connect resolve to be
triggered when keep alive fails need to happen when using a dynamic
DNS.

The problem you have is when you find out your IP address on the
Wireguard server as changed then attempt to inform clients of
Wireguard server by wireguard that the IP address has changed this is
going to fail in NAT guarded clients.   Why NAT rejected IP addresses
that clients behind nat have not attempted to contact.   This is why
you example only really works with Wireguard clients behind nat with
dyanmic IP but if you put a Wireguard Server behind a nat with a
dynamic IP your example code completely fails.

Wireguard idea that Wireguard server can update clients when server IP
address changes only works when you have two public IP addresses old
and new so you can send change messages from old IP address and have
new IP address receive clients.   Problem is reality in dynamic IP
address you don't know what your new IP address will be until after
you have change IP address to the new IP address.   So wireguard
design of how to make server handle changing IP address fails in real
world.  Server changing IP address need clients to drop back to a
resolve solution and when server is behind nat and needing hole
punching rerunning hole punching.

The reality is you cannot run resolve just once you are operating in
dynamic IP address with NAT.



>> Jason the issue here to be able to use STUN/TURN combination in all
>> cases I need Wireguard to be able to be directed to use TURN.
>
> Great. I already outlined how this could be done, and I provided
> example code. Plug your STUN or TURN library into that, and you'll be
> all set.
>
> No, I'm not going to write it for you. You got a NAT-punching example
> from me. You can get a STUN/TURN library from somebody else. All
> that's left for you is putting them together.
>> Current model is
>> [Client]-[Server]
>> What is needed is
>> [Client]-[Relay]-[Server] with [Client]-[Server] to cover all usage
>> cases.   Of course the relay being something standard and common
>> reduces the number of servers that have to be publicly deployed and
>> maintained.
>
> Yes, I'm aware. And this is exactly what the example code demonstrates
> is possible.
>
No relay is something different to what you demoed.   TURN is relay.
When using TURN Wireguard clients/server would not be able to connect
to each other so would be sending all packets by the TURN server.

Relay is not exactly possible to performing with existing code.   From
what I have seen of wireguard I cannot tell it to use a standard
socket file.  Like /var/lib/mysql/mysql.sock with mysql.   Because
when you are going to be relaying all packets you don't need a IP
addressed port.   Ok now we have extra performance overhead and over
expanded packet attempting to get into TURN effecting every packet
sent by current Wireguard design.

Current from client packets roughly look like(yes this is rough I have
skipped over lots.
cleint send [destination IP address of wireguard server][source IP
address of wireguard client][wirguard data]
server send [destination IP address of wireguard client ][source IP
address of wireguard server ][wirguard data]
In Turn it need to do the following after connection stuff on the turn
server is setup.
wireguard cleint send [destination IP address turn server][source IP
address of wireguard client][Turn channel ID for server][wirguard
data]
wireguard server send [destination IP address turn server][source IP
address of wireguard server][Turn channel ID for client][wirguard
data]

Of course all packets the wireguard client or server would be
receiving would have address of TURN server because it a relay.

So current design would be rewriting all the packet headers to send
wireguard to TURN.   This is why with TURN it would be way better
built into the base design.

If connection fails due to any number of causes a connection using
TURN need to some how trigger running resolve again.

 Using TURN you basically are not using IP addresses any more but an
abstraction system.   Using TURN you can technically send packets
between two~16 thousand clients with identical IP address.

Relay solution is basically for when you cannot punch a hole through
the nat or have direct connection between client/server.

What you code demonstrates is a resolve server with client code to
interface with it.    This is what STUN is.   STUN is just a rfc
standard defining how to go about implementing a resolve server and
clients with all the nice stuff.

STUN rfc goes and documents all the ways a resolve sever solution with
NAT will hit brick walls.   So you want to design something for
punching holes in NAT read the STUN rfc completely.
 https://tools.ietf.org/html/rfc5389

Basically you have been doing NIH syndrome.   The punching though NAT
has been done very completely with STUN to make viop and other
services like it work.   In fact STUN has done it about as far as can
be done this is why TURN support for relaying was added as it own rfc
to cover the cases where there is no way to punch though the NAT.

My step one is somehow have wireguard support calling/registering with
a userspace resolve program for when connection fails.   The one thing
about implementing going though nat with dyanmic IP address is that
connection failure is a given.  The key trick is handling connection
failure not making the connection.   Basically punching a NAT hole to
make a connection in most cases is either easy or not possible
maintaining the hole is the hard bit because nats can clear the
information ip address can change.....  Wireguard servers and clients
have a keep alive message these messages not turning up can be a clear
sign that resolve program need to rerun.

I am not going to write something that I know end users are going to
have trouble with.   Both STUN/TURN mandates that resolve interface to
be workable.

Basically you did the easy bit and punched the hole and have missed
all the steps required to maintain that hole as the network
environment changes around you.      Resolver should not be running on
the clients all the time and should not need to be perfectly fast just
run when ever the connection looks dead to make sure settings are
right before calling the connection absolutely dead if the settings
are wrong correct settings and attempt to rejoin up.  This is a
interface for auto healing of failures..

Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Dan Lüdtke]

Hi Peter,

I followed this thread and like to express some concerns. Although I see the=
 problem and ran into it myself, I would like to see a solution outside the w=
ireguard code. Like the one Jason proposed or even a new approach. I am afra=
id that network layers problems (legacy IP and especially NAT) are about to u=
glify yet another beautiful protocol. None of the problems stated in this th=
read have I ever observed in an dual stack or in a IP (read IPv6) environmen=
t. This is all inconvenience that legacy IP (read IPv4) brings and that is c=
aused by ten+ years of overprovisioning and not taking care of the network l=
ayer of the infrastructure. It is 2017, run IPv6. There should not be a sing=
le line of code in wireguard that deals with broken infrastructure. There is=
 plenty of room for all kinds of workarounds in the userspace. Like the scri=
pts in the Wireguard repository. I see the problem, agree on it, but It is o=
ut of the scope for wireguard to solve. The infrastructure must be able to s=
omehow connect to peers via UDP. That is I think the least one can expect fr=
om a network layer. Whatever _outside_ magic it may need due to historical p=
rotocol usage.

My concerns expressed and all that said, I would love to see some code or Po=
C. Code and pcaps are king :)

I solved the problem using ipv6 only when I ran into it. May require to fina=
lly invest in state of the art layer 3 protocol usage in some cases. However=
, it's overdue anyway. Wireguards roaming feature tool care of the sites whe=
re even the ipv6 prefix changes from time to time. HTH.

Cheers,

Dan

> On 9 Jan 2017, at 14:43, Peter Dolding <oiaohm@gmail.com> wrote:
>=20
>> On Fri, Jan 6, 2017 at 6:33 AM, Jason A. Donenfeld <Jason@zx2c4.com> wrot=
e:
>> Hi Peter,
>>=20
>> On Thu, Jan 5, 2017 at 12:08 PM, Peter Dolding <oiaohm@gmail.com> wrote:
>>>> 1. Dynamic IPs.
>>>> 2. Both peers behind NAT.
>>> That misses one completely
>>> 3. Server and Peers both behind NAT.   Yes there is a usage case for thi=
s one.
>>=20
>> WireGuard has no concept of client/server. There are only peers. So,
>> when I wrote "both peers behind NAT", I most certainly had in mind
>> what you refer to as "server and peer". Please reread my answer in
>> light of this new understanding.
>>=20
>>> Dynamic DNS has it weaknesses.   Transparent DNS caching and DNS
>>> access restrictions in some networks mess with the solution you
>>> describe.
>>>=20
>>> https://tools.ietf.org/html/rfc3489
>>>=20
>>> For Voip STUN was developed for many reasons three key reasons.
>>> 1. you can username and password protect a STUN server so restricting
>>> the users who can find out about the service.
>>> 2 . It does support TLS so encrypted.
>>> 3. Information on a STUN server is not replaced to other servers like
>>> lots of dynamic DNS are so in case of attack there are limited sources
>>> of information.
>>>=20
>>> Dynamic DNS option really using a hack that was not suitable for voip
>>> yet for some reason people think it suitable to use for VPN.   Dynamic
>>> DNS is not designed for punching though NAT solutions for the server
>>> address like STUN is or designed to limit access to the address
>>> information like STUN is.
>>=20
>> So, as I already wrote in my previous email, implement a STUN tool for
>> setting up initial WireGuard communications. The example code I linked
>> to already provides a framework on how this might be done. Just
>> replace my homebaked hooks with whatever STUN library tickles your
>> fancy.
>>=20
>>> This example the WireGuard server has a public IP address.   The case
>>> I am mentioning Wireguard server may not have a public IP address.
>>=20
>> Um, no. Did you even read the example? Both WireGuard peers have
>> private IP addresses. Only the NAT-helper server has a public IP
>> address. This is the same model as STUN. Spend some time actually
>> reading and studying the work already done on this before wasting more
>> time with long emails.
>>=20
>>> Now STUN will attempt hole punching in the case you don't have a
>>> public IP address for the WireGuard server if the NAT in use are
>>> cooperative.   Of course if you read STUN rfc they state the case
>>> where STUN cannot be used to create a link between server and client
>>> both behind NATs as long as the STUN server is in the open.
>>=20
>> The example code I linked to presents the same model.
>>=20
> It is not the same model some critical is missing.
>=20
> You example gets you a connection.   You example does not cope with IP
> change as that happens in NAT environments.
>=20
>=20
> You need a particular pattern of operations.
>=20
> NAT Hole punch/dynamic DNS resolve.
> Start VPN.
> VPN detects connection lost triggers resolve again to check if the IP
> address/port it is using is still current and correct.
>=20
> This way VPN does not fall over and die.   Implementing STUN
> completely I need to way of connecting an application to Wireguard
> that Wireguard will resort to when connection fails keep alive message
> and before informing applications or users VPN is lost.
>=20
> So I need how to connect resolve to Wireguard so it can be done as
> part of maintaining VPN connection.   This connect resolve to be
> triggered when keep alive fails need to happen when using a dynamic
> DNS.
>=20
> The problem you have is when you find out your IP address on the
> Wireguard server as changed then attempt to inform clients of
> Wireguard server by wireguard that the IP address has changed this is
> going to fail in NAT guarded clients.   Why NAT rejected IP addresses
> that clients behind nat have not attempted to contact.   This is why
> you example only really works with Wireguard clients behind nat with
> dyanmic IP but if you put a Wireguard Server behind a nat with a
> dynamic IP your example code completely fails.
>=20
> Wireguard idea that Wireguard server can update clients when server IP
> address changes only works when you have two public IP addresses old
> and new so you can send change messages from old IP address and have
> new IP address receive clients.   Problem is reality in dynamic IP
> address you don't know what your new IP address will be until after
> you have change IP address to the new IP address.   So wireguard
> design of how to make server handle changing IP address fails in real
> world.  Server changing IP address need clients to drop back to a
> resolve solution and when server is behind nat and needing hole
> punching rerunning hole punching.
>=20
> The reality is you cannot run resolve just once you are operating in
> dynamic IP address with NAT.
>=20
>=20
>=20
>>> Jason the issue here to be able to use STUN/TURN combination in all
>>> cases I need Wireguard to be able to be directed to use TURN.
>>=20
>> Great. I already outlined how this could be done, and I provided
>> example code. Plug your STUN or TURN library into that, and you'll be
>> all set.
>>=20
>> No, I'm not going to write it for you. You got a NAT-punching example
>> from me. You can get a STUN/TURN library from somebody else. All
>> that's left for you is putting them together.
>>> Current model is
>>> [Client]-[Server]
>>> What is needed is
>>> [Client]-[Relay]-[Server] with [Client]-[Server] to cover all usage
>>> cases.   Of course the relay being something standard and common
>>> reduces the number of servers that have to be publicly deployed and
>>> maintained.
>>=20
>> Yes, I'm aware. And this is exactly what the example code demonstrates
>> is possible.
>>=20
> No relay is something different to what you demoed.   TURN is relay.
> When using TURN Wireguard clients/server would not be able to connect
> to each other so would be sending all packets by the TURN server.
>=20
> Relay is not exactly possible to performing with existing code.   From
> what I have seen of wireguard I cannot tell it to use a standard
> socket file.  Like /var/lib/mysql/mysql.sock with mysql.   Because
> when you are going to be relaying all packets you don't need a IP
> addressed port.   Ok now we have extra performance overhead and over
> expanded packet attempting to get into TURN effecting every packet
> sent by current Wireguard design.
>=20
> Current from client packets roughly look like(yes this is rough I have
> skipped over lots.
> cleint send [destination IP address of wireguard server][source IP
> address of wireguard client][wirguard data]
> server send [destination IP address of wireguard client ][source IP
> address of wireguard server ][wirguard data]
> In Turn it need to do the following after connection stuff on the turn
> server is setup.
> wireguard cleint send [destination IP address turn server][source IP
> address of wireguard client][Turn channel ID for server][wirguard
> data]
> wireguard server send [destination IP address turn server][source IP
> address of wireguard server][Turn channel ID for client][wirguard
> data]
>=20
> Of course all packets the wireguard client or server would be
> receiving would have address of TURN server because it a relay.
>=20
> So current design would be rewriting all the packet headers to send
> wireguard to TURN.   This is why with TURN it would be way better
> built into the base design.
>=20
> If connection fails due to any number of causes a connection using
> TURN need to some how trigger running resolve again.
>=20
> Using TURN you basically are not using IP addresses any more but an
> abstraction system.   Using TURN you can technically send packets
> between two~16 thousand clients with identical IP address.
>=20
> Relay solution is basically for when you cannot punch a hole through
> the nat or have direct connection between client/server.
>=20
> What you code demonstrates is a resolve server with client code to
> interface with it.    This is what STUN is.   STUN is just a rfc
> standard defining how to go about implementing a resolve server and
> clients with all the nice stuff.
>=20
> STUN rfc goes and documents all the ways a resolve sever solution with
> NAT will hit brick walls.   So you want to design something for
> punching holes in NAT read the STUN rfc completely.
> https://tools.ietf.org/html/rfc5389
>=20
> Basically you have been doing NIH syndrome.   The punching though NAT
> has been done very completely with STUN to make viop and other
> services like it work.   In fact STUN has done it about as far as can
> be done this is why TURN support for relaying was added as it own rfc
> to cover the cases where there is no way to punch though the NAT.
>=20
> My step one is somehow have wireguard support calling/registering with
> a userspace resolve program for when connection fails.   The one thing
> about implementing going though nat with dyanmic IP address is that
> connection failure is a given.  The key trick is handling connection
> failure not making the connection.   Basically punching a NAT hole to
> make a connection in most cases is either easy or not possible
> maintaining the hole is the hard bit because nats can clear the
> information ip address can change.....  Wireguard servers and clients
> have a keep alive message these messages not turning up can be a clear
> sign that resolve program need to rerun.
>=20
> I am not going to write something that I know end users are going to
> have trouble with.   Both STUN/TURN mandates that resolve interface to
> be workable.
>=20
> Basically you did the easy bit and punched the hole and have missed
> all the steps required to maintain that hole as the network
> environment changes around you.      Resolver should not be running on
> the clients all the time and should not need to be perfectly fast just
> run when ever the connection looks dead to make sure settings are
> right before calling the connection absolutely dead if the settings
> are wrong correct settings and attempt to rejoin up.  This is a
> interface for auto healing of failures..
>=20
> Peter Dolding
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Jason A. Donenfeld]

Hi Peter,

On Mon, Jan 9, 2017 at 2:43 PM, Peter Dolding <oiaohm@gmail.com> wrote:
> You example gets you a connection.   You example does not cope with IP
> change as that happens in NAT environments.

Yes, example only code.

> VPN detects connection lost triggers resolve again to check if the IP
> address/port it is using is still current and correct.

As mentioned elsewhere, at some point there will be userspace
notification for these sorts of events. I agree it's necessary for
this kind of detection.

> if you put a Wireguard Server behind a nat with a
> dynamic IP your example code completely fails.

Yea. This was similarly discussed in the other thread from Emmanuel.
That discussion veered toward a multiendpoint-like solution. What
you're suggesting, here, is a STUN-based solution, which depends on
userspace connectivity notification. Your suggestion indeed is
interesting to me.

> The reality is you cannot run resolve just once you are operating in
> dynamic IP address with NAT.

If the server is behind a dynamic IP firewalled situation, right.

> Relay is not exactly possible to performing with existing code.   From
> what I have seen of wireguard I cannot tell it to use a standard
> socket file.  Like /var/lib/mysql/mysql.sock with mysql.   Because
> when you are going to be relaying all packets you don't need a IP
> addressed port.   Ok now we have extra performance overhead and over
> expanded packet attempting to get into TURN effecting every packet
> sent by current Wireguard design.

I think you could set the WireGuard endpoint to be localhost pointing
to some TURN daemon to do it in userspace?

By the way, would that kind of hack be an acceptable way to go about
STUN for you? A userspace daemon that forwards packets to a
dynamically changing IP? Maybe that's simpler to implement than the
userspace state notification stuff I mentioned before.

> So current design would be rewriting all the packet headers to send
> wireguard to TURN.   This is why with TURN it would be way better
> built into the base design.

There's zero chance I'm going to incorporate TURN into the wireguard
core protocol, but I am amenable to discussing ways in which it could
be built on top in some nice composable fashion.

> STUN rfc goes and documents all the ways a resolve sever solution with
> NAT will hit brick walls.   So you want to design something for
> punching holes in NAT read the STUN rfc completely.
>  https://tools.ietf.org/html/rfc5389

Yes, I know what STUN is; I know what TURN is; I've read your linked RFC.

> Basically you have been doing NIH syndrome.

No I haven't. I made some quick example code as a PoC in a hundred
lines of C, so that somebody wanting to incorporate STUN later on
would have something concrete to visualize the design. Maybe that
person is you? Perhaps you'll be the one to contribute something
useful.

> My step one is somehow have wireguard support calling/registering with
> a userspace resolve program for when connection fails.

I'll work on this and give some thought about the cleanest solution.

> I am not going to write something that I know end users are going to
> have trouble with.   Both STUN/TURN mandates that resolve interface to
> be workable.

I don't understand your sentence. Are you writing these long rambling
emails because you intend to write some code? Or because you don't?
I'm a bit lost by your intentions.

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Jason A. Donenfeld]

On Sun, Jan 15, 2017 at 9:39 AM, Dan L=C3=BCdtke <mail@danrl.com> wrote:
> Although I see the problem and ran into it myself, I would like to see a =
solution outside the
> wireguard code. Like the one Jason proposed or even a new approach. I am =
afraid that
> network layers problems (legacy IP and especially NAT) are about to uglif=
y yet another
> beautiful protocol.

Yea -- worry not. I'm not going to add big cludges into core
WireGuard. I would like to provide some useful facilities for people
to do interesting composable solutions to disgusting networking
problems. But I think this solution space is more in the realm of
"API" than "protocol".

I could also imagine people making "wireguard UDP proxy daemons" --
little programs that listen on 127.0.0.1:xxxxx and then forward
packets to some dynamically learned MySQL-connected ASN1-parsed IP
while doing things like "if multiple packets that start with a 0x1 and
are 148 bytes long are sent in a row, the server has probably changed
IPs and we should get STUNed again".

Or, maybe this kind of proxy is objectionable and people would prefer
to use netlink notification for connectivity events instead.

Either way, there's plenty of room for building terrible things _on
top of_, rather than inside of, wireguard.

> My concerns expressed and all that said, I would love to see some code or=
 PoC. Code and pcaps are king :)

:) As the Reverend Doctor Pastor says, PoC||GTFO.

> Wireguards roaming feature tool care of the sites where even the ipv6 pre=
fix changes from time to time.

Or when your laptop or cellphone is moving around between IP addresses
frequently.

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

On Sun, Jan 15, 2017 at 6:39 PM, Dan L=C3=BCdtke <mail@danrl.com> wrote:
> Hi Peter,
>
> I followed this thread and like to express some concerns. Although I see =
the problem and ran into it myself, I would like to see a solution outside =
the wireguard code. Like the one Jason proposed or even a new approach. I a=
m afraid that network layers problems (legacy IP and especially NAT) are ab=
out to uglify yet another beautiful protocol. None of the problems stated i=
n this thread have I ever observed in an dual stack or in a IP (read IPv6) =
environment. This is all inconvenience that legacy IP (read IPv4) brings an=
d that is caused by ten+ years of overprovisioning and not taking care of t=
he network layer of the infrastructure. It is 2017, run IPv6. There should =
not be a single line of code in wireguard that deals with broken infrastruc=
ture. There is plenty of room for all kinds of workarounds in the userspace=
. Like the scripts in the Wireguard repository. I see the problem, agree on=
 it, but It is out of the scope for wireguard to solve. The infrastructure =
must be able to somehow connect to peers via UDP. That is I think the least=
 one can expect from a network layer. Whatever _outside_ magic it may need =
due to historical protocol usage.
>
> My concerns expressed and all that said, I would love to see some code or=
 PoC. Code and pcaps are king :)
>
> I solved the problem using ipv6 only when I ran into it. May require to f=
inally invest in state of the art layer 3 protocol usage in some cases. How=
ever, it's overdue anyway. Wireguards roaming feature tool care of the site=
s where even the ipv6 prefix changes from time to time. HTH.
>
> Cheers,
>
> Dan
>
https://mirrors.bieringer.de/Linux+IPv6-HOWTO/nat-netfilter6..html

I am sorry to say double nat happens even in IPv6 not common in most
countries.   You find it when you have country firewalls.    Nat
issues are not fixed by going IPv6 only reduced how often people will
hit them.  Only delayed until the point that was giving you NAT issues
under IPv4 decides to update their hardware to IPv6 and leaves the
same Nat settings in place.


IPv6 has all the same nat configurations as IPv4.   The larger address
range of ipv6 means you should not need to do it.  But if country or
company wants to create a internet access limitation point  forcing
all traffic to be NAT converted before internet access is one of those
things.

Calling NAT legacy in IPv6 has something very wrong.   If you are
behind a nat that is IPv6 aware even attempting to tunnel out by ipv4
does not work.    Even if everyone was using IPv6 right now there
would still be users due to country and carrier ideals behind all the
types of NAT STUN rfc documents.

If NAT was not part of IPv6 systems in the same forms as what was in
IPv4 you would have an arguement.   You have made a big error in
thinking that the problem of punching though NAT disappears when IPv6
is deployed.   The need to punch though NAT with IPv6 reduces are
places switch to IPv6 but you have the stubborn hold outs(government
regulations stating nats be used) or the incompetent operators who
duplicate configuration from IPv4 to IPv6 so leaving carrier NAT in
place or Carriers intentionally putting NAT to have two tier pricing
for business and home.   Yes that dual stack stunt you are currently
using to get around NAT if you carrier upgrades to a smarter IPv6/IPv4
NAT is going to fail over night as your direct connection by IPv6
disappears.

Why would a carrier intentionally deploy a NAT that STUN and other NAT
punching solutions cannot cope with even when you have IPv6 simple so
they can charge 2 different plans 1 for home and 1 for business of
course those on home who are not meant to be running servers get stuck
behind the NAT.     NAT is not just about over-provisioning it also
about being a jerk.    If NAT was only used for over-provisioning then
IPv6 increased addresses would make it disappear problem is it also
used to limited what users can do with their ISP account.

Dan your IPv6 work around only working because you are not seeing
ip/v6 aware NATs.     In a world of pure IPV6  like it or not NAT
punching and working around is part of it.     Wireguard current
roaming feature will fail when it hitting different IPv6 nats as the
change address packet will be blocked for the same reason as being
from a IP address that the NAT has not communicated with.    Dan you
are not alone thinking that IPv6 fixes more than what it does because
their carriers have not updated their hardware or their carriers have
decided for IPv6 to reduce the NAT count..

Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Dan Lüdtke]

Two things I have not seen so far:
- government regulations that enforce NAT
- ISPs (let alone carriers) "upgrading" their networks to ipv6 nat (i myself=
 have run both, isp + carrier networks, and i call BS on your future outlook=
 regarding nat ipv6)
- code from you in this thread



> On 18 Jan 2017, at 06:55, Peter Dolding <oiaohm@gmail.com> wrote:
>=20
> government
> regulations stating nats be used

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

On Sun, Jan 15, 2017 at 8:40 PM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> Hi Peter,
>
> On Mon, Jan 9, 2017 at 2:43 PM, Peter Dolding <oiaohm@gmail.com> wrote:
>> You example gets you a connection.   You example does not cope with IP
>> change as that happens in NAT environments.
>
> Yes, example only code.
>
>> VPN detects connection lost triggers resolve again to check if the IP
>> address/port it is using is still current and correct.
>
> As mentioned elsewhere, at some point there will be userspace
> notification for these sorts of events. I agree it's necessary for
> this kind of detection.
>
>> if you put a Wireguard Server behind a nat with a
>> dynamic IP your example code completely fails.
>
> Yea. This was similarly discussed in the other thread from Emmanuel.
> That discussion veered toward a multiendpoint-like solution. What
> you're suggesting, here, is a STUN-based solution, which depends on
> userspace connectivity notification. Your suggestion indeed is
> interesting to me.
>
>> The reality is you cannot run resolve just once you are operating in
>> dynamic IP address with NAT.
>
> If the server is behind a dynamic IP firewalled situation, right.

Firewall or NAT.   IPv6 has NATs not seen in large numbers these don't
have to be officially called firewalls.   Just like IPv4 NAT they go
has the source address be contacted by someone behind the NAT if no
drop packet.

Yes a dynamic firewall could be configured to have the same behaviour
as a IPv6 nat.   Routers provided by ISPs will most likely still have
a IPv6 nat in time so ISP can allocate 1 ipv6 address per customer for
simple accounting.

So mutable machines behind 1 address is still in ipv6 world.    Ipv6
address change could be local ipv6 address change by router and the
router being stupid.   Routers being buggy I don't see that
disappearing any time soon either.
>
>> Relay is not exactly possible to performing with existing code.   From
>> what I have seen of wireguard I cannot tell it to use a standard
>> socket file.  Like /var/lib/mysql/mysql.sock with mysql.   Because
>> when you are going to be relaying all packets you don't need a IP
>> addressed port.   Ok now we have extra performance overhead and over
>> expanded packet attempting to get into TURN effecting every packet
>> sent by current Wireguard design.
>
> I think you could set the WireGuard endpoint to be localhost pointing
> to some TURN daemon to do it in userspace?
>
> By the way, would that kind of hack be an acceptable way to go about
> STUN for you? A userspace daemon that forwards packets to a
> dynamically changing IP? Maybe that's simpler to implement than the
> userspace state notification stuff I mentioned before.

The issue for a socket is when you use a file socket you can skip
putting the IP packet  and UDP information on the data to be sent.
So that the system sending the data out by TURN or some other protocol
is not having to strip ip and UDP information put on by Wireguard
before adding the ip/tcp/udp information the relay server wants to
keep packet as compact as possible.   Of course the relay sever header
size is not going to match the Wireguards packet in current design so
you cannot edit the already allocated packet memory to make the packet
being sent on to relay..

Using localhost means calculating checksums and creating real packets
  This means wireguard would be calculating checksum on every packet
TURN relay got from localhost then TURN relay would have to be
calculating a checksum for every packet it sends to wireguard what are
both really wasting CPU time if the UDP checksum is not important to
wireguard.    If that checksum

File Socket you would have a process per connection.    So for TURN if
it not going to be in kernel is a wireguard support file socket mode
without IP/UDP information as anything relaying will be straight up
junking this information and putting the information on the relay
protocol needs.   Wireguard would be able to tell what connection its
meant to be by the process..

In fact wireguard for supporting Relay solution asking for so much
blank space/ignore space.   So a relay solution knows is need x number
of bytes for its ip/nat/relayid and is able to tell wireguard to send
to packets with that free.  Of course wireguard could also process
packets with the relay solutions ip/udp/relayid on by knowing how far
to skip forwards so avoiding reworking memory.

Using STUN without TURN it only need to be informed when connection
has disappeared to NAT punch the connection back open.   Otherwise a
successful NAT punching works exactly like a normal connection.

Even so providing a file socket method would allow STUN in the middle
to be fully dynamic.   By fully dynamic I mean changing from direct
connection to NAT punching to TURN all without having to stop the VPN.

Now if for some reason Wiregaurd was being used between containers
split by network namespace could be sharing a file namespace and use a
socket this is also where local address fails.

>
>> So current design would be rewriting all the packet headers to send
>> wireguard to TURN.   This is why with TURN it would be way better
>> built into the base design.
>
> There's zero chance I'm going to incorporate TURN into the wireguard
> core protocol, but I am amenable to discussing ways in which it could
> be built on top in some nice composable fashion.

I should be more correct built into the base design.  I include this
as supporting file sockets so you can cleanly be used by something
relay like TURN without items a Relaying program really does not need.

So when you draw a a diagram of how wireguard works there is a relay
interfaces go here box this  could be in userspace would be still
being part of the base design in my eyes.

TURN protocol is just a good standard relay protocol that shows what
is needed and what is not.     So going though the process of
implementing TURN with the least modifications to wireguard at the
best performance will cover fairly much every case.

Going though a relay and not having excess cpu usage or memory
allocations takes serous thinking.   Its really easy to end up having
to duplicate memory to modify packets to add the information relay
server needs or waste time calculating checksums that are not
important when attempting to relay that could have be not generated in
the first place.

Issue is going relay is not the same as going direct.   Its a
different problem because no matter the relay there is always extra
information need at the start after the either TCP/UDP header to tell
the relay what end point to relay the data to.   So a relay packet
always bigger than standard packet.   Relays don't always accept
fragmented packets either.

Supporting a relay be it turn or any other relay solution is in fact
different to direct connections.   It is really easy to end up round
peg square hole.  vpn set-up for direct connection attempting to be
sent by relay can have time out issues due to overhead of relay and
userspace relay program needing to break up packets then have assembly
problems


If wireguard design include a design for doing something like TURN
without massive pain possible.    Now if the design like it current
does does not include anything for doing relay.

>
>> STUN rfc goes and documents all the ways a resolve sever solution with
>> NAT will hit brick walls.   So you want to design something for
>> punching holes in NAT read the STUN rfc completely.
>>  https://tools.ietf.org/html/rfc5389
>
> Yes, I know what STUN is; I know what TURN is; I've read your linked RFC.
>
>> Basically you have been doing NIH syndrome.
>
> No I haven't. I made some quick example code as a PoC in a hundred
> lines of C, so that somebody wanting to incorporate STUN later on
> would have something concrete to visualize the design. Maybe that
> person is you? Perhaps you'll be the one to contribute something
> useful.
>
>> My step one is somehow have wireguard support calling/registering with
>> a userspace resolve program for when connection fails.
>
> I'll work on this and give some thought about the cleanest solution.
>
>> I am not going to write something that I know end users are going to
>> have trouble with.   Both STUN/TURN mandates that resolve interface to
>> be workable.
>
I am not sure if I will implement code.   But with the problems I have
in different places I may end up doing so.

Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

On Wed, Jan 18, 2017 at 4:11 PM, Dan L=C3=BCdtke <mail@danrl.com> wrote:
> Two things I have not seen so far:
> - government regulations that enforce NAT
> - ISPs (let alone carriers) "upgrading" their networks to ipv6 nat (i mys=
elf have run both, isp + carrier networks, and i call BS on your future out=
look regarding nat ipv6)
> - code from you in this thread
>
https://en.wikipedia.org/wiki/Internet_censorship
When you start looking into countries that are red in the "World map
showing the status of YouTube blocking"  you will find some of those
its mandatory to have a NAT between ISP and open internet even for
IPv6.  Yes the area of infect users is currently small.  But when you
look a countries implementing more regulations we cannot be sure how
small this will remain.

I would say your outlook is wishful thinking that is willing to ignore
about 10 percent of the users on the internet who don't have well
behaved Carriers or Governments.

So Dan you are doing a works for me arguement what is the most invalid
arguement to-do in many cases.   Its lets sweep a bug under a carpet
and not consider it.

The problem is the type of NAT used.
https://en.wikipedia.org/wiki/Network_address_translation#Symmetric_NAT

Symmetric NAT  this nicely randomises what address users behind it are
coming from.    Usage of Symmetric NAT does not have to have anything
to-do with reducing the number of IP addresses in usage.   Symmetric
NAT can have equal number of users to internet address.

Symmetric NAT is the brick wall from hell to hole punching.    The
main objective of a Symmetric NAT is that something in the internet
that has not had a packet from something behind the Symmetric NAT
blocked by default.   Add in symmetric NAT randomising IP to IP
mapping.     So after IPv4 disappears what Symmetric NAT still has a
usage in IPv6.

Teredo that is IPv6 over IPv4 fails if both ends are behind Symmetric
NAT.    Normal STUN for NAT punching falls over if both ends are
behind Symmetric NAT this does not matter if it IPv4 or iPv6.
Symmetric NAT randomising ip to ip mapping bring hell.   So you opened
up a connection after so much time the Symmetric NAT forgets and you
attempt to send another packet to a end and it picks out a new IP
address at random to use.

The three types cone style NAT will stay in usage by client routers by
different Carriers even after IPv6 is dominate everywhere as it make
sense at that point so being able to punch though those at times will
still be required.

So the 4 types of common NAT are not going anywhere were they were
used for common sense reasons.   The Carrier NAT attempting to push
massive numbers of users though limited addresses will hopefully
disappear due to IPv6.

 Basically Dan is about time you step back look at NAT how it used and
where is used and why.    The change to IPv6 is only really getting
rid of one form of NAT  being the carrier nat that were non Symmetric
NAT based on Symmetric NAT ideas that at times had massive problems
like completely running out of ports due to not enough address because
attempting to push too many clients out too few of addresses.

 If somewhere has a pure Symmetric NAT where the number of external
address match the number of internal address for IPv4 for security
reasons doing the same thing on IPv6 has the same logical reasons.
So logically sane placed Symmetric NAT will remain and when you have
to get though them the same problems will remain.

The reality is the 4 common types of NAT can be deployed sanely
without massive over-stacking.   Under IPv6 the worse we should see is
hopefully only 2 NAT deep.     Possible 3 mode cone NAT in router and
Carrier with a Symmetric NAT between you and the internet. as long as
what ever is design can get though this worse case all cases will be
covered.   .  Why hopefully only 2 nat deep It is possible to have
like ADSL router NAT + a WIFI router doing NAT and Carrier doing
Symmetric NAT but the wifi NAT level is kinda self inflicted by user
on self not forced on user by carrier this is an improvement over the
3 to 4 deep in nat by carrier in some places..

Dan attempting to code when the required interfaces to make it work
don't exist and have not been debated does not make much sense.   Also
attempting to tell boss time this will need roughly to give something
functional also not be guessed when you are in the location that there
is a framework problem.

IPv6 is improving something things.   But IPv6 is not a magic bullet
to cure all the issues of having at times to get through NAT.

Basically it is BS the that existence of NAT can be ignored because
IPv6 fixes everything.   IPv6 pure once fully deployed by all carriers
should reduce the number of NAT you have to cross on the internet.
The big elephant in the room is that IPv6 is only going to reduce the
number of NAT in the internet not remove them all.

So working out how to handle the case that end user has found
themselves on the wrong side of deployed NATs applies to IPv4 and IPv6
with IPv6 hopefully being less glitch due to lower numbers of NAT in
the mix.

 Think if a company can use an accounts that is behind a Symmetric NAT
without having to pay extra or do extra government regulation  for a
static public internet IP address might reduce their costs of doing
business.    Also consider the ones most commonly going to be stuck
behind Symmetric NAT are also the places that will have massive
amounts of government regulation that can forbid having private keys
on overseas servers and and using overseas vpn servers.   Using a
Relay/TURN server is hair splitting as it not a overseas VPN server
its only an overseas relay at worst.

Reality Dan look out side your small conner of the earth.   New
standards does not change how messed up world internet regulations by
different governments are or different carriers stunts to make more
money.

.
Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Dan Lüdtke]

I don't see a bug here. And no patches. And still no code. Only plenty of tl=
;dr. I think the only thing we can do is to agree to disagree.

> On 18 Jan 2017, at 12:21, Peter Dolding <oiaohm@gmail.com> wrote:
>=20
>> On Wed, Jan 18, 2017 at 4:11 PM, Dan L=C3=BCdtke <mail@danrl.com> wrote:
>> Two things I have not seen so far:
>> - government regulations that enforce NAT
>> - ISPs (let alone carriers) "upgrading" their networks to ipv6 nat (i mys=
elf have run both, isp + carrier networks, and i call BS on your future outl=
ook regarding nat ipv6)
>> - code from you in this thread
>>=20
> https://en.wikipedia.org/wiki/Internet_censorship
> When you start looking into countries that are red in the "World map
> showing the status of YouTube blocking"  you will find some of those
> its mandatory to have a NAT between ISP and open internet even for
> IPv6.  Yes the area of infect users is currently small.  But when you
> look a countries implementing more regulations we cannot be sure how
> small this will remain.
>=20
> I would say your outlook is wishful thinking that is willing to ignore
> about 10 percent of the users on the internet who don't have well
> behaved Carriers or Governments.
>=20
> So Dan you are doing a works for me arguement what is the most invalid
> arguement to-do in many cases.   Its lets sweep a bug under a carpet
> and not consider it.
>=20
> The problem is the type of NAT used.
> https://en.wikipedia.org/wiki/Network_address_translation#Symmetric_NAT
>=20
> Symmetric NAT  this nicely randomises what address users behind it are
> coming from.    Usage of Symmetric NAT does not have to have anything
> to-do with reducing the number of IP addresses in usage.   Symmetric
> NAT can have equal number of users to internet address.
>=20
> Symmetric NAT is the brick wall from hell to hole punching.    The
> main objective of a Symmetric NAT is that something in the internet
> that has not had a packet from something behind the Symmetric NAT
> blocked by default.   Add in symmetric NAT randomising IP to IP
> mapping.     So after IPv4 disappears what Symmetric NAT still has a
> usage in IPv6.
>=20
> Teredo that is IPv6 over IPv4 fails if both ends are behind Symmetric
> NAT.    Normal STUN for NAT punching falls over if both ends are
> behind Symmetric NAT this does not matter if it IPv4 or iPv6.
> Symmetric NAT randomising ip to ip mapping bring hell.   So you opened
> up a connection after so much time the Symmetric NAT forgets and you
> attempt to send another packet to a end and it picks out a new IP
> address at random to use.
>=20
> The three types cone style NAT will stay in usage by client routers by
> different Carriers even after IPv6 is dominate everywhere as it make
> sense at that point so being able to punch though those at times will
> still be required.
>=20
> So the 4 types of common NAT are not going anywhere were they were
> used for common sense reasons.   The Carrier NAT attempting to push
> massive numbers of users though limited addresses will hopefully
> disappear due to IPv6.
>=20
> Basically Dan is about time you step back look at NAT how it used and
> where is used and why.    The change to IPv6 is only really getting
> rid of one form of NAT  being the carrier nat that were non Symmetric
> NAT based on Symmetric NAT ideas that at times had massive problems
> like completely running out of ports due to not enough address because
> attempting to push too many clients out too few of addresses.
>=20
> If somewhere has a pure Symmetric NAT where the number of external
> address match the number of internal address for IPv4 for security
> reasons doing the same thing on IPv6 has the same logical reasons.
> So logically sane placed Symmetric NAT will remain and when you have
> to get though them the same problems will remain.
>=20
> The reality is the 4 common types of NAT can be deployed sanely
> without massive over-stacking.   Under IPv6 the worse we should see is
> hopefully only 2 NAT deep.     Possible 3 mode cone NAT in router and
> Carrier with a Symmetric NAT between you and the internet. as long as
> what ever is design can get though this worse case all cases will be
> covered.   .  Why hopefully only 2 nat deep It is possible to have
> like ADSL router NAT + a WIFI router doing NAT and Carrier doing
> Symmetric NAT but the wifi NAT level is kinda self inflicted by user
> on self not forced on user by carrier this is an improvement over the
> 3 to 4 deep in nat by carrier in some places..
>=20
> Dan attempting to code when the required interfaces to make it work
> don't exist and have not been debated does not make much sense.   Also
> attempting to tell boss time this will need roughly to give something
> functional also not be guessed when you are in the location that there
> is a framework problem.
>=20
> IPv6 is improving something things.   But IPv6 is not a magic bullet
> to cure all the issues of having at times to get through NAT.
>=20
> Basically it is BS the that existence of NAT can be ignored because
> IPv6 fixes everything.   IPv6 pure once fully deployed by all carriers
> should reduce the number of NAT you have to cross on the internet.
> The big elephant in the room is that IPv6 is only going to reduce the
> number of NAT in the internet not remove them all.
>=20
> So working out how to handle the case that end user has found
> themselves on the wrong side of deployed NATs applies to IPv4 and IPv6
> with IPv6 hopefully being less glitch due to lower numbers of NAT in
> the mix.
>=20
> Think if a company can use an accounts that is behind a Symmetric NAT
> without having to pay extra or do extra government regulation  for a
> static public internet IP address might reduce their costs of doing
> business.    Also consider the ones most commonly going to be stuck
> behind Symmetric NAT are also the places that will have massive
> amounts of government regulation that can forbid having private keys
> on overseas servers and and using overseas vpn servers.   Using a
> Relay/TURN server is hair splitting as it not a overseas VPN server
> its only an overseas relay at worst.
>=20
> Reality Dan look out side your small conner of the earth.   New
> standards does not change how messed up world internet regulations by
> different governments are or different carriers stunts to make more
> money.
>=20
> .
> Peter Dolding

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Peter Dolding]

On Wed, Jan 18, 2017 at 10:07 PM, Dan L=C3=BCdtke <mail@danrl.com> wrote:
> I don't see a bug here. And no patches. And still no code. Only plenty of=
 tl;dr. I think the only thing we can do is to agree to disagree.

Go to Australia try internode notice you have IPv6 options  Dual Stack
or Dynamic IPv6.   If you choose Dynamic IPv6 say hello Symmetric NAT
.

Now IPv6 offered in Symmetric NAT is here to stay.   Now Australia is
not a area who government regulation forcing it.   Dynamic IPv6 is
sold a security feature for users so that web sites don't see them
coming from the same IP addresses all the time.

Current wireguard cannot punch though a Symmetric NAT be it IPv6 or
IPv4.   Yes in Australia and other countries IPv6 where you will not
be able to connect using basic NAT hole punching due to Symmetric NAT
IPv6.

Really dual stack Ipv6 is not all that is out there.

Question is how to handle it the solution to a Symmetric NAT at both
ends is a Relay of some form.   This does not matter if it IPv6 or
IPv4.

Dynamic IP that is not Symmetric Nat be it IPv6 or IPv4 will have the
same problem of client becoming disconnected from server and needing
to rerun resolve step.

So like it or no Dan there is a bug because the world of Networking
pain points don't change one bit what they are with the introduction
of IPv6 just become less common.   So every problem you have to battle
past with Ipv4 you have to battle past with Ipv6 just less often.

Write code is not something to straight up do.   I am still looking
around at ways to do it.  File socket is one option, memfd is another
is like ebpf using a mmap ring buffer.

Really when wireguard objective is to be fast.   It makes limit sense
to come back to user space just to put packet headers on so the packet
can go to a relay.   Yes one option is also user defined template like
ebpf loaded up by usermode program that wireguard uses and it use a
ring buffer to call back to usermode program if connection appears to
disappear.    Now a special connection for relay or using dynamic dns
because you have dynamic IPv6 of one form or another.

Its not that I have not read carefully before commenting either.   Go
and Rust are market as the usermode programming languages.   Neither
of those languages I use commonly.


Yes Dan you response that hey this is just going to be over head
because with the introduction of IPv6 we don't have to deal with the
old problems is just so far wrong its not funny.

Please not respond with tl;dr all that said is you did not bother
reading.   The issue is you have not read up fully on what IPv6 has
fixed and not fixed and will never fix.   Places were using dynamic
IPv4 when there were tones of IPv4 left for security reasons.
Dyanmic IPv6 is coming back for exactly the same security reasons.
Issue here most don't know the history that dynamic IPv4 and Symmetric
Nats were in use before IPv4 address limits were a problem.   So IPv6
adding more addresses was not going to change everything it will just
restore the networking world back to the way it was before we got
short of IP addresses that world had 4 forms of NAT and dynamic IP
addresses..

Peter Dolding


>
>> On 18 Jan 2017, at 12:21, Peter Dolding <oiaohm@gmail.com> wrote:
>>
>>> On Wed, Jan 18, 2017 at 4:11 PM, Dan L=C3=BCdtke <mail@danrl.com> wrote=
:
>>> Two things I have not seen so far:
>>> - government regulations that enforce NAT
>>> - ISPs (let alone carriers) "upgrading" their networks to ipv6 nat (i m=
yself have run both, isp + carrier networks, and i call BS on your future o=
utlook regarding nat ipv6)
>>> - code from you in this thread
>>>
>> https://en.wikipedia.org/wiki/Internet_censorship
>> When you start looking into countries that are red in the "World map
>> showing the status of YouTube blocking"  you will find some of those
>> its mandatory to have a NAT between ISP and open internet even for
>> IPv6.  Yes the area of infect users is currently small.  But when you
>> look a countries implementing more regulations we cannot be sure how
>> small this will remain.
>>
>> I would say your outlook is wishful thinking that is willing to ignore
>> about 10 percent of the users on the internet who don't have well
>> behaved Carriers or Governments.
>>
>> So Dan you are doing a works for me arguement what is the most invalid
>> arguement to-do in many cases.   Its lets sweep a bug under a carpet
>> and not consider it.
>>
>> The problem is the type of NAT used.
>> https://en.wikipedia.org/wiki/Network_address_translation#Symmetric_NAT
>>
>> Symmetric NAT  this nicely randomises what address users behind it are
>> coming from.    Usage of Symmetric NAT does not have to have anything
>> to-do with reducing the number of IP addresses in usage.   Symmetric
>> NAT can have equal number of users to internet address.
>>
>> Symmetric NAT is the brick wall from hell to hole punching.    The
>> main objective of a Symmetric NAT is that something in the internet
>> that has not had a packet from something behind the Symmetric NAT
>> blocked by default.   Add in symmetric NAT randomising IP to IP
>> mapping.     So after IPv4 disappears what Symmetric NAT still has a
>> usage in IPv6.
>>
>> Teredo that is IPv6 over IPv4 fails if both ends are behind Symmetric
>> NAT.    Normal STUN for NAT punching falls over if both ends are
>> behind Symmetric NAT this does not matter if it IPv4 or iPv6.
>> Symmetric NAT randomising ip to ip mapping bring hell.   So you opened
>> up a connection after so much time the Symmetric NAT forgets and you
>> attempt to send another packet to a end and it picks out a new IP
>> address at random to use.
>>
>> The three types cone style NAT will stay in usage by client routers by
>> different Carriers even after IPv6 is dominate everywhere as it make
>> sense at that point so being able to punch though those at times will
>> still be required.
>>
>> So the 4 types of common NAT are not going anywhere were they were
>> used for common sense reasons.   The Carrier NAT attempting to push
>> massive numbers of users though limited addresses will hopefully
>> disappear due to IPv6.
>>
>> Basically Dan is about time you step back look at NAT how it used and
>> where is used and why.    The change to IPv6 is only really getting
>> rid of one form of NAT  being the carrier nat that were non Symmetric
>> NAT based on Symmetric NAT ideas that at times had massive problems
>> like completely running out of ports due to not enough address because
>> attempting to push too many clients out too few of addresses.
>>
>> If somewhere has a pure Symmetric NAT where the number of external
>> address match the number of internal address for IPv4 for security
>> reasons doing the same thing on IPv6 has the same logical reasons.
>> So logically sane placed Symmetric NAT will remain and when you have
>> to get though them the same problems will remain.
>>
>> The reality is the 4 common types of NAT can be deployed sanely
>> without massive over-stacking.   Under IPv6 the worse we should see is
>> hopefully only 2 NAT deep.     Possible 3 mode cone NAT in router and
>> Carrier with a Symmetric NAT between you and the internet. as long as
>> what ever is design can get though this worse case all cases will be
>> covered.   .  Why hopefully only 2 nat deep It is possible to have
>> like ADSL router NAT + a WIFI router doing NAT and Carrier doing
>> Symmetric NAT but the wifi NAT level is kinda self inflicted by user
>> on self not forced on user by carrier this is an improvement over the
>> 3 to 4 deep in nat by carrier in some places..
>>
>> Dan attempting to code when the required interfaces to make it work
>> don't exist and have not been debated does not make much sense.   Also
>> attempting to tell boss time this will need roughly to give something
>> functional also not be guessed when you are in the location that there
>> is a framework problem.
>>
>> IPv6 is improving something things.   But IPv6 is not a magic bullet
>> to cure all the issues of having at times to get through NAT.
>>
>> Basically it is BS the that existence of NAT can be ignored because
>> IPv6 fixes everything.   IPv6 pure once fully deployed by all carriers
>> should reduce the number of NAT you have to cross on the internet.
>> The big elephant in the room is that IPv6 is only going to reduce the
>> number of NAT in the internet not remove them all.
>>
>> So working out how to handle the case that end user has found
>> themselves on the wrong side of deployed NATs applies to IPv4 and IPv6
>> with IPv6 hopefully being less glitch due to lower numbers of NAT in
>> the mix.
>>
>> Think if a company can use an accounts that is behind a Symmetric NAT
>> without having to pay extra or do extra government regulation  for a
>> static public internet IP address might reduce their costs of doing
>> business.    Also consider the ones most commonly going to be stuck
>> behind Symmetric NAT are also the places that will have massive
>> amounts of government regulation that can forbid having private keys
>> on overseas servers and and using overseas vpn servers.   Using a
>> Relay/TURN server is hair splitting as it not a overseas VPN server
>> its only an overseas relay at worst.
>>
>> Reality Dan look out side your small conner of the earth.   New
>> standards does not change how messed up world internet regulations by
>> different governments are or different carriers stunts to make more
>> money.
>>
>> .
>> Peter Dolding
>

Re: Built-in Roaming is limited due to a design fault adding STUN and TURN support would be good and make wire-guard connections more durable.

[Jason A. Donenfeld]

Hello,

You raise interesting points. Symmetric NAT hole punching requires
lots of notification hooks to implement. Seeing as these types of
hooks and expanded userspace API are already something useful for
other things people have been talking about, I expect to expand the
userspace API. I imagine on top of those expansions, you'll be able to
implement performant STUN punching or TURN relaying. I thought I had
made these points clear much earlier, and asked whether you'd be
willing to implement some code.

In short, if you're interesting in writing code, you'll have what you
want, and you'll have a partner in me writing APIs for the kernel side
of things.

However, if it's going to be more extremely long rambling emails,
which are borderline offensive at times, that's not really going to
work.

It's now gotten to the point where people have been writing to me,
"hey you have a great mailing list and project, and now this guy is
really burdensome to read, what's the deal?" And indeed, a simple
google search reveals a history of odd trolling.

Therefore, I ask you to keep subsequent replies on this thread short
and efficient. If you'd like to write some interesting network code,
let's do it. Otherwise, at the very least I certainly do thank you for
your suggestion. You've made more clear in my mind exactly what the
challenges of symmetric NAT are. I'm quite confident we'll get to the
point where these network concerns can be addressed, and where
wireguard has the right notification mechanisms to enable this to
happen.

Regards,
Jason