Fragmentation on UDP layer possible?

Fragmentation on UDP layer possible?

[StarBrilliant]

Hello Wireguard developers and uesrs,

Thank you for having built up this software!

Recently I found some problems transmitting large UDP packets at
certain network environments.

My network architecture consists of 2 layers (Wireguard + VXLAN) to
enable mesh routing based on BGP and Babel routing protocols. To reach
a minimum MTU of 1280 bytes required by IPv6, the network must be able
to send and receive UDP datagrams of 1362 bytes. However some public
Wi-Fi does not allow me to send UDP packets of this big. No matter I
do fragmentation on my box with DF Flag on, or disable PMTUD with DF
flag off, the firewall simply rejects any fragmented IP packet.

I know Wireguard can already do IP layer fragmentation. (Just set
tunnel MTU >= 1441 then fragmentation will be turned on) But in this
network condition we might need to solve the problem on UDP layer.
OpenVPN supports UDP layer fragmentation through the "--fragment X"
option. By specifying this option, UDP packets will be no more than X
bytes. It adds an additional 4-byte overhead per fragment.

My question is, is UDP layer fragmentation technically possible for
Wireguard? If possible, will it introduce incompatibility to old
clients? How much overhead might it have?


And again, thank you!

Re: Fragmentation on UDP layer possible?

[Roman Mamedov]

On Mon, 13 Aug 2018 02:53:44 +1000
StarBrilliant <coder@poorlab.com> wrote:

> I know Wireguard can already do IP layer fragmentation. (Just set
> tunnel MTU >= 1441 then fragmentation will be turned on)

Is that really expected to work? I tried setting MTU 9000 on both ends of a WG
tunnel, but large packets still do not seem to come through properly. Did you
try using it like that in any kind of environment (aside from that one
restrictive network)?

In theory using MTU 9000 or such would help lower the huge overhead percentage
of running IP over VXLAN over IP over WG over IP. I was looking into that the
other day, but my idea was to fragment VXLAN packets across multiple WG ones,
which turned out to be impossible (VXLAN RFC forbids fragmentation).

-- 
With respect,
Roman

Re: Fragmentation on UDP layer possible?

[StarBrilliant]

On Mon, Aug 13, 2018 at 5:06 AM Roman Mamedov <rm@romanrm.net> wrote:
>
> On Mon, 13 Aug 2018 02:53:44 +1000
> StarBrilliant <coder@poorlab.com> wrote:
>
> > I know Wireguard can already do IP layer fragmentation. (Just set
> > tunnel MTU >= 1441 then fragmentation will be turned on)
>
> Is that really expected to work? I tried setting MTU 9000 on both ends of a WG
> tunnel, but large packets still do not seem to come through properly. Did you
> try using it like that in any kind of environment (aside from that one
> restrictive network)?
>
> In theory using MTU 9000 or such would help lower the huge overhead percentage
> of running IP over VXLAN over IP over WG over IP. I was looking into that the
> other day, but my idea was to fragment VXLAN packets across multiple WG ones,
> which turned out to be impossible (VXLAN RFC forbids fragmentation).

I have succeeded in setting a MTU of 1966 bytes inside VXLAN with a
non-restrictive Ethernet.
Due to a Linux bug, you need to do "sudo ethtool -K vxlan0 rx off tx
off", or all UDP packets will have wrong checksums and being dropped.

You might want to check my project on generating a Wireguard+VXLAN
with a tool https://github.com/m13253/VxWireguard-Generator
(Note that this is not production-ready. In other words, please back
up your main database often)

Re: Fragmentation on UDP layer possible?

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]

On 13-08-18, Roman Mamedov wrote:
> On Mon, 13 Aug 2018 02:53:44 +1000
> StarBrilliant <coder@poorlab.com> wrote:
> 
> > I know Wireguard can already do IP layer fragmentation. (Just set
> > tunnel MTU >= 1441 then fragmentation will be turned on)
> 
> Is that really expected to work? I tried setting MTU 9000 on both ends of a WG
> tunnel, but large packets still do not seem to come through properly. Did you
> try using it like that in any kind of environment (aside from that one
> restrictive network)?

Yes, it works: we use that to enforce a 1500 MTU on the wg interface, it
avoids a lot of headache.  Wireguard may end up sending UDP packets larger
than the MTU, which the kernel fragments at the IP layer.  The kernel of
the remote endpoint then reassembles these packets before giving them to
wireguard.

That being said, if you have a nasty firewall or middlebox in the (public)
path between your endpoints, it might indeed drop fragmented IP packets,
breaking this use-case.

Baptiste

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Fragmentation on UDP layer possible?

[Derek Fawcus]

UDP may (eventually) get a native segmentation capability,
but the work is still under discussion, and possibly subject
to change.

  See the FRAG option within draft-ietf-tsvwg-udp-options
   (https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options-05)

DF

Re: Fragmentation on UDP layer possible?

[Jason A. Donenfeld]

WireGuard's socket.c calls udp_tunnel_xmit with the DF bit set to 0,
which means if the underlying endpoint's path has an MTU that is too
small, the UDP packet will simply be fragmented, not dropped.

Re: Fragmentation on UDP layer possible?

[StarBrilliant]

On Mon, Aug 13, 2018 at 4:41 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> WireGuard's socket.c calls udp_tunnel_xmit with the DF bit set to 0,
> which means if the underlying endpoint's path has an MTU that is too
> small, the UDP packet will simply be fragmented, not dropped.

Hi Jason,

Thank you for your answer.

Yes. This is how Wireguard handles fragmenting on IP layer. And thanks
to this feature I can build my network with high MTU inside.

For Roman Mamedov's question, I said there is such bug if you use
VXLAN inside Wireguard: You need to turn checksum offload off if
fragment is on.

But my doubt is "can we do fragment on UDP layer", or "on application
layer". That is because I found some public Wi-Fi simply does not
allow me to transmit any fragmented IP packets. Would you please
comment on this question? (For details, you may see my first post.)

Thanks!