[WireGuard] OpenWRT package for wireguard

[WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1.1: Type: text/plain, Size: 306 bytes --]

Hi there,

I am working on an OpenWRT/LEDE package for wireguard.  It's mostly done,
but I want to test it before pushing it to the repositories.  Hopefully, I
will be able to do that this week-end.

In the meantime, if somebody wants to test the package, the Makefile is
attached to this email.

Baptiste

[-- Attachment #1.2: Makefile --]
[-- Type: text/plain, Size: 2992 bytes --]

# 
# Copyright (C) 2016 Baptiste Jonglez <openwrt@bitsofnetworks.org>
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.

include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=wireguard

PKG_RELEASE:=1
# Temporary
PKG_VERSION:=0.0.0
PKG_RELEASE=$(PKG_SOURCE_VERSION)
PKG_LICENSE:=GPL-2.0
PKG_LICENSE_FILES:=COPYING
PKG_USE_MIPS16:=0

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://git.zx2c4.com/WireGuard
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE_VERSION:=aa8fcc678092aaab4a8f986e45b696d72d2029ce
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION)-$(PKG_SOURCE_VERSION).tar.gz

PKG_BUILD_PARALLEL:=1

# Wireguard's makefile needs this to know where to build the kernel module
export KERNELDIR:=$(LINUX_DIR)

include $(INCLUDE_DIR)/package.mk

define Package/wireguard
  SECTION:=net
  CATEGORY:=Network
  URL:=https://www.wireguard.io
  MAINTAINER:=Baptiste Jonglez <openwrt@bitsofnetworks.org>
  TITLE:=Wireguard userspace control program
  DEPENDS:=+libmnl +kmod-wireguard
endef

define Package/wireguard/description
  WireGuard is a novel VPN that runs inside the Linux Kernel and utilizes
  state-of-the-art cryptography. It aims to be faster, simpler, leaner, and
  more useful than IPSec, while avoiding the massive headache. It intends to
  be considerably more performant than OpenVPN.  WireGuard is designed as a
  general purpose VPN for running on embedded interfaces and super computers
  alike, fit for many different circumstances.
  It runs over UDP.

  This package provides the userspace control program for wireguard, wg.
endef

define KernelPackage/wireguard
  SECTION:=kernel
  CATEGORY:=Kernel modules
  SUBMENU:=Network Support
  TITLE:=Wireguard kernel module
  KCONFIG:=CONFIG_NET_UDP_TUNNEL CONFIG_IPV6 CONFIG_NETFILTER_XT_MATCH_HASHLIMIT
  DEPENDS:=@IPV6 +kmod-udptunnel4 +kmod-udptunnel6 +kmod-ipt-hashlimit
  FILES:= $(PKG_BUILD_DIR)/wireguard.$(LINUX_KMOD_SUFFIX)
  AUTOLOAD:=$(call AutoLoad,33,wireguard)
endef

define KernelPackage/wireguard/description
  WireGuard is a novel VPN that runs inside the Linux Kernel and utilizes
  state-of-the-art cryptography. It aims to be faster, simpler, leaner, and
  more useful than IPSec, while avoiding the massive headache. It intends to
  be considerably more performant than OpenVPN.  WireGuard is designed as a
  general purpose VPN for running on embedded interfaces and super computers
  alike, fit for many different circumstances.
  It runs over UDP.

  This package provides the kernel module for wireguard.
endef

define Build/Prepare
	$(call Build/Prepare/Default)
	mkdir -p $(PKG_BUILD_DIR)
	$(CP) ./src/* $(PKG_BUILD_DIR)/
endef

define Package/wireguard/install
	$(INSTALL_DIR) $(1)/usr/bin/
	$(INSTALL_BIN) $(PKG_BUILD_DIR)/tools/wg $(1)/usr/bin/
endef

$(eval $(call BuildPackage,wireguard))
$(eval $(call KernelPackage,wireguard))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

On Wed, Jun 29, 2016 at 01:24:23PM +0200, Baptiste Jonglez wrote:
> Hi there,
> 
> I am working on an OpenWRT/LEDE package for wireguard.  It's mostly done,
> but I want to test it before pushing it to the repositories.  Hopefully, I
> will be able to do that this week-end.

Here is a working version:

  https://github.com/zorun/packages-1/blob/wireguard/net/wireguard/Makefile

However, you have to manually select needed kernel symbols in `make
kernel_menuconfig`, which is very annoying.  If you want to give it a try,
look for these symbols:

    +CONFIG_IP6_NF_IPTABLES=m
    +CONFIG_IPV6=y
    +CONFIG_NETFILTER=y
    +CONFIG_NETFILTER_ADVANCED=y
    +CONFIG_NETFILTER_XTABLES=m
    +CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
    +CONFIG_NET_FOU=m
    +CONFIG_NET_IP_TUNNEL=m
    +CONFIG_NET_UDP_TUNNEL=m

I guess that's the price to pay for being out-of-tree.  I will have a look
at how to improve this situation (maybe integrating the kernel module
directly in Lede/OpenWRT?).

Baptiste

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[WireGuard] Kernel symbol dependencies and KCONFIG usage for kernel packages

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

Hi,

I am packaging a new kernel module (wireguard), it is currently at [1].

However, I have some trouble understanding what KCONFIG does.  Wireguard
needs a few kernel options, so I thought that KCONFIG was the place to
declare such dependencies:

  https://github.com/zorun/packages-1/blob/wireguard/net/wireguard/Makefile#L90

However, it does not seem to enable these options.  The module compiles
fine, it installs fine [2], but then it fails to load at runtime:

Sat Jul  2 10:04:50 2016 kern.warn kernel: [   95.015838] wireguard: Unknown symbol udp_sock_create4 (err 0)
Sat Jul  2 10:04:50 2016 kern.warn kernel: [   95.022021] wireguard: Unknown symbol udp_tunnel6_xmit_skb (err 0)
Sat Jul  2 10:04:50 2016 kern.warn kernel: [   95.028767] wireguard: Unknown symbol udp_tunnel_sock_release (err 0)
Sat Jul  2 10:04:50 2016 kern.warn kernel: [   95.035570] wireguard: Unknown symbol setup_udp_tunnel_sock (err 0)
Sat Jul  2 10:04:50 2016 kern.warn kernel: [   95.042136] wireguard: Unknown symbol udp_sock_create6 (err 0)
Sat Jul  2 10:04:50 2016 kern.warn kernel: [   95.048297] wireguard: Unknown symbol udp_tunnel_xmit_skb (err 0)

It works when selecting the needed options manually in kernel_menuconfig,
but it's a huge pain.

Is there a way to get the needed kernel options automatically?  Maybe this
only happens for packages that are part of the core Lede distribution?

Thanks,
Baptiste

[1] https://github.com/zorun/packages-1/blob/wireguard/net/wireguard/Makefile
[2] http://paste.aliens-lyon.fr/CBI

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

On Jul 2, 2016 12:28 PM, "Baptiste Jonglez" <baptiste@bitsofnetworks.org>
wrote:
> However, you have to manually select needed kernel symbols in `make
> kernel_menuconfig`, which is very annoying.  If you want to give it a try,
> look for these symbols:
>
>     +CONFIG_IP6_NF_IPTABLES=m
>     +CONFIG_IPV6=y
>     +CONFIG_NETFILTER=y
>     +CONFIG_NETFILTER_ADVANCED=y
>     +CONFIG_NETFILTER_XTABLES=m
>     +CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
>     +CONFIG_NET_FOU=m
>     +CONFIG_NET_IP_TUNNEL=m
>     +CONFIG_NET_UDP_TUNNEL=m

On SMP systems, wireguard will benefit from CONFIG_PADATA which is
selectable by the parallel crypto engine option in the crypto API menu of
menuconfig.

[-- Attachment #2: Type: text/html, Size: 917 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

On Wed, Jun 29, 2016 at 01:24:23PM +0200, Baptiste Jonglez wrote:
> Hi there,
> 
> I am working on an OpenWRT/LEDE package for wireguard.  It's mostly done,
> but I want to test it before pushing it to the repositories.  Hopefully, I
> will be able to do that this week-end.

Here is the package finally:

  https://github.com/openwrt/packages/blob/master/net/wireguard/Makefile

The built packages should be available in the snapshots quite soon, e.g.:

- wireguard-tools: https://downloads.lede-project.org/snapshots/packages/mips_34kc/packages/
- kmod-wireguard: https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/packages/

If you are running a recent snapshot version of Lede, this should be
enough if you have a compatible kernel:

  opkg install wireguard-tools kmod-wireguard

If you want build it yourself, you need this patch for proper dependency
handling, it has been included in Lede two days ago:

  https://git.lede-project.org/?p=source.git;a=commit;h=1d15a96b29dcd0947690951a7c36aead79a27129

One final note: there is no "multi-threaded" support for now, sorry Jason.
So, if you run this on your super modern 128-cores router, don't expect
exceptional performance :)  Well, you could always hack your kernel config
to enable CONFIG_PADATA and CONFIG_SMP before building wireguard.

Other than that, feedback on the packages is welcome!

Baptiste

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

Hey Baptiste,

Wonderful news! Thanks for your hard work on the OpenWRT integration.
Looking forward to seeing how users like it.

Jason

Re: [WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 3220 bytes --]

On Mon, Jul 04, 2016 at 09:48:38AM +0200, Baptiste Jonglez wrote:
> Here is the package finally:
> 
>   https://github.com/openwrt/packages/blob/master/net/wireguard/Makefile
> 
> The built packages should be available in the snapshots quite soon, e.g.:
> 
> - wireguard-tools: https://downloads.lede-project.org/snapshots/packages/mips_34kc/packages/
> - kmod-wireguard: https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/packages/

For reference, here is the size of wireguard and its dependencies for
ar71xx, taken from the snapshots above:

 2.4 KB  kmod-udptunnel6_4.4.14-1_mips_34kc.ipk
 2.7 KB  kmod-udptunnel4_4.4.14-1_mips_34kc.ipk
 6.2 KB  kmod-ipt-hashlimit_4.4.14-1_mips_34kc.ipk
 6.2 KB  libmnl_1.0.3-2_mips_34kc.ipk
  19 KB  wireguard-tools_0.0.20160630-1_mips_34kc.ipk
  59 KB  kmod-wireguard_4.4.14+0.0.20160630-1_mips_34kc.ipk
  95 KB  total

That's really not bad!  It largely fits in any small router with 4 MB of
flash.

As a comparison, here is the size for fastd:

  12 KB  kmod-tun_4.4.14-1_mips_34kc.ipk
  15 KB  libjson-c_0.12-1_mips_34kc.ipk
  53 KB  fastd_18-1_mips_34kc.ipk
  79 KB  total

fastd, which is "optimized on small code size and few dependencies", is
indeed a bit smaller.  Also note that libjson-c is already there by
default on OpenWRT/LEDE, so the real additional size of fastd is only 65 KB.

Openvpn with polarssl takes between 3 and 4 times as much space as wireguard:

  12 KB  kmod-tun_4.4.14-1_mips_34kc.ipk
  33 KB  liblzo_2.09-1_mips_34kc.ipk
 141 KB  libpolarssl_1.3.16-2_mips_34kc.ipk
 138 KB  openvpn-polarssl_2.3.11-2_mips_34kc.ipk
 323 KB  total

Of course, the openssl variant for openvpn is really much bigger:

  12 KB  kmod-tun_4.4.14-1_mips_34kc.ipk
  33 KB  liblzo_2.09-1_mips_34kc.ipk
 719 KB  libopenssl_1.0.2h-1_mips_34kc.ipk
 141 KB  openvpn-openssl_2.3.11-2_mips_34kc.ipk
 904 KB  total

Here is the size for tinc (only available with openssl):

  12 KB  kmod-tun_4.4.14-1_mips_34kc.ipk
  33 KB  liblzo_2.09-1_mips_34kc.ipk
 719 KB  libopenssl_1.0.2h-1_mips_34kc.ipk
  60 KB  tinc_1.0.28-1_mips_34kc.ipk
 823 KB  total


So, now, the remaining question is: what is the performance of wireguard
on such small devices?

> If you are running a recent snapshot version of Lede, this should be
> enough if you have a compatible kernel:
> 
>   opkg install wireguard-tools kmod-wireguard
> 
> If you want build it yourself, you need this patch for proper dependency
> handling, it has been included in Lede two days ago:
> 
>   https://git.lede-project.org/?p=source.git;a=commit;h=1d15a96b29dcd0947690951a7c36aead79a27129
> 
> One final note: there is no "multi-threaded" support for now, sorry Jason.
> So, if you run this on your super modern 128-cores router, don't expect
> exceptional performance :)  Well, you could always hack your kernel config
> to enable CONFIG_PADATA and CONFIG_SMP before building wireguard.
> 
> Other than that, feedback on the packages is welcome!
> 
> Baptiste



> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

Hi Baptiste,

Cool! Looks very promising. Looking forward to seeing the performance
measures, and then optimizing WireGuard to improve whatever the
results happen to be.

One aspect that contributes to the size of the kernel module is that
it can't be stripped in the same way a userspace binary can, because
it has to be dynamically linked into the kernel.

Jason

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

On Wed, Jul 6, 2016 at 11:53 AM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> One aspect that contributes to the size of the kernel module is that
> it can't be stripped in the same way a userspace binary can, because
> it has to be dynamically linked into the kernel.

*ALTHOUGH* looking at the binary in the ipk closely, it looks like a
LOT of those symbols actually could be stripped, since they're not
exported and aren't external...

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

I don't have a MIPS toolchain on this computer to test this at the
moment, but maybe this will shave off a little bit from the .ko...

strip $(nm --defined-only wireguard.ko | grep -Ev '(^0+ | __)' | cut
-d ' ' -f 3 | xargs -n 1 echo -N) wireguard.ko

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

There's also a lot to be done with -flto. I was working on getting
this working last year but ran into some snags. I'll give it another
try.

There's also the more bootleg version of running with -fwhole-program
and then using the __visible attribute on the module init/cleanup
functions. I don't know if this would actually work properly though.

Re: [WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

On Wed, Jul 06, 2016 at 04:54:46PM +0200, Jason A. Donenfeld wrote:
> I don't have a MIPS toolchain on this computer to test this at the
> moment, but maybe this will shave off a little bit from the .ko...
> 
> strip $(nm --defined-only wireguard.ko | grep -Ev '(^0+ | __)' | cut
> -d ' ' -f 3 | xargs -n 1 echo -N) wireguard.ko

Doesn't seem very effective: http://paste.aliens-lyon.fr/HDg

The size of the module has not changed.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

Ahh, position independence, right. Okay, I'll look into getting LTO
working, which will certainly have good results.

Re: [WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

On Sun, Jul 10, 2016 at 07:52:51PM +0200, Jason A. Donenfeld wrote:
> Ahh, position independence, right. Okay, I'll look into getting LTO
> working, which will certainly have good results.

Note that at this point, reducing the size is not really necessary, it
would just be nice.  But if LTO reduces the code size *and* improves
performance, that's the way to go!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 890 bytes --]

On Wed, Jul 06, 2016 at 11:56:28AM +0200, Jason A. Donenfeld wrote:
> On Wed, Jul 6, 2016 at 11:53 AM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> > One aspect that contributes to the size of the kernel module is that
> > it can't be stripped in the same way a userspace binary can, because
> > it has to be dynamically linked into the kernel.
> 
> *ALTHOUGH* looking at the binary in the ipk closely, it looks like a
> LOT of those symbols actually could be stripped, since they're not
> exported and aren't external...

Interesting.  I found this script in LEDE:

  https://git.lede-project.org/?p=source.git;a=blob;f=scripts/strip-kmod.sh;hb=HEAD

But it's indeed possible that it's not run for kernel modules that are
built outside of the core system.  I will take a closer look.

Can you check that, for instance, the kmod-udptunnel modules are properly
stripped?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] OpenWRT package for wireguard

[Jason A. Donenfeld]

On Wed, Jul 6, 2016 at 5:08 PM, Baptiste Jonglez
<baptiste@bitsofnetworks.org> wrote:
> Interesting.  I found this script in LEDE:
>
>   https://git.lede-project.org/?p=source.git;a=blob;f=scripts/strip-kmod.sh;hb=HEAD

Cool script. I'll play with it when I get home. It's possible that it
doesn't do enough, though. WireGuard is split into several files, so
there are some non-static function. But, those functions aren't in use
elsewhere in the kernel -- they shouldn't be exported. So it's a
matter of tuning things so that these symbols are removed. Will get
back to you on this...