Sending just ssh traffic via wg

Sending just ssh traffic via wg

[Konstantin Ryabitsev]

Hi, all:

I'm trying to figure out the right PostUP/PostDown incantations to send
just ssh traffic (on port tcp/22) via the wg tunnel, but I'm having a
bit of a hard time. I should be able to do this with --set-mark for
iptables/PREROUTING and a fwmark ip route rule, but it doesn't appear to
be working as described in many guides I've consulted.

Has anyone done something like this before?

-K
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Jason A. Donenfeld]

Hey Konstantin,

When you're doing policy routing with packets that are being forwarded
by the system -- a router, for example -- then the prerouting table is
sufficient. But for locally generated packets, you have to use the
OUTPUT table and also probably MASQUERADE. I just reproduced
everything here and confirm this works:

ip route add default dev wg0 table 2468
ip rule add fwmark 1234 table 2468
wg set wg0 peer [...] allowed-ips 0.0.0.0/0
sysctl net.ipv4.conf.wg0.rp_filter=0
iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype
--src-type LOCAL -j MASQUERADE
iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234

That works pretty well for me. If you don't want to disable rp_filter,
you can do a little dance of setting and restoring the connmark in
egress and ingress so that incoming packets are matched against that
routing table too.

Alternatively, if your goal is actually to just send certain processes
through the tunnel, you have three more options:
- Network namespaces, and then `ip netns exec chicken ssh 1.2.3.4 ...`
- VRFs, and then `ip vrf exec chicken ssh 1.2.3.4 ...`
- Cgroups and net_cls.

All three work well and are differently convenient depending on your
needs. I wrote up the netns stuff on wireguard.com/netns/ but haven't
gotten around to documenting VRFs and cgroups with wireguard, but they
in fact should work the same as for every other situation that uses
those, so any old tutorial will do.

Jason
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Toke Høiland-Jørgensen]

"Jason A. Donenfeld" <Jason@zx2c4.com> writes:

> Hey Konstantin,
>
> When you're doing policy routing with packets that are being forwarded
> by the system -- a router, for example -- then the prerouting table is
> sufficient. But for locally generated packets, you have to use the
> OUTPUT table and also probably MASQUERADE. I just reproduced
> everything here and confirm this works:
>
> ip route add default dev wg0 table 2468
> ip rule add fwmark 1234 table 2468
> wg set wg0 peer [...] allowed-ips 0.0.0.0/0
> sysctl net.ipv4.conf.wg0.rp_filter=0
> iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype
> --src-type LOCAL -j MASQUERADE
> iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234

Any reason why you can't just do

ip rule add dport 22 lookup 2468

?

-Toke
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Jason A. Donenfeld]

[-- Attachment #1.1: Type: text/plain, Size: 1102 bytes --]

On Fri, Oct 5, 2018, 12:03 Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> "Jason A. Donenfeld" <Jason@zx2c4.com> writes:
>
> > Hey Konstantin,
> >
> > When you're doing policy routing with packets that are being forwarded
> > by the system -- a router, for example -- then the prerouting table is
> > sufficient. But for locally generated packets, you have to use the
> > OUTPUT table and also probably MASQUERADE. I just reproduced
> > everything here and confirm this works:
> >
> > ip route add default dev wg0 table 2468
> > ip rule add fwmark 1234 table 2468
> > wg set wg0 peer [...] allowed-ips 0.0.0.0/0
> > sysctl net.ipv4.conf.wg0.rp_filter=0
> > iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype
> > --src-type LOCAL -j MASQUERADE
> > iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234
>
> Any reason why you can't just do
>
> ip rule add dport 22 lookup 2468
>

That's indeed the best by far as long as other netfilter fanciness isn't
desired. Probably should set ipproto to tcp too in the rule.

Jason


> ?
>
> -Toke
>

[-- Attachment #1.2: Type: text/html, Size: 1903 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Konstantin Ryabitsev]

[-- Attachment #1.1: Type: text/plain, Size: 3311 bytes --]

On Fri, Oct 05, 2018 at 12:03:04PM +0200, Toke Høiland-Jørgensen wrote:
> > When you're doing policy routing with packets that are being forwarded
> > by the system -- a router, for example -- then the prerouting table is
> > sufficient. But for locally generated packets, you have to use the
> > OUTPUT table and also probably MASQUERADE. I just reproduced
> > everything here and confirm this works:
> >
> > ip route add default dev wg0 table 2468
> > ip rule add fwmark 1234 table 2468
> > wg set wg0 peer [...] allowed-ips 0.0.0.0/0
> > sysctl net.ipv4.conf.wg0.rp_filter=0
> > iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype
> > --src-type LOCAL -j MASQUERADE
> > iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234
> 
> Any reason why you can't just do
> 
> ip rule add dport 22 lookup 2468

Yeah, this works, too, and is quite a bit simpler. Jason, any reason why
I wouldn't use this?

To help explain the use-case here, here's what I'm trying to achieve.

We currently require that all admin-level access goes through 2-factor
authentication. For ssh, we require using SmartCard-capable hardware
tokens (Yubikey, Nitrokey), such that the private key for establishing a
connection is never exposed to the main OS -- so all our ssh connections
are inherently 2-factor authenticated.

However, that is not a mechanism we can use for accessing non-ssh things
such as various web-management interfaces on internal networks.
Therefore, we require that the VPN connection itself goes through
2-factor authentication step -- admins put in their password and their
TOTP/HOTP token value when establishing a connection. This works, but is
annoying to have to do when most of the time all an admin needs is ssh.
Every time there is a network blip, the admin loses their OpenVPN link
and, if they don't re-establish it quickly enough (typing in their
username, password, TOTP token value), then their ssh sessions reset.
Quite possibly the worst thing to happen to an admin in the middle of
troubleshooting something.

Similarly, if there's an alert in the middle of the night that requires
checking something out, it's annoying to have to first establish an
OpenVPN connection before being able to ssh in to a system.

So, we're working on a new setup where admins would have an always-on
WireGuard connection to the infra, but that connection only allows ssh
traffic. In this case, don't need 2-factor on the wireguard link, just
packet encapsulation. But should the admin need to bring up the OpenVPN
link for accessing something like an iDrac interface on a Dell, they
need to be able to do this without needing to shut down their WireGuard
tunnel first (since both WG and OpenVPN provide routing to the same
internal ip ranges). Therefore, I was looking for a way to *only* send
port 22 traffic on the wg link.

The following achieves what we need:

[Interface]
PrivateKey = [omitted]
Address = [omitted]
DNS = 127.0.0.1
Table = 2468
PostUp = ip rule add to 10.10.0.0/16 dport 22 lookup 2468
PostDown = ip rule del to 10.10.0.0/16 dport 22 lookup 2468

[Peer]
PublicKey = [omitted]
AllowedIPs = 10.10.0.0/16
Endpoint = [omitted]

This achieves what we need *quite* nicely!

Thanks for your help!

-K

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Matthias Urlichs]

On 05.10.18 17:53, Konstantin Ryabitsev wrote:
> But should the admin need to bring up the OpenVPN link

This may be a stupid question, but why do you need OpenVPN any more, if
you have Wireguard?

I'd set up a simple server-side login page that allows people to use
their user+pass+TOTP to enable non-SSH traffic on "their" link for the
next N minutes, with an easily-clickable Refresh button (and a
browser-based notification that the timeout is imminent), plus a small
(= easily-verified-to-be-correct) backend that enables/disables your
link's iptables rules. Problem solved.

-- 
-- Matthias Urlichs

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Konstantin Ryabitsev]

[-- Attachment #1.1: Type: text/plain, Size: 2593 bytes --]

On Fri, Oct 05, 2018 at 06:32:44PM +0200, Matthias Urlichs wrote:
>On 05.10.18 17:53, Konstantin Ryabitsev wrote:
>> But should the admin need to bring up the OpenVPN link
>
>This may be a stupid question, but why do you need OpenVPN any more, if
>you have Wireguard?

Because it's already there? :)

Furthermore, some members of our IT team use macs (gasp!) and for them 
it would be much easier to continue to use OpenVPN than to set up 
wireguard-go.

>I'd set up a simple server-side login page that allows people to use
>their user+pass+TOTP to enable non-SSH traffic on "their" link for the
>next N minutes, with an easily-clickable Refresh button (and a
>browser-based notification that the timeout is imminent), plus a small
>(= easily-verified-to-be-correct) backend that enables/disables your
>link's iptables rules. Problem solved.

Well, not quite. OpenVPN requires 2-factor validation each time there is 
a renegotiation -- so even if an attacker steals the private key and 
username/password of the administrator, they won't be able to establish 
a new session without having the TOTP secret as well. In the situation 
you describe, we create a window during which an attacker *would* be 
able to establish a VPN connection if they have the wireguard private 
key. We can make this window very short, but this would probably greatly 
annoy sysadmins, who will not enjoy needing to have a browser window 
open so they can re-validate. If we make this window sufficiently long 
not to annoy sysadmins, then this weakens our setup.

Furthermore, doing what you suggest would require writing a webapp and 
having some kind of queueing system for the backend process to read and 
adjust firewalls based on the data in the queue. It's a pretty 
complicated setup that seems orthogonal to the simplicity offered by 
WireGuard.

I've actually considered something like this -- to elevate their access 
privileges, an admin would need to simply ssh into the wireguard vpn 
server using their wireguard connection. A backend process could react 
to the ssh session being established and would apply additional firewall 
rules to give more access to that peer address. When that ssh session 
terminates, the same process would then drop these firewall rules back 
to their "ssh only" level. However, this still seems hacky and I would 
rather wait for Jason to come up with a More Proper way to do 2-factor, 
especially since the OpenVPN tunnel is already set up and functioning, 
so I don't have to invent anything to get what we need.

-K

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Jason A. Donenfeld]

Hey Konstantin,

On Fri, Oct 5, 2018 at 5:53 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
> > Any reason why you can't just do
> >
> > ip rule add dport 22 lookup 2468
>
> Yeah, this works, too, and is quite a bit simpler. Jason, any reason why
> I wouldn't use this?

Definitely use that. A reason for preferring netfilter for this would
be if you're doing lots of crazier netfilter stuff as well and want
complex rules. But for just tcp:22 matching, Toke's suggestion is by
far the best. I imagine internally, the kernel can just look into
`struct flowi` during the route lookups and doen't need to do much
subsequent parsing.

The one thing I'd change is you should add "ipproto tcp" to the
command so you don't match udp:22 as well.

> Every time there is a network blip, the admin loses their OpenVPN link
> and, if they don't re-establish it quickly enough (typing in their
> username, password, TOTP token value), then their ssh sessions reset.
> Quite possibly the worst thing to happen to an admin in the middle of
> troubleshooting something.
>
> Similarly, if there's an alert in the middle of the night that requires
> checking something out, it's annoying to have to first establish an
> OpenVPN connection before being able to ssh in to a system.
>
> So, we're working on a new setup where admins would have an always-on
> WireGuard connection to the infra, but that connection only allows ssh
> traffic. In this case, don't need 2-factor on the wireguard link, just
> packet encapsulation. But should the admin need to bring up the OpenVPN
> link for accessing something like an iDrac interface on a Dell, they
> need to be able to do this without needing to shut down their WireGuard
> tunnel first (since both WG and OpenVPN provide routing to the same
> internal ip ranges). Therefore, I was looking for a way to *only* send
> port 22 traffic on the wg link.

This seems like a reasonable and simple way of doing it. You could,
instead, make a little ssh wrapper that does the netns/vrf/cgroup
stuff if you wanted this at the process level, but probably the
heuristic of ssh==22 is a totally good and acceptable one that will be
less error prone.

By the way, hopefully as core development simmers down, I'll be able
to focus a bit more on infrastructure projects like adding 2FA on top
of wireguard.

> The following achieves what we need:
>
> [Interface]
> PrivateKey = [omitted]
> Address = [omitted]
> DNS = 127.0.0.1
> Table = 2468
> PostUp = ip rule add to 10.10.0.0/16 dport 22 lookup 2468
> PostDown = ip rule del to 10.10.0.0/16 dport 22 lookup 2468
>
> [Peer]
> PublicKey = [omitted]
> AllowedIPs = 10.10.0.0/16
> Endpoint = [omitted]
>
> This achieves what we need *quite* nicely!

I've add this example to the wg-quick(8) man page:
https://git.zx2c4.com/WireGuard/commit/?id=3e2f5495ea684d7f06fbefc50290e7d8985fc3de

Regards,
Jason
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Brian Candler]

[-- Attachment #1.1: Type: text/plain, Size: 909 bytes --]

On 06/10/2018 11:00, wireguard-request@lists.zx2c4.com wrote:
>> This may be a stupid question, but why do you need OpenVPN any more, if
>> you have Wireguard?
> Because it's already there?:)
>
> Furthermore, some members of our IT team use macs (gasp!) and for them
> it would be much easier to continue to use OpenVPN than to set up
> wireguard-go.
>
I use wireguard on a Mac and it was as simple as "brew install 
wireguard-tools"; create config; "sudo wg-quick up wg0".

My even more stupid question is "why use wireguard if the only thing 
it's carrying is ssh?" - but I guess it's a convenient way to tunnel to 
a network which doesn't have public-routed addresses.

(Aside: I wish ssh had a feature like SNI, so that you could build an 
ssh proxy that forwards incoming connections to the right host.  I have 
done this before using an inbound SOCKS proxy, but it's messy to use)


[-- Attachment #1.2: Type: text/html, Size: 1606 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Roman Mamedov]

On Sat, 6 Oct 2018 11:21:01 +0100
Brian Candler <b.candler@pobox.com> wrote:

> (Aside: I wish ssh had a feature like SNI, so that you could build an 
> ssh proxy that forwards incoming connections to the right host.  I have 
> done this before using an inbound SOCKS proxy, but it's messy to use)

What insane things people invent only not to use IPv6 :)

-- 
With respect,
Roman
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Brian Candler]

[-- Attachment #1.1: Type: text/plain, Size: 485 bytes --]

On 06/10/2018 11:27, Roman Mamedov wrote:
>> (Aside: I wish ssh had a feature like SNI, so that you could build an
>> ssh proxy that forwards incoming connections to the right host.  I have
>> done this before using an inbound SOCKS proxy, but it's messy to use)
> What insane things people invent only not to use IPv6:)

Quite the opposite: I want it so that I can reach my IPv6-addressable 
devices when I'm on an IPv4-only network.  Which means, almost 
everywhere :-)


[-- Attachment #1.2: Type: text/html, Size: 995 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Konstantin Ryabitsev]

On Sat, Oct 06, 2018 at 11:21:01AM +0100, Brian Candler wrote:
> My even more stupid question is "why use wireguard if the only thing it's
> carrying is ssh?" - but I guess it's a convenient way to tunnel to a network
> which doesn't have public-routed addresses.

Right -- and I also don't want to expose ssh ports to the world when not
necessary. It's still a root-perms daemon with a (remote) possibility of
unknown vulnerabilities in it.

> (Aside: I wish ssh had a feature like SNI, so that you could build an ssh
> proxy that forwards incoming connections to the right host.  I have done
> this before using an inbound SOCKS proxy, but it's messy to use)

It also has important downsides that are similar to those in ssh bastion
hosts. When you use a proper VPN, every user gets their own internal IP
address, so their traffic can be still easily distinguished from traffic
belonging to another admin. This is useful for auditing reasons and for
identifying unusual activity (e.g. Alex normally accesses hosts
belonging to project X, but suddenly starts accessing a lot of hosts
that belong to project Y). With bastion hosts or with DNAT-ing proxies
this auditing becomes impossible, since all traffic comes from the same
IP.

-K
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard