Re: RFC: XenSock brainstorming

[Paul Durrant <Paul.Durrant@citrix.com>]

> -----Original Message-----
> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
> Andrew Cooper
> Sent: 06 June 2016 10:58
> To: Stefano Stabellini; xen-devel@lists.xenproject.org
> Cc: joao.m.martins@oracle.com; Wei Liu; Roger Pau Monne
> Subject: Re: [Xen-devel] RFC: XenSock brainstorming
> 
> On 06/06/16 10:33, Stefano Stabellini wrote:
> > Hi all,
> >
> > a couple of months ago I started working on a new PV protocol for
> > virtualizing syscalls. I named it XenSock, as its main purpose is to
> > allow the implementation of the POSIX socket API in a domain other than
> > the one of the caller. It allows connect, accept, recvmsg, sendmsg, etc
> > to be implemented directly in Dom0. In a way this is conceptually
> > similar to virtio-9pfs, but for sockets rather than filesystem APIs.
> > See this diagram as reference:
> >
> > https://docs.google.com/presentation/d/1z4AICTY2ejAjZ-
> Ul15GTL3i_wcmhKQJA7tcXwhI3dys/edit?usp=sharing
> >
> > The frontends and backends could live either in userspace or kernel
> > space, with different trade-offs. My current prototype is based on Linux
> > kernel drivers but it would be nice to have userspace drivers too.
> > Discussing where the drivers could be implemented it's beyond the scope
> > of this email.
> 
> Just to confirm, you are intending to create a cross-domain transport
> for all AF_ socket types, or just some?
> 
> >
> >
> > # Goals
> >
> > The goal of the protocol is to provide networking capabilities to any
> > guests, with the following added benefits:
> 
> Throughout, s/Dom0/the backend/
> 
> I expect running the backend in dom0 will be the overwhelmingly common
> configuration, but you should avoid designing the protocol for just this
> usecase.
> 
> >
> > * guest networking should work out of the box with VPNs, wireless
> >   networks and any other complex network configurations in Dom0
> >
> > * guest services should listen on ports bound directly to Dom0 IP
> >   addresses, fitting naturally in a Docker based workflow, where guests
> >   are Docker containers
> >
> > * Dom0 should have full visibility on the guest behavior and should be
> >   able to perform inexpensive filtering and manipulation of guest calls
> >
> > * XenSock should provide excellent performance. Unoptimized early code
> >   reaches 22 Gbit/sec TCP single stream and scales to 60 Gbit/sec with 3
> >   streams.
> 
> What happens if domU tries to open an AF_INET socket, and the domain has
> both sockfront and netfront ?  What happens if a domain has multiple
> sockfronts?
> 

This sounds awfully like a class of problem that the open onload (http://www.openonload.org/) stack had to solve, and it involved having to track updates to various kernel tables involved in inet routing and having to keep a 'standard' inet socket in hand even when setting up an intercepted (read 'PV' for this connect ) socket since, until connect, you don’t know what the far end is or how to get to it.

Having your own AF is definitely a much easier starting point. It also means you get to define all the odd corner-case semantics rather than having to emulate Linux/BSD/Solaris/etc. quirks.

  Paul

> ~Andrew
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel