Re: [Libguestfs] Provide NBD via Browser over Websockets

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Eric Blake]

[adding qemu list]

On 5/29/20 4:37 AM, Richard W.M. Jones wrote:
> Going back to the original email from 2018:
> 
>> It might be neat to attach ISOs to KVM guests via websockets.  Basically
>> the  browser would be the NBD "server" and an NBD client would run on the
>> hypervisor, then use `virsh change-media vm1 hdc --insert /dev/nbd0` could
>> use an ISO from my desk to boot from.
>>
>> Here's an HTML5 open file example:
>> https://stackoverflow.com/questions/3582671/how-to-open-a-local-disk-file-with-javascript
>>
>> and the NBD protocol looks simple enough to implement in javascript:
>> https://stackoverflow.com/questions/17295140/where-is-the-network-block-device-format-describled
> 
> So I think what you mean here is that in a browser you'd open a local
> (eg) ISO, and then that ISO could be shared with a remote VM.  The
> browser runs a Javascript NBD server.  The remote VM is qemu.  Between
> the two is a WebSocket.
> 
> I've seen this being done with an HP blade server of some kind and
> IIRC the client side used a Java applet.  No idea what the protocol
> was but likely something proprietary.  It was nevertheless a useful
> feature, eg to boot the server from an install CD that you have
> locally.
> 
> I guess the problem is two-fold:
> 
> (1) You need to write an NBD server in Javascript.  Not especially
> difficult, particularly if you avoid any complicated features, and I
> guess you only need read support.

Or use an existing NBD server over a Unix socket paired to a WebSocket 
proxy that forwards all traffic from the Unix socket over a WebSocket. 
That may be easier than writing the NBD server itself in Javascript.

> 
> (2) You need to persuade qemu's NBD client to read from a WebSocket.
> I didn't really know anything about WebSockets until today but it
> seems as if they are a full-duplex protocol layered on top of HTTP [a].
> Is there a WebSocket proxy that turns WS into plain TCP (a bit like
> stunnel)?  Google suggests [b].
> 
> [a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
> [b] https://github.com/novnc/websockify

qemu already knows how to connect as a client to websockets; Dan 
Berrange knows more about that setup.  I suspect it would not be too 
difficult to teach the qemu NBD client code to use a WebSocket instead 
of a Unix or TCP socket as its data source.

> 
> ...
> 
>> When qemu is running headless using a VNC or Spice display we can access
>> the display of https+websockets using things like noVNC---which is out of
>> scope to this converstation---but I'm just saying that such an existing
>> web front-end for the display could be extended to have an "Insert CDROM"
>> button and use the javascript file IO for the user to reference a local
>> file and pass it to qemu over nbd, perhaps via nbdkit.
> 
> I'm not sure how nbdkit would be involved, unless it was compiled
> to WASM or something like that.
> 
> But the plan above sounds feasible, albeit a chunk of work.
> 
> Rich.
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Daniel P. Berrangé]

On Fri, May 29, 2020 at 07:50:14AM -0500, Eric Blake wrote:
> [adding qemu list]
> 
> On 5/29/20 4:37 AM, Richard W.M. Jones wrote:
> > Going back to the original email from 2018:
> > 
> > > It might be neat to attach ISOs to KVM guests via websockets.  Basically
> > > the  browser would be the NBD "server" and an NBD client would run on the
> > > hypervisor, then use `virsh change-media vm1 hdc --insert /dev/nbd0` could
> > > use an ISO from my desk to boot from.
> > > 
> > > Here's an HTML5 open file example:
> > > https://stackoverflow.com/questions/3582671/how-to-open-a-local-disk-file-with-javascript
> > > 
> > > and the NBD protocol looks simple enough to implement in javascript:
> > > https://stackoverflow.com/questions/17295140/where-is-the-network-block-device-format-describled
> > 
> > So I think what you mean here is that in a browser you'd open a local
> > (eg) ISO, and then that ISO could be shared with a remote VM.  The
> > browser runs a Javascript NBD server.  The remote VM is qemu.  Between
> > the two is a WebSocket.
> > 
> > I've seen this being done with an HP blade server of some kind and
> > IIRC the client side used a Java applet.  No idea what the protocol
> > was but likely something proprietary.  It was nevertheless a useful
> > feature, eg to boot the server from an install CD that you have
> > locally.
> > 
> > I guess the problem is two-fold:
> > 
> > (1) You need to write an NBD server in Javascript.  Not especially
> > difficult, particularly if you avoid any complicated features, and I
> > guess you only need read support.
> 
> Or use an existing NBD server over a Unix socket paired to a WebSocket proxy
> that forwards all traffic from the Unix socket over a WebSocket. That may be
> easier than writing the NBD server itself in Javascript.
> 
> > 
> > (2) You need to persuade qemu's NBD client to read from a WebSocket.
> > I didn't really know anything about WebSockets until today but it
> > seems as if they are a full-duplex protocol layered on top of HTTP [a].
> > Is there a WebSocket proxy that turns WS into plain TCP (a bit like
> > stunnel)?  Google suggests [b].
> > 
> > [a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
> > [b] https://github.com/novnc/websockify
> 
> qemu already knows how to connect as a client to websockets; Dan Berrange
> knows more about that setup.  I suspect it would not be too difficult to
> teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP
> socket as its data source.

Actually the inverse. The QIOChannelWebsocket impl is only the server
side of the problem, as used by QEMU's VNC server. We've never implemented
the client side. There is nothing especially stopping us doing that - just
needs someone motivated with time to work on it.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Eric Blake]

On 5/29/20 8:50 AM, Daniel P. Berrangé wrote:

>>> (2) You need to persuade qemu's NBD client to read from a WebSocket.
>>> I didn't really know anything about WebSockets until today but it
>>> seems as if they are a full-duplex protocol layered on top of HTTP [a].
>>> Is there a WebSocket proxy that turns WS into plain TCP (a bit like
>>> stunnel)?  Google suggests [b].
>>>
>>> [a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
>>> [b] https://github.com/novnc/websockify
>>
>> qemu already knows how to connect as a client to websockets; Dan Berrange
>> knows more about that setup.  I suspect it would not be too difficult to
>> teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP
>> socket as its data source.
> 
> Actually the inverse. The QIOChannelWebsocket impl is only the server
> side of the problem, as used by QEMU's VNC server. We've never implemented
> the client side. There is nothing especially stopping us doing that - just
> needs someone motivated with time to work on it.

In the meantime, you may still be able to set up something like:

local machine:
iso -> NBD server -> Unix socket -> websockify -> WebSocket

remote machine:
WebSocket -> websockify -> Unix socket -> qemu NBD client

Adding websocket client support into qemu would reduce the length of the 
chain slightly (for less data copying) by getting rid of a websockify 
proxy middleman, but would not necessarily improve performance (it's 
hard to say where the latency bottlenecks will be in the chain).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Richard W.M. Jones]

On Fri, May 29, 2020 at 08:58:06AM -0500, Eric Blake wrote:
> On 5/29/20 8:50 AM, Daniel P. Berrangé wrote:
> 
> >>>(2) You need to persuade qemu's NBD client to read from a WebSocket.
> >>>I didn't really know anything about WebSockets until today but it
> >>>seems as if they are a full-duplex protocol layered on top of HTTP [a].
> >>>Is there a WebSocket proxy that turns WS into plain TCP (a bit like
> >>>stunnel)?  Google suggests [b].
> >>>
> >>>[a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
> >>>[b] https://github.com/novnc/websockify
> >>
> >>qemu already knows how to connect as a client to websockets; Dan Berrange
> >>knows more about that setup.  I suspect it would not be too difficult to
> >>teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP
> >>socket as its data source.
> >
> >Actually the inverse. The QIOChannelWebsocket impl is only the server
> >side of the problem, as used by QEMU's VNC server. We've never implemented
> >the client side. There is nothing especially stopping us doing that - just
> >needs someone motivated with time to work on it.
> 
> In the meantime, you may still be able to set up something like:
> 
> local machine:
> iso -> NBD server -> Unix socket -> websockify -> WebSocket

I guess the idea is to have a zero-install solution for the browser.
As I said in the email earlier this is very common for IPMI-type
remote access to blade servers and in my experience is implemented
using a Java applet and a proprietary protocol terminated at the BMC
(which then emulates a virtual CDROM to the server).  There are some
HP blade servers on Red Hat's internal Beaker instance where you can
play with this.  For qemu we wouldn't need to invent a new protocol
when NBD is available and already implemented (albeit not yet on top
of WebSockets).

The NBD server must run inside the browser and therefore be either
written from scratch in Javascript, or an existing server
cross-compiled to WASM (if that is possible - I don't really know).

> remote machine:
> WebSocket -> websockify -> Unix socket -> qemu NBD client
> 
> Adding websocket client support into qemu would reduce the length of
> the chain slightly (for less data copying) by getting rid of a
> websockify proxy middleman, but would not necessarily improve
> performance (it's hard to say where the latency bottlenecks will be
> in the chain).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Eric Wheeler]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3018 bytes --]

On Fri, 29 May 2020, Richard W.M. Jones wrote:
> On Fri, May 29, 2020 at 08:58:06AM -0500, Eric Blake wrote:
> > On 5/29/20 8:50 AM, Daniel P. Berrangé wrote:
> > 
> > >>>(2) You need to persuade qemu's NBD client to read from a WebSocket.
> > >>>I didn't really know anything about WebSockets until today but it
> > >>>seems as if they are a full-duplex protocol layered on top of HTTP [a].
> > >>>Is there a WebSocket proxy that turns WS into plain TCP (a bit like
> > >>>stunnel)?  Google suggests [b].
> > >>>
> > >>>[a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
> > >>>[b] https://github.com/novnc/websockify
> > >>
> > >>qemu already knows how to connect as a client to websockets; Dan Berrange
> > >>knows more about that setup.  I suspect it would not be too difficult to
> > >>teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP
> > >>socket as its data source.
> > >
> > >Actually the inverse. The QIOChannelWebsocket impl is only the server
> > >side of the problem, as used by QEMU's VNC server. We've never implemented
> > >the client side. There is nothing especially stopping us doing that - just
> > >needs someone motivated with time to work on it.
> > 
> > In the meantime, you may still be able to set up something like:
> > 
> > local machine:
> > iso -> NBD server -> Unix socket -> websockify -> WebSocket
> 
> I guess the idea is to have a zero-install solution for the browser.
> As I said in the email earlier this is very common for IPMI-type
> remote access to blade servers and in my experience is implemented
> using a Java applet and a proprietary protocol terminated at the BMC
> (which then emulates a virtual CDROM to the server).  There are some
> HP blade servers on Red Hat's internal Beaker instance where you can
> play with this.  For qemu we wouldn't need to invent a new protocol
> when NBD is available and already implemented (albeit not yet on top
> of WebSockets).
> 
> The NBD server must run inside the browser and therefore be either
> written from scratch in Javascript, or an existing server
> cross-compiled to WASM (if that is possible - I don't really know).

Interesting idea about WASM.  I'll see if I can build one of the simple 
nbd servers that are around.  Not sure how to link it to the JS file IO, 
however.

--
Eric Wheeler


> > remote machine:
> > WebSocket -> websockify -> Unix socket -> qemu NBD client
> > 
> > Adding websocket client support into qemu would reduce the length of
> > the chain slightly (for less data copying) by getting rid of a
> > websockify proxy middleman, but would not necessarily improve
> > performance (it's hard to say where the latency bottlenecks will be
> > in the chain).
> 
> Rich.
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
> 
> 

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Richard W.M. Jones]

On Fri, May 29, 2020 at 09:08:29PM +0000, Eric Wheeler wrote:
> On Fri, 29 May 2020, Richard W.M. Jones wrote:
> > On Fri, May 29, 2020 at 08:58:06AM -0500, Eric Blake wrote:
> > > On 5/29/20 8:50 AM, Daniel P. Berrangé wrote:
> > > 
> > > >>>(2) You need to persuade qemu's NBD client to read from a WebSocket.
> > > >>>I didn't really know anything about WebSockets until today but it
> > > >>>seems as if they are a full-duplex protocol layered on top of HTTP [a].
> > > >>>Is there a WebSocket proxy that turns WS into plain TCP (a bit like
> > > >>>stunnel)?  Google suggests [b].
> > > >>>
> > > >>>[a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
> > > >>>[b] https://github.com/novnc/websockify
> > > >>
> > > >>qemu already knows how to connect as a client to websockets; Dan Berrange
> > > >>knows more about that setup.  I suspect it would not be too difficult to
> > > >>teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP
> > > >>socket as its data source.
> > > >
> > > >Actually the inverse. The QIOChannelWebsocket impl is only the server
> > > >side of the problem, as used by QEMU's VNC server. We've never implemented
> > > >the client side. There is nothing especially stopping us doing that - just
> > > >needs someone motivated with time to work on it.
> > > 
> > > In the meantime, you may still be able to set up something like:
> > > 
> > > local machine:
> > > iso -> NBD server -> Unix socket -> websockify -> WebSocket
> > 
> > I guess the idea is to have a zero-install solution for the browser.
> > As I said in the email earlier this is very common for IPMI-type
> > remote access to blade servers and in my experience is implemented
> > using a Java applet and a proprietary protocol terminated at the BMC
> > (which then emulates a virtual CDROM to the server).  There are some
> > HP blade servers on Red Hat's internal Beaker instance where you can
> > play with this.  For qemu we wouldn't need to invent a new protocol
> > when NBD is available and already implemented (albeit not yet on top
> > of WebSockets).
> > 
> > The NBD server must run inside the browser and therefore be either
> > written from scratch in Javascript, or an existing server
> > cross-compiled to WASM (if that is possible - I don't really know).
> 
> Interesting idea about WASM.  I'll see if I can build one of the simple 
> nbd servers that are around.  Not sure how to link it to the JS file IO, 
> however.

After reading a bit about compiling to WebSockets it sounds like you
can cross-compile a C program, but there's no library support at all.
IOW to port an existing server you'd have to implement enough of POSIX
to make it work.  nbdkit has a liberal license deliberately to make it
possible to chop it up and incorporate it into completely forked
codebases (nbdkit is a plot to make NBD more popular).

But since NBD is pretty simple, a fresh Javascript server might be
easier, especially if you stick to only implementing reads.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Re: [Libguestfs] Provide NBD via Browser over Websockets

[Eric Wheeler]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3767 bytes --]

On Sat, 30 May 2020, Richard W.M. Jones wrote:
> On Fri, May 29, 2020 at 09:08:29PM +0000, Eric Wheeler wrote:
> > On Fri, 29 May 2020, Richard W.M. Jones wrote:
> > > On Fri, May 29, 2020 at 08:58:06AM -0500, Eric Blake wrote:
> > > > On 5/29/20 8:50 AM, Daniel P. Berrang�© wrote:
> > > > 
> > > > >>>(2) You need to persuade qemu's NBD client to read from a WebSocket.
> > > > >>>I didn't really know anything about WebSockets until today but it
> > > > >>>seems as if they are a full-duplex protocol layered on top of HTTP [a].
> > > > >>>Is there a WebSocket proxy that turns WS into plain TCP (a bit like
> > > > >>>stunnel)?  Google suggests [b].
> > > > >>>
> > > > >>>[a] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
> > > > >>>[b] https://github.com/novnc/websockify
> > > > >>
> > > > >>qemu already knows how to connect as a client to websockets; Dan Berrange
> > > > >>knows more about that setup.  I suspect it would not be too difficult to
> > > > >>teach the qemu NBD client code to use a WebSocket instead of a Unix or TCP
> > > > >>socket as its data source.
> > > > >
> > > > >Actually the inverse. The QIOChannelWebsocket impl is only the server
> > > > >side of the problem, as used by QEMU's VNC server. We've never implemented
> > > > >the client side. There is nothing especially stopping us doing that - just
> > > > >needs someone motivated with time to work on it.
> > > > 
> > > > In the meantime, you may still be able to set up something like:
> > > > 
> > > > local machine:
> > > > iso -> NBD server -> Unix socket -> websockify -> WebSocket
> > > 
> > > I guess the idea is to have a zero-install solution for the browser.
> > > As I said in the email earlier this is very common for IPMI-type
> > > remote access to blade servers and in my experience is implemented
> > > using a Java applet and a proprietary protocol terminated at the BMC
> > > (which then emulates a virtual CDROM to the server).  There are some
> > > HP blade servers on Red Hat's internal Beaker instance where you can
> > > play with this.  For qemu we wouldn't need to invent a new protocol
> > > when NBD is available and already implemented (albeit not yet on top
> > > of WebSockets).
> > > 
> > > The NBD server must run inside the browser and therefore be either
> > > written from scratch in Javascript, or an existing server
> > > cross-compiled to WASM (if that is possible - I don't really know).
> > 
> > Interesting idea about WASM.  I'll see if I can build one of the simple 
> > nbd servers that are around.  Not sure how to link it to the JS file IO, 
> > however.
> 
> After reading a bit about compiling to WebSockets it sounds like you
> can cross-compile a C program, but there's no library support at all.
> IOW to port an existing server you'd have to implement enough of POSIX
> to make it work.  nbdkit has a liberal license deliberately to make it
> possible to chop it up and incorporate it into completely forked
> codebases (nbdkit is a plot to make NBD more popular).
> 
> But since NBD is pretty simple, a fresh Javascript server might be
> easier, especially if you stick to only implementing reads.

Good point, I'll wait on trying WASM.  

If anyone plans to implement NBD in JS let me know, otherwise I'll 
probably implement a stripped down verion to integrate as an nbdkit plugin 
to avoid re-writing all the handshake and version bits.

--
Eric Wheeler

> Rich.
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-top is 'top' for virtual machines.  Tiny program with many
> powerful monitoring features, net stats, disk stats, logging, etc.
> http://people.redhat.com/~rjones/virt-top
> 
>