All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH FYI 00/46] Generic TLS support across VNC/chardev/migration
@ 2015-09-03 15:38 Daniel P. Berrange
  2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 01/46] sockets: add helpers for creating SocketAddress from a socket Daniel P. Berrange
                   ` (45 more replies)
  0 siblings, 46 replies; 57+ messages in thread
From: Daniel P. Berrange @ 2015-09-03 15:38 UTC (permalink / raw)
  To: qemu-devel
  Cc: Juan Quintela, Dr. David Alan Gilbert, Gerd Hoffmann, Amit Shah,
	Paolo Bonzini

Earlier in the year I posted an RFC series providing impl of
generic TLS support for VNC and chardevs.

  https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg02038.html

Since that time the initial QOM enhancements and crypto API
consolidation patches from that RFC have been merged. I
currently have a set of patches extracting the VNC server
TLS support into a generic crypto API for TLS that is under
active review, hopefully getting close to merge (v6):

  https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00437.html

What I'm posting here is the 46 remaining patches I have that
introduce TLS support across chardevs and migration code, and in
the process standardize I/O channel handling across VNC, chardevs
and migration. I'm doing this so people can see the use context
of the TLS patches I have on review, as well as show the kind of
changes I'm making to the migration, chardev & sockets code to
facilitate conversations with other devs working on the same bits
of code. As compared to the original RFC patches, this series is
a more complete job - there is much more unit testing added and
win32 compatibility verified, and many todos/hacks cleaned up.

I still need to figure out NBD integration, which is harder than
I first thought, due to the need for me to understand the special
needs of the block AIO framework, which doesn't seem to use the
normal APIs for registering I/O callbacks with the main event loop.

I also realize that the ACL code needs to be QOM'ified to make
it possible to create/delete ACLs more generically and share then
across services.

I consider pretty much all of these patches as finished and ready
for review, but this series is a bit large to be reviewed in one
go, so I'm going to continue drip-feed it in reasonable sized chunks.

Some of these 46 patches are fairly generic and can be easily posted
in isolation for quicker merge, so I will be focusing on identifying
such patches and posting them individually to appropriate maintainers.

This series needs to be applied on top of the patch series here

  https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00437.html

For convience I have pushed this entire branch to github

  https://github.com/berrange/qemu/commits/qemu-io-channel-18

Daniel P. Berrange (46):
  sockets: add helpers for creating SocketAddress from a socket
  sockets: move qapi_copy_SocketAddress into qemu-sockets.c
  sockets: allow port to be NULL when listening on IP address
  osdep: add qemu_fork() wrapper for safely handling signals
  coroutine: move into libqemuutil.a library
  io: add abstract QIOChannel classes
  io: add helper module for creating watches on FDs
  io: pull Buffer code out of VNC module
  io: add QIOTask class for async operations
  io: add QIOChannelSocket class
  io: add QIOChannelFile class
  io: add QIOChannelTLS class
  io: add QIOChannelWebsock class
  io: add QIOChannelCommand class
  ui: convert VNC startup code to use SocketAddress
  ui: convert VNC server to use QIOChannelSocket
  ui: convert VNC server to use QIOChannelTLS
  ui: convert VNC server to use QIOChannelWebsock
  char: remove fixed length filename allocation
  char: convert from GIOChannel to QIOChannel
  char: don't assume telnet initialization will not block
  char: introduce support for TLS encrypted TCP chardev backend
  nbd: convert to use the QAPI SocketAddress object
  qemu-nbd: convert to use the QAPI SocketAddress object
  sockets: remove use of QemuOpts from header file
  sockets: remove use of QemuOpts from socket_listen
  sockets: remove use of QemuOpts from socket_connect
  sockets: remove use of QemuOpts from socket_dgram
  migration: remove use of qemu_bufopen from vmstate tests
  migration: remove memory buffer based QEMUFile backend
  migration: move definition of struct QEMUFile back into qemu-file.c
  migration: split migration hooks out of QEMUFileOps
  migration: ensure qemu_fflush() always writes full data amount
  migration: introduce qemu_fset_blocking function on QEMUFile
  migration: force QEMUFile to blocking mode for outgoing migration
  migration: introduce a new QEMUFile impl based on QIOChannel
  migration: convert unix socket protocol to use QIOChannel
  migration: convert tcp socket protocol to use QIOChannel
  migration: convert fd socket protocol to use QIOChannel
  migration: convert exec socket protocol to use QIOChannel
  migration: convert RDMA to use QIOChannel interface
  migration: convert savevm to use QIOChannel for writing to files
  migration: delete QEMUFile sockets implementation
  migration: delete QEMUFile stdio implementation
  migration: support TLS encryption with TCP migration backend
  migration: remove support for non-iovec based write handlers

 MAINTAINERS                             |   7 +
 Makefile                                |   2 +
 Makefile.objs                           |  12 +-
 Makefile.target                         |   2 +
 block.c                                 |   2 +-
 block/nbd.c                             |  66 +--
 block/qcow2.h                           |   2 +-
 block/vdi.c                             |   2 +-
 block/write-threshold.c                 |   2 +-
 blockjob.c                              |   2 +-
 configure                               |  11 +
 coroutine-gthread.c                     |   2 +-
 coroutine-sigaltstack.c                 |   2 +-
 coroutine-ucontext.c                    |   2 +-
 coroutine-win32.c                       |   2 +-
 hw/9pfs/codir.c                         |   2 +-
 hw/9pfs/cofile.c                        |   2 +-
 hw/9pfs/cofs.c                          |   2 +-
 hw/9pfs/coxattr.c                       |   2 +-
 hw/9pfs/virtio-9p-coth.c                |   2 +-
 hw/9pfs/virtio-9p-coth.h                |   2 +-
 hw/9pfs/virtio-9p.h                     |   2 +-
 include/block/block.h                   |   2 +-
 include/block/block_int.h               |   2 +-
 include/io/buffer.h                     | 118 ++++
 include/io/channel-command.h            |  91 +++
 include/io/channel-file.h               |  93 +++
 include/io/channel-socket.h             | 297 ++++++++++
 include/io/channel-tls.h                | 142 +++++
 include/io/channel-watch.h              |  72 +++
 include/io/channel-websock.h            | 108 ++++
 include/io/channel.h                    | 374 +++++++++++++
 include/io/task.h                       | 256 +++++++++
 include/migration/qemu-file.h           |  48 +-
 include/{block => qemu}/coroutine.h     |   0
 include/{block => qemu}/coroutine_int.h |   2 +-
 include/qemu/osdep.h                    |  16 +
 include/qemu/sockets.h                  |  44 +-
 io/Makefile.objs                        |   9 +
 io/buffer.c                             |  65 +++
 io/channel-command.c                    | 370 ++++++++++++
 io/channel-file.c                       | 209 +++++++
 io/channel-socket.c                     | 718 ++++++++++++++++++++++++
 io/channel-tls.c                        | 381 +++++++++++++
 io/channel-watch.c                      | 200 +++++++
 io/channel-websock.c                    | 965 ++++++++++++++++++++++++++++++++
 io/channel.c                            | 229 ++++++++
 io/task.c                               | 150 +++++
 migration/Makefile.objs                 |   6 +-
 migration/exec.c                        |  48 +-
 migration/fd.c                          |  57 +-
 migration/migration.c                   |   9 +-
 migration/qemu-file-buf.c               | 462 ---------------
 migration/qemu-file-channel.c           | 200 +++++++
 migration/qemu-file-internal.h          |  53 --
 migration/qemu-file-stdio.c             | 194 -------
 migration/qemu-file-unix.c              | 238 --------
 migration/qemu-file.c                   | 113 ++--
 migration/rdma.c                        | 252 ++++++---
 migration/savevm.c                      |  16 +-
 migration/tcp.c                         | 372 ++++++++++--
 migration/unix.c                        | 103 ++--
 nbd.c                                   |   2 +-
 qapi-schema.json                        |   2 +
 qemu-char.c                             | 938 +++++++++++++++----------------
 qemu-coroutine-io.c                     |   2 +-
 qemu-coroutine-lock.c                   |   4 +-
 qemu-coroutine-sleep.c                  |   2 +-
 qemu-coroutine.c                        |   4 +-
 qemu-nbd.c                              | 102 ++--
 qemu-options.hx                         |  16 +-
 scripts/create_config                   |   9 +
 tests/.gitignore                        |   7 +
 tests/Makefile                          |  22 +-
 tests/io-channel-helpers.c              | 247 ++++++++
 tests/io-channel-helpers.h              |  33 ++
 tests/test-coroutine.c                  |   4 +-
 tests/test-io-channel-command.c         | 121 ++++
 tests/test-io-channel-file.c            |  89 +++
 tests/test-io-channel-socket.c          | 349 ++++++++++++
 tests/test-io-channel-tls.c             | 335 +++++++++++
 tests/test-io-task.c                    | 276 +++++++++
 tests/test-vmstate.c                    |  57 +-
 thread-pool.c                           |   2 +-
 ui/vnc-auth-sasl.c                      |  87 ++-
 ui/vnc-auth-vencrypt.c                  |  93 ++-
 ui/vnc-enc-tight.c                      |  38 +-
 ui/vnc-enc-zlib.c                       |   6 +-
 ui/vnc-enc-zrle.c                       |  18 +-
 ui/vnc-jobs.c                           |  25 +-
 ui/vnc-ws.c                             | 400 +++----------
 ui/vnc-ws.h                             |  71 +--
 ui/vnc.c                                | 799 ++++++++++++--------------
 ui/vnc.h                                |  77 +--
 util/oslib-posix.c                      |  71 +++
 util/oslib-win32.c                      |   9 +
 util/qemu-sockets.c                     | 427 ++++++++------
 97 files changed, 8875 insertions(+), 3085 deletions(-)
 create mode 100644 include/io/buffer.h
 create mode 100644 include/io/channel-command.h
 create mode 100644 include/io/channel-file.h
 create mode 100644 include/io/channel-socket.h
 create mode 100644 include/io/channel-tls.h
 create mode 100644 include/io/channel-watch.h
 create mode 100644 include/io/channel-websock.h
 create mode 100644 include/io/channel.h
 create mode 100644 include/io/task.h
 rename include/{block => qemu}/coroutine.h (100%)
 rename include/{block => qemu}/coroutine_int.h (98%)
 create mode 100644 io/Makefile.objs
 create mode 100644 io/buffer.c
 create mode 100644 io/channel-command.c
 create mode 100644 io/channel-file.c
 create mode 100644 io/channel-socket.c
 create mode 100644 io/channel-tls.c
 create mode 100644 io/channel-watch.c
 create mode 100644 io/channel-websock.c
 create mode 100644 io/channel.c
 create mode 100644 io/task.c
 delete mode 100644 migration/qemu-file-buf.c
 create mode 100644 migration/qemu-file-channel.c
 delete mode 100644 migration/qemu-file-internal.h
 delete mode 100644 migration/qemu-file-stdio.c
 delete mode 100644 migration/qemu-file-unix.c
 create mode 100644 tests/io-channel-helpers.c
 create mode 100644 tests/io-channel-helpers.h
 create mode 100644 tests/test-io-channel-command.c
 create mode 100644 tests/test-io-channel-file.c
 create mode 100644 tests/test-io-channel-socket.c
 create mode 100644 tests/test-io-channel-tls.c
 create mode 100644 tests/test-io-task.c

-- 
2.4.3

^ permalink raw reply	[flat|nested] 57+ messages in thread

end of thread, other threads:[~2015-09-07 16:29 UTC | newest]

Thread overview: 57+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-03 15:38 [Qemu-devel] [PATCH FYI 00/46] Generic TLS support across VNC/chardev/migration Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 01/46] sockets: add helpers for creating SocketAddress from a socket Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 02/46] sockets: move qapi_copy_SocketAddress into qemu-sockets.c Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 03/46] sockets: allow port to be NULL when listening on IP address Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 04/46] osdep: add qemu_fork() wrapper for safely handling signals Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 05/46] coroutine: move into libqemuutil.a library Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 06/46] io: add abstract QIOChannel classes Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 07/46] io: add helper module for creating watches on FDs Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 08/46] io: pull Buffer code out of VNC module Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 09/46] io: add QIOTask class for async operations Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 10/46] io: add QIOChannelSocket class Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 11/46] io: add QIOChannelFile class Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 12/46] io: add QIOChannelTLS class Daniel P. Berrange
2015-09-07 15:31   ` Dr. David Alan Gilbert
2015-09-07 15:41     ` Daniel P. Berrange
2015-09-07 15:51       ` Dr. David Alan Gilbert
2015-09-07 16:04         ` Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 13/46] io: add QIOChannelWebsock class Daniel P. Berrange
2015-09-07 15:44   ` Dr. David Alan Gilbert
2015-09-07 15:50     ` Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 14/46] io: add QIOChannelCommand class Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 15/46] ui: convert VNC startup code to use SocketAddress Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 16/46] ui: convert VNC server to use QIOChannelSocket Daniel P. Berrange
2015-09-03 15:38 ` [Qemu-devel] [PATCH FYI 17/46] ui: convert VNC server to use QIOChannelTLS Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 18/46] ui: convert VNC server to use QIOChannelWebsock Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 19/46] char: remove fixed length filename allocation Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 20/46] char: convert from GIOChannel to QIOChannel Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 21/46] char: don't assume telnet initialization will not block Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 22/46] char: introduce support for TLS encrypted TCP chardev backend Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 23/46] nbd: convert to use the QAPI SocketAddress object Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 24/46] qemu-nbd: " Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 25/46] sockets: remove use of QemuOpts from header file Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 26/46] sockets: remove use of QemuOpts from socket_listen Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 27/46] sockets: remove use of QemuOpts from socket_connect Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 28/46] sockets: remove use of QemuOpts from socket_dgram Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 29/46] migration: remove use of qemu_bufopen from vmstate tests Daniel P. Berrange
2015-09-07 16:08   ` Dr. David Alan Gilbert
2015-09-07 16:17     ` Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 30/46] migration: remove memory buffer based QEMUFile backend Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 31/46] migration: move definition of struct QEMUFile back into qemu-file.c Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 32/46] migration: split migration hooks out of QEMUFileOps Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 33/46] migration: ensure qemu_fflush() always writes full data amount Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 34/46] migration: introduce qemu_fset_blocking function on QEMUFile Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 35/46] migration: force QEMUFile to blocking mode for outgoing migration Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 36/46] migration: introduce a new QEMUFile impl based on QIOChannel Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 37/46] migration: convert unix socket protocol to use QIOChannel Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 38/46] migration: convert tcp " Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 39/46] migration: convert fd " Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 40/46] migration: convert exec " Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 41/46] migration: convert RDMA to use QIOChannel interface Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 42/46] migration: convert savevm to use QIOChannel for writing to files Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 43/46] migration: delete QEMUFile sockets implementation Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 44/46] migration: delete QEMUFile stdio implementation Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 45/46] migration: support TLS encryption with TCP migration backend Daniel P. Berrange
2015-09-07 16:23   ` Dr. David Alan Gilbert
2015-09-07 16:29     ` Daniel P. Berrange
2015-09-03 15:39 ` [Qemu-devel] [PATCH FYI 46/46] migration: remove support for non-iovec based write handlers Daniel P. Berrange

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.