* [PATCH 0/3] Migration mechanism with FD
@ 2020-02-19 16:33 Oksana Vohchana
2020-02-19 16:33 ` [PATCH 1/3] Adding functions _send_fds and _recv_fds Oksana Vohchana
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Oksana Vohchana @ 2020-02-19 16:33 UTC (permalink / raw)
To: qemu-devel; +Cc: philmd, ehabkost, wainersm, crosa
To test migration through the file descriptor we should build and provide
a path to socket_scm_helper file. This way is inconvenient for acceptance
testing.
This series provides new functions to receive and send messages over a UNIX
socket. And adds a new migration test.
Oksana Vohchana (3):
Adding functions _send_fds and _recv_fds
Updates send_fd_scm function
Acceptance test: FD migration
python/qemu/machine.py | 74 ++++++++++++++++++++++++-----------
tests/acceptance/migration.py | 20 ++++++++++
2 files changed, 72 insertions(+), 22 deletions(-)
--
2.21.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/3] Adding functions _send_fds and _recv_fds
2020-02-19 16:33 [PATCH 0/3] Migration mechanism with FD Oksana Vohchana
@ 2020-02-19 16:33 ` Oksana Vohchana
2020-02-19 16:33 ` [PATCH 2/3] Updates send_fd_scm function Oksana Vohchana
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Oksana Vohchana @ 2020-02-19 16:33 UTC (permalink / raw)
To: qemu-devel; +Cc: philmd, ehabkost, wainersm, crosa
It provides new possibilities to send or receive data through the Unix domain
socket file descriptor.
This is useful for obtaining a socket that belongs to a different network
namespace.
Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
---
python/qemu/machine.py | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 183d8f3d38..8c5bd64795 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -24,6 +24,7 @@ import subprocess
import shutil
import socket
import tempfile
+import array
from . import qmp
@@ -155,6 +156,23 @@ class QEMUMachine(object):
self._args.append(','.join(options))
return self
+ def _recv_fds(self, sock, msglen=8192, maxfds=4096):
+ """
+ Function from https://docs.python.org/3/library/socket.html#socket.socket.recvmsg
+ """
+ fds = array.array("i")
+ msg, ancdata, flags, addr = sock.recvmsg(msglen, socket.CMSG_LEN(maxfds * fds.itemsize))
+ for cmsg_level, cmsg_type, cmsg_data in ancdata:
+ if cmsg_level == socket.SOL_SOCKET and cmsg_type == socket.SCM_RIGHTS:
+ fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])
+ return msg, list(fds)
+
+ def _send_fds(self, sock, msg, fds):
+ """
+ Function from https://docs.python.org/3/library/socket.html#socket.socket.sendmsg
+ """
+ return sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, array.array("i", fds))])
+
def send_fd_scm(self, fd=None, file_path=None):
"""
Send an fd or file_path to socket_scm_helper.
--
2.21.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/3] Updates send_fd_scm function
2020-02-19 16:33 [PATCH 0/3] Migration mechanism with FD Oksana Vohchana
2020-02-19 16:33 ` [PATCH 1/3] Adding functions _send_fds and _recv_fds Oksana Vohchana
@ 2020-02-19 16:33 ` Oksana Vohchana
2020-02-19 16:33 ` [PATCH 3/3] Acceptance test: FD migration Oksana Vohchana
2020-02-19 18:02 ` [PATCH 0/3] Migration mechanism with FD no-reply
3 siblings, 0 replies; 5+ messages in thread
From: Oksana Vohchana @ 2020-02-19 16:33 UTC (permalink / raw)
To: qemu-devel; +Cc: philmd, ehabkost, wainersm, crosa
A qemu-iotest uses for FD-migration test a helper program "socket_scm_helper".
And it makes some problems if you didn't build it with a QEMU. And now we can
use new methods for the socket that allow us to send a file/socket descriptor
(with access and permissions) from one process to another.
Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
---
python/qemu/machine.py | 56 +++++++++++++++++++++++++-----------------
1 file changed, 34 insertions(+), 22 deletions(-)
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 8c5bd64795..0936b71856 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -173,19 +173,24 @@ class QEMUMachine(object):
"""
return sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, array.array("i", fds))])
- def send_fd_scm(self, fd=None, file_path=None):
+ def send_fd_scm(self, fd=None, file_path=None, data=None):
"""
- Send an fd or file_path to socket_scm_helper.
+ Can be used in two different cases.
+ Send an fd or file_path to socket_scm_helper or
+ provide data and fd to send it to the socket.
- Exactly one of fd and file_path must be given.
+ Exactly one of fd and file_path must be given to the case of socket_scm_helper
If it is file_path, the helper will open that file and pass its own fd.
+
+ To second case need adds data that include a QMP request and fd
"""
# In iotest.py, the qmp should always use unix socket.
assert self._qmp.is_scm_available()
- if self._socket_scm_helper is None:
- raise QEMUMachineError("No path to socket_scm_helper set")
- if not os.path.exists(self._socket_scm_helper):
- raise QEMUMachineError("%s does not exist" %
+ if data is None:
+ if self._socket_scm_helper is None:
+ raise QEMUMachineError("No path to socket_scm_helper set or data not provided")
+ if not os.path.exists(self._socket_scm_helper):
+ raise QEMUMachineError("%s does not exist" %
self._socket_scm_helper)
# This did not exist before 3.4, but since then it is
@@ -195,24 +200,31 @@ class QEMUMachine(object):
if fd is not None:
os.set_inheritable(fd, True)
- fd_param = ["%s" % self._socket_scm_helper,
- "%d" % self._qmp.get_sock_fd()]
+ if data is None:
+ fd_param = ["%s" % self._socket_scm_helper,
+ "%d" % self._qmp.get_sock_fd()]
+ if file_path is not None:
+ assert fd is None
+ fd_param.append(file_path)
+ else:
+ assert fd is not None
+ fd_param.append(str(fd))
- if file_path is not None:
- assert fd is None
- fd_param.append(file_path)
- else:
- assert fd is not None
- fd_param.append(str(fd))
+ devnull = open(os.path.devnull, 'rb')
+ proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT, close_fds=False)
+ output = proc.communicate()[0]
+ if output:
+ LOG.debug(output)
- devnull = open(os.path.devnull, 'rb')
- proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE,
- stderr=subprocess.STDOUT, close_fds=False)
- output = proc.communicate()[0]
- if output:
- LOG.debug(output)
+ return proc.returncode
- return proc.returncode
+ else:
+ sock_fd = socket.fromfd(self._qmp.get_sock_fd(), socket.AF_UNIX, socket.SOCK_STREAM)
+ fds_param = [fd, self._qmp.get_sock_fd()]
+ self._send_fds(sock_fd, data, fds_param)
+ self._recv_fds(sock_fd)
+ return self
@staticmethod
def _remove_if_exists(path):
--
2.21.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/3] Acceptance test: FD migration
2020-02-19 16:33 [PATCH 0/3] Migration mechanism with FD Oksana Vohchana
2020-02-19 16:33 ` [PATCH 1/3] Adding functions _send_fds and _recv_fds Oksana Vohchana
2020-02-19 16:33 ` [PATCH 2/3] Updates send_fd_scm function Oksana Vohchana
@ 2020-02-19 16:33 ` Oksana Vohchana
2020-02-19 18:02 ` [PATCH 0/3] Migration mechanism with FD no-reply
3 siblings, 0 replies; 5+ messages in thread
From: Oksana Vohchana @ 2020-02-19 16:33 UTC (permalink / raw)
To: qemu-devel; +Cc: philmd, ehabkost, wainersm, crosa
Adds a new migration test through the file descriptor.
Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
---
tests/acceptance/migration.py | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py
index a8367ca023..b96a897f3b 100644
--- a/tests/acceptance/migration.py
+++ b/tests/acceptance/migration.py
@@ -10,7 +10,10 @@
# later. See the COPYING file in the top-level directory.
+import os
import tempfile
+from socket import socketpair, AF_UNIX, SOCK_STREAM
+
from avocado_qemu import Test
from avocado import skipUnless
@@ -75,3 +78,20 @@ class Migration(Test):
"""
free_port = self._get_free_port()
dest_uri = 'exec:nc -l localhost %u' % free_port
+
+ def test_migration_with_fd(self):
+ opaque = 'fd-migration'
+ data_to_send = b"{\"execute\": \"getfd\", \"arguments\": {\"fdname\": \"fd-migration\"}}"
+ send_socket, recv_socket = socketpair(AF_UNIX, SOCK_STREAM)
+ fd1 = send_socket.fileno()
+ fd2 = recv_socket.fileno()
+ os.set_inheritable(fd2, True)
+
+ source_vm = self.get_vm()
+ source_vm.launch()
+ source_vm.send_fd_scm(fd=fd1, data=data_to_send)
+
+ dest_vm = self.get_vm('-incoming', 'fd:%s' % fd2)
+ dest_vm.launch()
+ source_vm.qmp('migrate', uri='fd:%s' % opaque)
+ self.assert_migration(source_vm, dest_vm)
--
2.21.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 0/3] Migration mechanism with FD
2020-02-19 16:33 [PATCH 0/3] Migration mechanism with FD Oksana Vohchana
` (2 preceding siblings ...)
2020-02-19 16:33 ` [PATCH 3/3] Acceptance test: FD migration Oksana Vohchana
@ 2020-02-19 18:02 ` no-reply
3 siblings, 0 replies; 5+ messages in thread
From: no-reply @ 2020-02-19 18:02 UTC (permalink / raw)
To: ovoshcha; +Cc: crosa, philmd, qemu-devel, wainersm, ehabkost
Patchew URL: https://patchew.org/QEMU/20200219163344.27651-1-ovoshcha@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Subject: [PATCH 0/3] Migration mechanism with FD
Message-id: 20200219163344.27651-1-ovoshcha@redhat.com
Type: series
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
5429c0a Acceptance test: FD migration
f415c4c Updates send_fd_scm function
810c13a Adding functions _send_fds and _recv_fds
=== OUTPUT BEGIN ===
1/3 Checking commit 810c13a52356 (Adding functions _send_fds and _recv_fds)
WARNING: line over 80 characters
#33: FILE: python/qemu/machine.py:161:
+ Function from https://docs.python.org/3/library/socket.html#socket.socket.recvmsg
ERROR: line over 90 characters
#36: FILE: python/qemu/machine.py:164:
+ msg, ancdata, flags, addr = sock.recvmsg(msglen, socket.CMSG_LEN(maxfds * fds.itemsize))
WARNING: line over 80 characters
#38: FILE: python/qemu/machine.py:166:
+ if cmsg_level == socket.SOL_SOCKET and cmsg_type == socket.SCM_RIGHTS:
ERROR: line over 90 characters
#39: FILE: python/qemu/machine.py:167:
+ fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) % fds.itemsize)])
WARNING: line over 80 characters
#44: FILE: python/qemu/machine.py:172:
+ Function from https://docs.python.org/3/library/socket.html#socket.socket.sendmsg
ERROR: line over 90 characters
#46: FILE: python/qemu/machine.py:174:
+ return sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, array.array("i", fds))])
total: 3 errors, 3 warnings, 30 lines checked
Patch 1/3 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/3 Checking commit f415c4c2bd6a (Updates send_fd_scm function)
WARNING: line over 80 characters
#32: FILE: python/qemu/machine.py:182:
+ Exactly one of fd and file_path must be given to the case of socket_scm_helper
ERROR: line over 90 characters
#45: FILE: python/qemu/machine.py:191:
+ raise QEMUMachineError("No path to socket_scm_helper set or data not provided")
WARNING: line over 80 characters
#68: FILE: python/qemu/machine.py:214:
+ proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE,
ERROR: line over 90 characters
#91: FILE: python/qemu/machine.py:223:
+ sock_fd = socket.fromfd(self._qmp.get_sock_fd(), socket.AF_UNIX, socket.SOCK_STREAM)
total: 2 errors, 2 warnings, 78 lines checked
Patch 2/3 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/3 Checking commit 5429c0afdc21 (Acceptance test: FD migration)
ERROR: line over 90 characters
#34: FILE: tests/acceptance/migration.py:84:
+ data_to_send = b"{\"execute\": \"getfd\", \"arguments\": {\"fdname\": \"fd-migration\"}}"
total: 1 errors, 0 warnings, 30 lines checked
Patch 3/3 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20200219163344.27651-1-ovoshcha@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-02-19 18:03 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-19 16:33 [PATCH 0/3] Migration mechanism with FD Oksana Vohchana
2020-02-19 16:33 ` [PATCH 1/3] Adding functions _send_fds and _recv_fds Oksana Vohchana
2020-02-19 16:33 ` [PATCH 2/3] Updates send_fd_scm function Oksana Vohchana
2020-02-19 16:33 ` [PATCH 3/3] Acceptance test: FD migration Oksana Vohchana
2020-02-19 18:02 ` [PATCH 0/3] Migration mechanism with FD no-reply
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).