Hi, Perhaps the main reason we cannot run important tests such as 041 in CI is that when they care Unix sockets in $TEST_DIR, the path may become too long to connect to them. To get by this problem, this series lets the check script create a new temporary directory (mktemp -d) and then makes the iotests use it for all Unix sockets. Max Reitz (23): iotests: Introduce $SOCK_DIR iotests.py: Store socket files in $SOCK_DIR iotests.py: Add @base_dir to FilePaths etc. iotests: Filter $SOCK_DIR iotests: Let common.nbd create socket in $SOCK_DIR iotests/083: Create socket in $SOCK_DIR iotests/140: Create socket in $SOCK_DIR iotests/143: Create socket in $SOCK_DIR iotests/147: Create socket in $SOCK_DIR iotests/181: Create socket in $SOCK_DIR iotests/182: Create socket in $SOCK_DIR iotests/183: Create socket in $SOCK_DIR iotests/192: Create socket in $SOCK_DIR iotests/194: Create sockets in $SOCK_DIR iotests/201: Create socket in $SOCK_DIR iotests/205: Create socket in $SOCK_DIR iotests/208: Create socket in $SOCK_DIR iotests/209: Create socket in $SOCK_DIR iotests/222: Create socket in $SOCK_DIR iotests/223: Create socket in $SOCK_DIR iotests/240: Create socket in $SOCK_DIR iotests/267: Create socket in $SOCK_DIR iotests: Drop TEST_DIR filter from _filter_nbd python/qemu/machine.py | 15 +++++++++++--- python/qemu/qtest.py | 9 ++++++--- tests/qemu-iotests/083 | 6 +++--- tests/qemu-iotests/083.out | 34 ++++++++++++++++---------------- tests/qemu-iotests/140 | 8 ++++---- tests/qemu-iotests/140.out | 2 +- tests/qemu-iotests/143 | 6 +++--- tests/qemu-iotests/143.out | 2 +- tests/qemu-iotests/147 | 2 +- tests/qemu-iotests/181 | 2 +- tests/qemu-iotests/182 | 4 ++-- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/192 | 4 ++-- tests/qemu-iotests/192.out | 2 +- tests/qemu-iotests/194 | 4 ++-- tests/qemu-iotests/201 | 2 +- tests/qemu-iotests/205 | 2 +- tests/qemu-iotests/208 | 2 +- tests/qemu-iotests/209 | 3 ++- tests/qemu-iotests/222 | 2 +- tests/qemu-iotests/223 | 14 ++++++------- tests/qemu-iotests/240 | 4 ++-- tests/qemu-iotests/241 | 2 -- tests/qemu-iotests/267 | 4 ++-- tests/qemu-iotests/267.out | 2 +- tests/qemu-iotests/check | 17 ++++++++++++++++ tests/qemu-iotests/common.filter | 7 +++++-- tests/qemu-iotests/common.nbd | 2 +- tests/qemu-iotests/iotests.py | 16 ++++++++------- 29 files changed, 107 insertions(+), 74 deletions(-) -- 2.21.0
Unix sockets generally have a maximum path length. Depending on your $TEST_DIR, it may be exceeded and then all tests that create and use Unix sockets there may fail. Circumvent this by adding a new scratch directory specifically for Unix socket files. It defaults to a temporary directory (mktemp -d) that is completely removed after the iotests are done. (By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which should be short enough for our use cases.) Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/check | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 588c453a94..a257215448 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -97,6 +97,7 @@ IMGFMT -- $FULL_IMGFMT_DETAILS IMGPROTO -- $IMGPROTO PLATFORM -- $FULL_HOST_DETAILS TEST_DIR -- $TEST_DIR +SOCK_DIR -- $SOCK_DIR SOCKET_SCM_HELPER -- $SOCKET_SCM_HELPER EOF @@ -121,6 +122,16 @@ if [ ! -e "$TEST_DIR" ]; then mkdir "$TEST_DIR" fi +tmp_sock_dir=false +if [ -z "$SOCK_DIR" ]; then + SOCK_DIR=$(mktemp -d) + tmp_sock_dir=true +fi + +if [ ! -d "$SOCK_DIR" ]; then + mkdir "$SOCK_DIR" +fi + diff="diff -u" verbose=false debug=false @@ -534,6 +545,7 @@ if [ -z "$SAMPLE_IMG_DIR" ]; then fi export TEST_DIR +export SOCK_DIR export SAMPLE_IMG_DIR if [ -s $tmp.list ] @@ -716,6 +728,11 @@ END { if (NR > 0) { rm -f "${TEST_DIR}"/*.out "${TEST_DIR}"/*.err "${TEST_DIR}"/*.time rm -f "${TEST_DIR}"/check.pid "${TEST_DIR}"/check.sts rm -f $tmp.* + + if $tmp_sock_dir + then + rm -rf "$SOCK_DIR" + fi } trap "_wrapup; exit \$status" 0 1 2 3 15 -- 2.21.0
iotests.py itself does not store socket files, but it machine.py and qtest.py do. iotests.py needs to pass the respective path to them, and they need to adhere to it. Signed-off-by: Max Reitz <mreitz@redhat.com> --- python/qemu/machine.py | 15 ++++++++++++--- python/qemu/qtest.py | 9 ++++++--- tests/qemu-iotests/iotests.py | 4 +++- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 128a3d1dc2..2024e8b1b1 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -71,7 +71,7 @@ class QEMUMachine(object): def __init__(self, binary, args=None, wrapper=None, name=None, test_dir="/var/tmp", monitor_address=None, - socket_scm_helper=None): + socket_scm_helper=None, sock_dir=None): ''' Initialize a QEMUMachine @@ -90,6 +90,8 @@ class QEMUMachine(object): wrapper = [] if name is None: name = "qemu-%d" % os.getpid() + if sock_dir is None: + sock_dir = test_dir self._name = name self._monitor_address = monitor_address self._vm_monitor = None @@ -106,12 +108,14 @@ class QEMUMachine(object): self._qemu_full_args = None self._test_dir = test_dir self._temp_dir = None + self._sock_dir = sock_dir self._launched = False self._machine = None self._console_set = False self._console_device_type = None self._console_address = None self._console_socket = None + self._remove_files = [] # just in case logging wasn't configured by the main script: logging.basicConfig() @@ -236,8 +240,9 @@ class QEMUMachine(object): if self._machine is not None: args.extend(['-machine', self._machine]) if self._console_set: - self._console_address = os.path.join(self._temp_dir, + self._console_address = os.path.join(self._sock_dir, self._name + "-console.sock") + self._remove_files.append(self._console_address) chardev = ('socket,id=console,path=%s,server,nowait' % self._console_address) args.extend(['-chardev', chardev]) @@ -253,8 +258,9 @@ class QEMUMachine(object): if self._monitor_address is not None: self._vm_monitor = self._monitor_address else: - self._vm_monitor = os.path.join(self._temp_dir, + self._vm_monitor = os.path.join(self._sock_dir, self._name + "-monitor.sock") + self._remove_files.append(self._vm_monitor) self._qemu_log_path = os.path.join(self._temp_dir, self._name + ".log") self._qemu_log_file = open(self._qemu_log_path, 'wb') @@ -279,6 +285,9 @@ class QEMUMachine(object): shutil.rmtree(self._temp_dir) self._temp_dir = None + while len(self._remove_files) > 0: + self._remove_if_exists(self._remove_files.pop()) + def launch(self): """ Launch the VM and make sure we cleanup and expose the diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index 3f1d2cb325..d24ad04256 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -84,14 +84,17 @@ class QEMUQtestMachine(QEMUMachine): '''A QEMU VM''' def __init__(self, binary, args=None, name=None, test_dir="/var/tmp", - socket_scm_helper=None): + socket_scm_helper=None, sock_dir=None): if name is None: name = "qemu-%d" % os.getpid() + if sock_dir is None: + sock_dir = test_dir super(QEMUQtestMachine, self).__init__(binary, args, name=name, test_dir=test_dir, - socket_scm_helper=socket_scm_helper) + socket_scm_helper=socket_scm_helper, + sock_dir=sock_dir) self._qtest = None - self._qtest_path = os.path.join(test_dir, name + "-qtest.sock") + self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") def _base_args(self): args = super(QEMUQtestMachine, self)._base_args() diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 3a8f378f90..49cd205a97 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -57,6 +57,7 @@ qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') test_dir = os.environ.get('TEST_DIR') +sock_dir = os.environ.get('SOCK_DIR') output_dir = os.environ.get('OUTPUT_DIR', '.') cachemode = os.environ.get('CACHEMODE') qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE') @@ -445,7 +446,8 @@ class VM(qtest.QEMUQtestMachine): name = "qemu%s-%d" % (path_suffix, os.getpid()) super(VM, self).__init__(qemu_prog, qemu_opts, name=name, test_dir=test_dir, - socket_scm_helper=socket_scm_helper) + socket_scm_helper=socket_scm_helper, + sock_dir=sock_dir) self._num_drives = 0 def add_object(self, opts): -- 2.21.0
Specifying this optional parameter allows creating temporary files in other directories than the test_dir; for example in sock_dir. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/iotests.py | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 49cd205a97..5373149ae1 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -375,10 +375,10 @@ class FilePaths(object): qemu_img('create', img_path, '1G') # migration_sock_path is automatically deleted """ - def __init__(self, names): + def __init__(self, names, base_dir=test_dir): self.paths = [] for name in names: - self.paths.append(os.path.join(test_dir, file_pattern(name))) + self.paths.append(os.path.join(base_dir, file_pattern(name))) def __enter__(self): return self.paths @@ -395,8 +395,8 @@ class FilePath(FilePaths): """ FilePath is a specialization of FilePaths that takes a single filename. """ - def __init__(self, name): - super(FilePath, self).__init__([name]) + def __init__(self, name, base_dir=test_dir): + super(FilePath, self).__init__([name], base_dir) def __enter__(self): return self.paths[0] @@ -409,7 +409,7 @@ def file_path_remover(): pass -def file_path(*names): +def file_path(*names, base_dir=test_dir): ''' Another way to get auto-generated filename that cleans itself up. Use is as simple as: @@ -425,7 +425,7 @@ def file_path(*names): paths = [] for name in names: filename = file_pattern(name) - path = os.path.join(test_dir, filename) + path = os.path.join(base_dir, filename) file_path_remover.paths.append(path) paths.append(path) -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/common.filter | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 9f418b4881..cd42f5e7e3 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -43,7 +43,8 @@ _filter_qom_path() # replace occurrences of the actual TEST_DIR value with TEST_DIR _filter_testdir() { - $SED -e "s#$TEST_DIR/#TEST_DIR/#g" + $SED -e "s#$TEST_DIR/#TEST_DIR/#g" \ + -e "s#$SOCK_DIR/#SOCK_DIR/#g" } # replace occurrences of the actual IMGFMT value with IMGFMT @@ -124,6 +125,7 @@ _filter_img_create() $SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ + -e "s#$SOCK_DIR#SOCK_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd:127.0.0.1:10810#TEST_DIR/t.IMGFMT#g' \ -e "s# encryption=off##g" \ @@ -160,6 +162,7 @@ _filter_img_info() $SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ + -e "s#$SOCK_DIR#SOCK_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \ -e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \ @@ -218,7 +221,8 @@ _filter_nbd() # Filter out the TCP port number since this changes between runs. $SED -e '/nbd\/.*\.c:/d' \ -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ - -e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \ + -e "s#?socket=$SOCK_DIR#?socket=TEST_DIR#g" \ + -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \ -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' } -- 2.21.0
In addition, drop the nbd_unix_socket assignment in 241 because it does not really do anything. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/241 | 2 -- tests/qemu-iotests/common.nbd | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/tests/qemu-iotests/241 b/tests/qemu-iotests/241 index 58b64ebf41..8dae8d39e4 100755 --- a/tests/qemu-iotests/241 +++ b/tests/qemu-iotests/241 @@ -23,8 +23,6 @@ echo "QA output created by $seq" status=1 # failure is the default! -nbd_unix_socket=$TEST_DIR/test_qemu_nbd_socket - _cleanup() { _cleanup_test_img diff --git a/tests/qemu-iotests/common.nbd b/tests/qemu-iotests/common.nbd index 24b01b60aa..a8cae8fe2c 100644 --- a/tests/qemu-iotests/common.nbd +++ b/tests/qemu-iotests/common.nbd @@ -19,7 +19,7 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # -nbd_unix_socket="${TEST_DIR}/qemu-nbd.sock" +nbd_unix_socket="${SOCK_DIR}/qemu-nbd.sock" nbd_tcp_addr="127.0.0.1" nbd_pid_file="${TEST_DIR}/qemu-nbd.pid" nbd_stderr_fifo="${TEST_DIR}/qemu-nbd.fifo" -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/083 | 6 +++--- tests/qemu-iotests/083.out | 34 +++++++++++++++++----------------- 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/tests/qemu-iotests/083 b/tests/qemu-iotests/083 index b270550d3e..10fdfc8ebb 100755 --- a/tests/qemu-iotests/083 +++ b/tests/qemu-iotests/083 @@ -28,7 +28,7 @@ status=1 # failure is the default! _cleanup() { - rm -f nbd.sock + rm -f "$SOCK_DIR/nbd.sock" rm -f nbd-fault-injector.out rm -f nbd-fault-injector.conf } @@ -80,10 +80,10 @@ EOF if [ "$proto" = "tcp" ]; then nbd_addr="127.0.0.1:0" else - nbd_addr="$TEST_DIR/nbd.sock" + nbd_addr="$SOCK_DIR/nbd.sock" fi - rm -f "$TEST_DIR/nbd.sock" + rm -f "$SOCK_DIR/nbd.sock" echo > "$TEST_DIR/nbd-fault-injector.out" $PYTHON nbd-fault-injector.py $extra_args "$nbd_addr" "$TEST_DIR/nbd-fault-injector.conf" >"$TEST_DIR/nbd-fault-injector.out" 2>&1 & diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out index eee6dd1379..2090ee693c 100644 --- a/tests/qemu-iotests/083.out +++ b/tests/qemu-iotests/083.out @@ -110,43 +110,43 @@ read failed: Input/output error === Check disconnect before neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect after neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 8 neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 16 neg1 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect before export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect after export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 4 export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 12 export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 16 export === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect before neg2 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect after neg2 === @@ -154,11 +154,11 @@ read failed: Input/output error === Check disconnect 8 neg2 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect 10 neg2 === -qemu-io: can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///foo?socket=SOCK_DIR/nbd.sock === Check disconnect before request === @@ -195,23 +195,23 @@ read 512/512 bytes at offset 0 === Check disconnect before neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 8 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 16 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 24 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect 28 neg-classic === -qemu-io: can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock +qemu-io: can't open device nbd+unix:///?socket=SOCK_DIR/nbd.sock === Check disconnect after neg-classic === -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/140 | 8 ++++---- tests/qemu-iotests/140.out | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index b965b1dd5d..8d2ce5d9e3 100755 --- a/tests/qemu-iotests/140 +++ b/tests/qemu-iotests/140 @@ -34,7 +34,7 @@ _cleanup() { _cleanup_qemu _cleanup_test_img - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -69,7 +69,7 @@ _send_qemu_cmd $QEMU_HANDLE \ _send_qemu_cmd $QEMU_HANDLE \ "{ 'execute': 'nbd-server-start', 'arguments': { 'addr': { 'type': 'unix', - 'data': { 'path': '$TEST_DIR/nbd' }}}}" \ + 'data': { 'path': '$SOCK_DIR/nbd' }}}}" \ 'return' _send_qemu_cmd $QEMU_HANDLE \ @@ -78,7 +78,7 @@ _send_qemu_cmd $QEMU_HANDLE \ 'return' $QEMU_IO_PROG -f raw -r -c 'read -P 42 0 64k' \ - "nbd+unix:///drv?socket=$TEST_DIR/nbd" 2>&1 \ + "nbd+unix:///drv?socket=$SOCK_DIR/nbd" 2>&1 \ | _filter_qemu_io | _filter_nbd _send_qemu_cmd $QEMU_HANDLE \ @@ -87,7 +87,7 @@ _send_qemu_cmd $QEMU_HANDLE \ 'return' $QEMU_IO_PROG -f raw -r -c close \ - "nbd+unix:///drv?socket=$TEST_DIR/nbd" 2>&1 \ + "nbd+unix:///drv?socket=$SOCK_DIR/nbd" 2>&1 \ | _filter_qemu_io | _filter_nbd _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/140.out b/tests/qemu-iotests/140.out index 67fe44a3e3..2511eb7369 100644 --- a/tests/qemu-iotests/140.out +++ b/tests/qemu-iotests/140.out @@ -8,7 +8,7 @@ wrote 65536/65536 bytes at offset 0 read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": {}} -qemu-io: can't open device nbd+unix:///drv?socket=TEST_DIR/nbd: Requested export not available +qemu-io: can't open device nbd+unix:///drv?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'drv' not present {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/143 | 6 +++--- tests/qemu-iotests/143.out | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/143 b/tests/qemu-iotests/143 index 92249ac8da..f649b36195 100755 --- a/tests/qemu-iotests/143 +++ b/tests/qemu-iotests/143 @@ -29,7 +29,7 @@ status=1 # failure is the default! _cleanup() { _cleanup_qemu - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -51,12 +51,12 @@ _send_qemu_cmd $QEMU_HANDLE \ _send_qemu_cmd $QEMU_HANDLE \ "{ 'execute': 'nbd-server-start', 'arguments': { 'addr': { 'type': 'unix', - 'data': { 'path': '$TEST_DIR/nbd' }}}}" \ + 'data': { 'path': '$SOCK_DIR/nbd' }}}}" \ 'return' # This should just result in a client error, not in the server crashing $QEMU_IO_PROG -f raw -c quit \ - "nbd+unix:///no_such_export?socket=$TEST_DIR/nbd" 2>&1 \ + "nbd+unix:///no_such_export?socket=$SOCK_DIR/nbd" 2>&1 \ | _filter_qemu_io | _filter_nbd _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/143.out b/tests/qemu-iotests/143.out index ee71b5aa42..037d34a409 100644 --- a/tests/qemu-iotests/143.out +++ b/tests/qemu-iotests/143.out @@ -1,7 +1,7 @@ QA output created by 143 {"return": {}} {"return": {}} -qemu-io: can't open device nbd+unix:///no_such_export?socket=TEST_DIR/nbd: Requested export not available +qemu-io: can't open device nbd+unix:///no_such_export?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'no_such_export' not present {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/147 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 index ab8480b9a4..03fc2fabcf 100755 --- a/tests/qemu-iotests/147 +++ b/tests/qemu-iotests/147 @@ -32,7 +32,7 @@ NBD_IPV6_PORT_START = NBD_PORT_END NBD_IPV6_PORT_END = NBD_IPV6_PORT_START + 1024 test_img = os.path.join(iotests.test_dir, 'test.img') -unix_socket = os.path.join(iotests.test_dir, 'nbd.socket') +unix_socket = os.path.join(iotests.sock_dir, 'nbd.socket') def flatten_sock_addr(crumpled_address): -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/181 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 index e317e63422..378c2899d1 100755 --- a/tests/qemu-iotests/181 +++ b/tests/qemu-iotests/181 @@ -26,7 +26,7 @@ echo "QA output created by $seq" status=1 # failure is the default! -MIG_SOCKET="${TEST_DIR}/migrate" +MIG_SOCKET="${SOCK_DIR}/migrate" _cleanup() { -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/182 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182 index 7f494eb9bb..1ccb850055 100755 --- a/tests/qemu-iotests/182 +++ b/tests/qemu-iotests/182 @@ -31,7 +31,7 @@ _cleanup() { _cleanup_test_img rm -f "$TEST_IMG.overlay" - rm -f "$TEST_DIR/nbd.socket" + rm -f "$SOCK_DIR/nbd.socket" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -133,7 +133,7 @@ success_or_failure=y _send_qemu_cmd $QEMU_HANDLE \ 'addr': { 'type': 'unix', 'data': { - 'path': '$TEST_DIR/nbd.socket' + 'path': '$SOCK_DIR/nbd.socket' } } } }" \ 'return' \ 'error' -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/183 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183 index 04fb344d08..bced83fae0 100755 --- a/tests/qemu-iotests/183 +++ b/tests/qemu-iotests/183 @@ -26,7 +26,7 @@ echo "QA output created by $seq" status=1 # failure is the default! -MIG_SOCKET="${TEST_DIR}/migrate" +MIG_SOCKET="${SOCK_DIR}/migrate" _cleanup() { -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/192 | 4 ++-- tests/qemu-iotests/192.out | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192 index 034432272f..d2ba55dd90 100755 --- a/tests/qemu-iotests/192 +++ b/tests/qemu-iotests/192 @@ -31,7 +31,7 @@ _cleanup() { _cleanup_qemu _cleanup_test_img - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -66,7 +66,7 @@ else QEMU_COMM_TIMEOUT=1 fi -_send_qemu_cmd $h "nbd_server_start unix:$TEST_DIR/nbd" "(qemu)" +_send_qemu_cmd $h "nbd_server_start unix:$SOCK_DIR/nbd" "(qemu)" _send_qemu_cmd $h "nbd_server_add -w drive0" "(qemu)" _send_qemu_cmd $h "q" "(qemu)" diff --git a/tests/qemu-iotests/192.out b/tests/qemu-iotests/192.out index 1e0be4c4d7..b9429dbe36 100644 --- a/tests/qemu-iotests/192.out +++ b/tests/qemu-iotests/192.out @@ -1,7 +1,7 @@ QA output created by 192 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 QEMU X.Y.Z monitor - type 'help' for more information -(qemu) nbd_server_start unix:TEST_DIR/nbd +(qemu) nbd_server_start unix:SOCK_DIR/nbd (qemu) nbd_server_add -w drive0 (qemu) q *** done -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/194 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 index d746ab1e21..72e47e8833 100755 --- a/tests/qemu-iotests/194 +++ b/tests/qemu-iotests/194 @@ -26,8 +26,8 @@ iotests.verify_platform(['linux']) with iotests.FilePath('source.img') as source_img_path, \ iotests.FilePath('dest.img') as dest_img_path, \ - iotests.FilePath('migration.sock') as migration_sock_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ + iotests.FilePaths(['migration.sock', 'nbd.sock'], iotests.sock_dir) as \ + [migration_sock_path, nbd_sock_path], \ iotests.VM('source') as source_vm, \ iotests.VM('dest') as dest_vm: -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/201 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/201 b/tests/qemu-iotests/201 index 7abf740fe4..86fa37e714 100755 --- a/tests/qemu-iotests/201 +++ b/tests/qemu-iotests/201 @@ -24,7 +24,7 @@ echo "QA output created by $seq" status=1 # failure is the default! -MIG_SOCKET="${TEST_DIR}/migrate" +MIG_SOCKET="${SOCK_DIR}/migrate" # get standard environment, filters and checks . ./common.rc -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/205 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205 index 76f6c5fa2b..4bb2c21e8b 100755 --- a/tests/qemu-iotests/205 +++ b/tests/qemu-iotests/205 @@ -24,7 +24,7 @@ import iotests import time from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive -nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock') +nbd_sock = os.path.join(iotests.sock_dir, 'nbd_sock') nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock disk = os.path.join(iotests.test_dir, 'disk') -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/208 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208 index 1e202388dc..546eb1de3e 100755 --- a/tests/qemu-iotests/208 +++ b/tests/qemu-iotests/208 @@ -26,7 +26,7 @@ iotests.verify_image_format(supported_fmts=['generic']) with iotests.FilePath('disk.img') as disk_img_path, \ iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ + iotests.FilePath('nbd.sock', iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: img_size = '10M' -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/209 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209 index 259e991ec6..e0f464bcbe 100755 --- a/tests/qemu-iotests/209 +++ b/tests/qemu-iotests/209 @@ -24,7 +24,8 @@ from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \ iotests.verify_image_format(supported_fmts=['qcow2']) -disk, nbd_sock = file_path('disk', 'nbd-sock') +disk = file_path('disk') +nbd_sock = file_path('nbd-sock', base_dir=iotests.sock_dir) nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock qemu_img_create('-f', iotests.imgfmt, disk, '1M') -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/222 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index 0ead56d574..3f9f934ad8 100644 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -48,7 +48,7 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock') as nbd_sock_path, \ + iotests.FilePath('nbd.sock', iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: log('--- Setting up images ---') -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/223 | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223 index 2ba3d8124b..b5a80e50bb 100755 --- a/tests/qemu-iotests/223 +++ b/tests/qemu-iotests/223 @@ -28,7 +28,7 @@ _cleanup() nbd_server_stop _cleanup_test_img _cleanup_qemu - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -125,11 +125,11 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", "arguments":{"device":"n"}}' "error" # Attempt add without server _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start", "arguments":{"addr":{"type":"unix", - "data":{"path":"'"$TEST_DIR/nbd"'"}}}}' "return" + "data":{"path":"'"$SOCK_DIR/nbd"'"}}}}' "return" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start", "arguments":{"addr":{"type":"unix", - "data":{"path":"'"$TEST_DIR/nbd"1'"}}}}' "error" # Attempt second server -$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd" + "data":{"path":"'"$SOCK_DIR/nbd"1'"}}}}' "error" # Attempt second server +$QEMU_NBD_PROG -L -k "$SOCK_DIR/nbd" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", "arguments":{"device":"n", "bitmap":"b"}}' "return" _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", @@ -145,14 +145,14 @@ _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", _send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", "arguments":{"device":"n", "name":"n2", "writable":true, "bitmap":"b2"}}' "return" -$QEMU_NBD_PROG -L -k "$TEST_DIR/nbd" +$QEMU_NBD_PROG -L -k "$SOCK_DIR/nbd" echo echo "=== Contrast normal status to large granularity dirty-bitmap ===" echo QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT -IMG="driver=nbd,export=n,server.type=unix,server.path=$TEST_DIR/nbd" +IMG="driver=nbd,export=n,server.type=unix,server.path=$SOCK_DIR/nbd" $QEMU_IO -r -c 'r -P 0x22 512 512' -c 'r -P 0 512k 512k' -c 'r -P 0x11 1m 1m' \ -c 'r -P 0x33 2m 2m' --image-opts "$IMG" | _filter_qemu_io $QEMU_IMG map --output=json --image-opts \ @@ -164,7 +164,7 @@ echo echo "=== Contrast to small granularity dirty-bitmap ===" echo -IMG="driver=nbd,export=n2,server.type=unix,server.path=$TEST_DIR/nbd" +IMG="driver=nbd,export=n2,server.type=unix,server.path=$SOCK_DIR/nbd" $QEMU_IMG map --output=json --image-opts \ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/240 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240 index f73bc07d80..8b4337b58d 100755 --- a/tests/qemu-iotests/240 +++ b/tests/qemu-iotests/240 @@ -29,7 +29,7 @@ status=1 # failure is the default! _cleanup() { - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -135,7 +135,7 @@ echo run_qemu <<EOF { "execute": "qmp_capabilities" } { "execute": "blockdev-add", "arguments": {"driver": "null-co", "read-zeroes": true, "node-name": "hd0", "read-only": true}} -{ "execute": "nbd-server-start", "arguments": {"addr":{"type":"unix","data":{"path":"$TEST_DIR/nbd"}}}} +{ "execute": "nbd-server-start", "arguments": {"addr":{"type":"unix","data":{"path":"$SOCK_DIR/nbd"}}}} { "execute": "nbd-server-add", "arguments": {"device":"hd0"}} { "execute": "object-add", "arguments": {"qom-type": "iothread", "id": "iothread0"}} { "execute": "device_add", "arguments": {"id": "scsi0", "driver": "${virtio_scsi}", "iothread": "iothread0"}} -- 2.21.0
Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/267 | 4 ++-- tests/qemu-iotests/267.out | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/267 b/tests/qemu-iotests/267 index d37a67c012..170e173c0a 100755 --- a/tests/qemu-iotests/267 +++ b/tests/qemu-iotests/267 @@ -29,7 +29,7 @@ status=1 # failure is the default! _cleanup() { _cleanup_test_img - rm -f "$TEST_DIR/nbd" + rm -f "$SOCK_DIR/nbd" } trap "_cleanup; exit \$status" 0 1 2 3 15 @@ -143,7 +143,7 @@ echo IMGOPTS="backing_file=$TEST_IMG.base" _make_test_img $size cat <<EOF | -nbd_server_start unix:$TEST_DIR/nbd +nbd_server_start unix:$SOCK_DIR/nbd nbd_server_add -w backing-fmt savevm snap0 info snapshots diff --git a/tests/qemu-iotests/267.out b/tests/qemu-iotests/267.out index 9d812e3c72..8dddb4baa4 100644 --- a/tests/qemu-iotests/267.out +++ b/tests/qemu-iotests/267.out @@ -161,7 +161,7 @@ Internal snapshots on backing file: Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base Testing: -blockdev driver=file,filename=TEST_DIR/t.IMGFMT.base,node-name=backing-file -blockdev driver=IMGFMT,file=backing-file,node-name=backing-fmt -blockdev driver=file,filename=TEST_DIR/t.IMGFMT,node-name=file -blockdev driver=IMGFMT,file=file,backing=backing-fmt,node-name=fmt QEMU X.Y.Z monitor - type 'help' for more information -(qemu) nbd_server_start unix:TEST_DIR/nbd +(qemu) nbd_server_start unix:SOCK_DIR/nbd (qemu) nbd_server_add -w backing-fmt (qemu) savevm snap0 (qemu) info snapshots -- 2.21.0
Sockets should be placed into $SOCK_DIR instead of $TEST_DIR, so remove the $TEST_DIR filter from _filter_nbd. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/common.filter | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index cd42f5e7e3..f870e00e44 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -221,7 +221,6 @@ _filter_nbd() # Filter out the TCP port number since this changes between runs. $SED -e '/nbd\/.*\.c:/d' \ -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ - -e "s#?socket=$SOCK_DIR#?socket=TEST_DIR#g" \ -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \ -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' } -- 2.21.0
On 10/10/19 10:24 AM, Max Reitz wrote: > Unix sockets generally have a maximum path length. Depending on your > $TEST_DIR, it may be exceeded and then all tests that create and use > Unix sockets there may fail. > > Circumvent this by adding a new scratch directory specifically for > Unix socket files. It defaults to a temporary directory (mktemp -d) > that is completely removed after the iotests are done. > > (By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which > should be short enough for our use cases.) > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/check | 17 +++++++++++++++++ > +tmp_sock_dir=false > +if [ -z "$SOCK_DIR" ]; then > + SOCK_DIR=$(mktemp -d) > + tmp_sock_dir=true > +fi > + > +if [ ! -d "$SOCK_DIR" ]; then > + mkdir "$SOCK_DIR" > +fi Should this use mkdir -p, in case two parallel processes compete with the same SOCK_DIR? What if SOCK_DIR is set to something that is not a directory (say a file), at which point mkdir fails, but you don't seem to be catching that failure. Otherwise looks good. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > iotests.py itself does not store socket files, but it machine.py and s/it // > qtest.py do. iotests.py needs to pass the respective path to them, and > they need to adhere to it. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > python/qemu/machine.py | 15 ++++++++++++--- > python/qemu/qtest.py | 9 ++++++--- > tests/qemu-iotests/iotests.py | 4 +++- > 3 files changed, 21 insertions(+), 7 deletions(-) > Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Specifying this optional parameter allows creating temporary files in
> other directories than the test_dir; for example in sock_dir.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/iotests.py | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/common.filter | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
> index 9f418b4881..cd42f5e7e3 100644
> --- a/tests/qemu-iotests/common.filter
> +++ b/tests/qemu-iotests/common.filter
> @@ -43,7 +43,8 @@ _filter_qom_path()
> # replace occurrences of the actual TEST_DIR value with TEST_DIR
> _filter_testdir()
> {
> - $SED -e "s#$TEST_DIR/#TEST_DIR/#g"
> + $SED -e "s#$TEST_DIR/#TEST_DIR/#g" \
> + -e "s#$SOCK_DIR/#SOCK_DIR/#g"
Do we want to output a literal 'SOCK_DIR' (every test that uses it has
to update their expected output), or can we make this also output a
literal 'TEST_DIR' (output is a bit more confusing on which dir to look
in, but fewer files to touch)? Your preference.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > In addition, drop the nbd_unix_socket assignment in 241 because it does > not really do anything. Agreed, it duplicates the setting inherited by sourcing common.nbd. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/241 | 2 -- > tests/qemu-iotests/common.nbd | 2 +- > 2 files changed, 1 insertion(+), 3 deletions(-) > Reviewed-by: Eric Blake <eblake@redhat.com> I'm assuming the series goes better through your iotests tree than my NBD tree. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/083 | 6 +++---
> tests/qemu-iotests/083.out | 34 +++++++++++++++++-----------------
> 2 files changed, 20 insertions(+), 20 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/140 | 8 ++++----
> tests/qemu-iotests/140.out | 2 +-
> 2 files changed, 5 insertions(+), 5 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/143 | 6 +++---
> tests/qemu-iotests/143.out | 2 +-
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/147 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 > index ab8480b9a4..03fc2fabcf 100755 > --- a/tests/qemu-iotests/147 > +++ b/tests/qemu-iotests/147 > @@ -32,7 +32,7 @@ NBD_IPV6_PORT_START = NBD_PORT_END > NBD_IPV6_PORT_END = NBD_IPV6_PORT_START + 1024 > > test_img = os.path.join(iotests.test_dir, 'test.img') > -unix_socket = os.path.join(iotests.test_dir, 'nbd.socket') > +unix_socket = os.path.join(iotests.sock_dir, 'nbd.socket') > > > def flatten_sock_addr(crumpled_address): > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/181 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/181 b/tests/qemu-iotests/181 > index e317e63422..378c2899d1 100755 > --- a/tests/qemu-iotests/181 > +++ b/tests/qemu-iotests/181 > @@ -26,7 +26,7 @@ echo "QA output created by $seq" > > status=1 # failure is the default! > > -MIG_SOCKET="${TEST_DIR}/migrate" > +MIG_SOCKET="${SOCK_DIR}/migrate" > > _cleanup() > { > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/182 | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/182 b/tests/qemu-iotests/182 > index 7f494eb9bb..1ccb850055 100755 > --- a/tests/qemu-iotests/182 > +++ b/tests/qemu-iotests/182 > @@ -31,7 +31,7 @@ _cleanup() > { > _cleanup_test_img > rm -f "$TEST_IMG.overlay" > - rm -f "$TEST_DIR/nbd.socket" > + rm -f "$SOCK_DIR/nbd.socket" > } > trap "_cleanup; exit \$status" 0 1 2 3 15 > > @@ -133,7 +133,7 @@ success_or_failure=y _send_qemu_cmd $QEMU_HANDLE \ > 'addr': { > 'type': 'unix', > 'data': { > - 'path': '$TEST_DIR/nbd.socket' > + 'path': '$SOCK_DIR/nbd.socket' > } } } }" \ > 'return' \ > 'error' > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/183 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183 > index 04fb344d08..bced83fae0 100755 > --- a/tests/qemu-iotests/183 > +++ b/tests/qemu-iotests/183 > @@ -26,7 +26,7 @@ echo "QA output created by $seq" > > status=1 # failure is the default! > > -MIG_SOCKET="${TEST_DIR}/migrate" > +MIG_SOCKET="${SOCK_DIR}/migrate" > > _cleanup() > { > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/192 | 4 ++--
> tests/qemu-iotests/192.out | 2 +-
> 2 files changed, 3 insertions(+), 3 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/194 | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 > index d746ab1e21..72e47e8833 100755 > --- a/tests/qemu-iotests/194 > +++ b/tests/qemu-iotests/194 > @@ -26,8 +26,8 @@ iotests.verify_platform(['linux']) > > with iotests.FilePath('source.img') as source_img_path, \ > iotests.FilePath('dest.img') as dest_img_path, \ > - iotests.FilePath('migration.sock') as migration_sock_path, \ > - iotests.FilePath('nbd.sock') as nbd_sock_path, \ > + iotests.FilePaths(['migration.sock', 'nbd.sock'], iotests.sock_dir) as \ > + [migration_sock_path, nbd_sock_path], \ > iotests.VM('source') as source_vm, \ > iotests.VM('dest') as dest_vm: > > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/201 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/201 b/tests/qemu-iotests/201 > index 7abf740fe4..86fa37e714 100755 > --- a/tests/qemu-iotests/201 > +++ b/tests/qemu-iotests/201 > @@ -24,7 +24,7 @@ echo "QA output created by $seq" > > status=1 # failure is the default! > > -MIG_SOCKET="${TEST_DIR}/migrate" > +MIG_SOCKET="${SOCK_DIR}/migrate" > > # get standard environment, filters and checks > . ./common.rc > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/205 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205 > index 76f6c5fa2b..4bb2c21e8b 100755 > --- a/tests/qemu-iotests/205 > +++ b/tests/qemu-iotests/205 > @@ -24,7 +24,7 @@ import iotests > import time > from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive > > -nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock') > +nbd_sock = os.path.join(iotests.sock_dir, 'nbd_sock') > nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock > disk = os.path.join(iotests.test_dir, 'disk') > > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/208 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208 > index 1e202388dc..546eb1de3e 100755 > --- a/tests/qemu-iotests/208 > +++ b/tests/qemu-iotests/208 > @@ -26,7 +26,7 @@ iotests.verify_image_format(supported_fmts=['generic']) > > with iotests.FilePath('disk.img') as disk_img_path, \ > iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \ > - iotests.FilePath('nbd.sock') as nbd_sock_path, \ > + iotests.FilePath('nbd.sock', iotests.sock_dir) as nbd_sock_path, \ > iotests.VM() as vm: > > img_size = '10M' > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/194 | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 > index d746ab1e21..72e47e8833 100755 > --- a/tests/qemu-iotests/194 > +++ b/tests/qemu-iotests/194 > @@ -26,8 +26,8 @@ iotests.verify_platform(['linux']) > > with iotests.FilePath('source.img') as source_img_path, \ > iotests.FilePath('dest.img') as dest_img_path, \ Any reason these are still two iotests.FilePath(), > - iotests.FilePath('migration.sock') as migration_sock_path, \ > - iotests.FilePath('nbd.sock') as nbd_sock_path, \ > + iotests.FilePaths(['migration.sock', 'nbd.sock'], iotests.sock_dir) as \ > + [migration_sock_path, nbd_sock_path], \ while you joined this into one iotests.FilePaths()? Doesn't affect correctness, but does raise a consistency issue (I noticed it again in the untouched part of patch 17). > iotests.VM('source') as source_vm, \ > iotests.VM('dest') as dest_vm: > > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/209 | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) Reviewed-by: Eric Blake <eblake@redhat.com> > > diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209 > index 259e991ec6..e0f464bcbe 100755 > --- a/tests/qemu-iotests/209 > +++ b/tests/qemu-iotests/209 > @@ -24,7 +24,8 @@ from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \ > > iotests.verify_image_format(supported_fmts=['qcow2']) > > -disk, nbd_sock = file_path('disk', 'nbd-sock') > +disk = file_path('disk') > +nbd_sock = file_path('nbd-sock', base_dir=iotests.sock_dir) > nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock > > qemu_img_create('-f', iotests.imgfmt, disk, '1M') > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/222 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Reviewed-by: Eric Blake <eblake@redhat.com> > diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 > index 0ead56d574..3f9f934ad8 100644 > --- a/tests/qemu-iotests/222 > +++ b/tests/qemu-iotests/222 > @@ -48,7 +48,7 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] > > with iotests.FilePath('base.img') as base_img_path, \ > iotests.FilePath('fleece.img') as fleece_img_path, \ > - iotests.FilePath('nbd.sock') as nbd_sock_path, \ > + iotests.FilePath('nbd.sock', iotests.sock_dir) as nbd_sock_path, \ > iotests.VM() as vm: > > log('--- Setting up images ---') > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/223 | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/240 | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Reviewed-by: Eric Blake <eblake@redhat.com> > > diff --git a/tests/qemu-iotests/240 b/tests/qemu-iotests/240 > index f73bc07d80..8b4337b58d 100755 > --- a/tests/qemu-iotests/240 > +++ b/tests/qemu-iotests/240 > @@ -29,7 +29,7 @@ status=1 # failure is the default! > > _cleanup() > { > - rm -f "$TEST_DIR/nbd" > + rm -f "$SOCK_DIR/nbd" > } > trap "_cleanup; exit \$status" 0 1 2 3 15 > > @@ -135,7 +135,7 @@ echo > run_qemu <<EOF > { "execute": "qmp_capabilities" } > { "execute": "blockdev-add", "arguments": {"driver": "null-co", "read-zeroes": true, "node-name": "hd0", "read-only": true}} > -{ "execute": "nbd-server-start", "arguments": {"addr":{"type":"unix","data":{"path":"$TEST_DIR/nbd"}}}} > +{ "execute": "nbd-server-start", "arguments": {"addr":{"type":"unix","data":{"path":"$SOCK_DIR/nbd"}}}} > { "execute": "nbd-server-add", "arguments": {"device":"hd0"}} > { "execute": "object-add", "arguments": {"qom-type": "iothread", "id": "iothread0"}} > { "execute": "device_add", "arguments": {"id": "scsi0", "driver": "${virtio_scsi}", "iothread": "iothread0"}} > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/267 | 4 ++--
> tests/qemu-iotests/267.out | 2 +-
> 2 files changed, 3 insertions(+), 3 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote: > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/common.filter | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > @@ -218,7 +221,8 @@ _filter_nbd() > # Filter out the TCP port number since this changes between runs. > $SED -e '/nbd\/.*\.c:/d' \ > -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ > - -e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \ > + -e "s#?socket=$SOCK_DIR#?socket=TEST_DIR#g" \ > + -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \ > -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' This goes away in 23, but this looks crazy. Don't you really want the first line to replace $TEST_DIR with TEST_DIR (not $SOCK_DIR with TEST_DIR)? Otherwise, bisection is likely to break until all the intermediate patches have made the conversion to stop using TEST_DIR. I already gave R-b, but you'll need to fix this one. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
On 10/10/19 10:24 AM, Max Reitz wrote:
> Sockets should be placed into $SOCK_DIR instead of $TEST_DIR, so remove
> the $TEST_DIR filter from _filter_nbd.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/common.filter | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
> index cd42f5e7e3..f870e00e44 100644
> --- a/tests/qemu-iotests/common.filter
> +++ b/tests/qemu-iotests/common.filter
> @@ -221,7 +221,6 @@ _filter_nbd()
> # Filter out the TCP port number since this changes between runs.
> $SED -e '/nbd\/.*\.c:/d' \
> -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \
> - -e "s#?socket=$SOCK_DIR#?socket=TEST_DIR#g" \
> -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \
Whoops - you have a bug in patch 4. Once that is fixed, then deleting
the line:
-e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \
here is appropriate. So with that amendment to the series,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
On 10/10/2019 17.24, Max Reitz wrote:
> iotests.py itself does not store socket files, but it machine.py and
> qtest.py do. iotests.py needs to pass the respective path to them, and
> they need to adhere to it.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> python/qemu/machine.py | 15 ++++++++++++---
> python/qemu/qtest.py | 9 ++++++---
> tests/qemu-iotests/iotests.py | 4 +++-
> 3 files changed, 21 insertions(+), 7 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Hi,
>
> Perhaps the main reason we cannot run important tests such as 041 in CI
> is that when they care Unix sockets in $TEST_DIR, the path may become
> too long to connect to them.
>
> To get by this problem, this series lets the check script create a new
> temporary directory (mktemp -d) and then makes the iotests use it for
> all Unix sockets.
Thanks a lot for tackling this!
I gave it a try, and most tests work fine now indeed when I run them in
a directory with a veeeery long file name.
I still get an error with 028 though:
$ ./check -qcow2 028
QEMU --
"/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64"
-nodefaults -display none -machine accel=qtest
QEMU_IMG --
"/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/../../qemu-img"
QEMU_IO --
"/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/../../qemu-io"
--cache writeback -f qcow2
QEMU_NBD --
"/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- qcow2 (compat=1.1)
IMGPROTO -- file
PLATFORM -- Linux/x86_64 thuth 4.18.0-80.11.2.el8_0.x86_64
TEST_DIR --
/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/scratch
SOCK_DIR -- /tmp/tmp.YEmubpCxRH
SOCKET_SCM_HELPER --
/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/socket_scm_helper
028 fail [09:11:52] [09:11:53] output
mismatch (see 028.out.bad)
--- /home/thuth/devel/qemu/tests/qemu-iotests/028.out 2019-08-16
18:00:39.258741027 +0200
+++
/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/028.out.bad
2019-10-11 09:11:53.822111901 +0200
@@ -468,7 +468,8 @@
block-backup
-Formatting 'TEST_DIR/t.IMGFMT.copy', fmt=IMGFMT size=4294968832
backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+(qemu)
+Timeout waiting for Formatting on handle 0
(qemu) info block-jobs
No active jobs
=== IO: pattern 195
Failures: 028
Failed 1 of 1 iotests
028 works fine if I start it from a directory with a short filename.
I also saw an error with "./check -raw 055" once, but it does not seem
to be reproducible:
055 fail [09:08:28] [09:10:57] output
mismatch (see 055.out.bad)
--- /home/thuth/devel/qemu/tests/qemu-iotests/055.out 2019-08-16
18:00:39.262741079 +0200
+++
/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/055.out.bad
2019-10-11 09:10:57.059254617 +0200
@@ -1,5 +1,35 @@
-..............................
+WARNING:qemu.machine:qemu received signal 9:
/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64
-chardev socket,id=mon,path=/tmp/tmp.3mKmQDU7Ft/qemu-24878-monitor.sock
-mon chardev=mon,mode=control -display none -vga none -qtest
unix:path=/tmp/tmp.3mKmQDU7Ft/qemu-24878-qtest.sock -accel qtest
-nodefaults -display none -machine accel=qtest -drive
if=virtio,id=drive0,file=blkdebug::/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/scratch/test.img,format=raw,cache=writeback
-drive
if=none,id=drive1,file=/home/thuth/tmp/qemu-with-very-long-directory-name-01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/tests/qemu-iotests/scratch/blockdev-target.img,format=vmdk,cache=writeback
+E.............................
+======================================================================
+ERROR: test_complete_compress_blockdev_backup
(__main__.TestDriveCompression)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../python/qemu/qmp.py",
line 122, in __get_events
+ ret = self.__json_read(only_event=True)
+ File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../python/qemu/qmp.py",
line 82, in __json_read
+ data = self.__sockfile.readline()
+ File "/usr/lib64/python3.6/socket.py", line 586, in readinto
+ return self._sock.recv_into(b)
+socket.timeout: timed out
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+ File "055", line 499, in test_complete_compress_blockdev_backup
+ target='drive1')
+ File "055", line 484, in do_test_compress_complete
+ self.wait_until_completed()
+ File "/home/thuth/devel/qemu/tests/qemu-iotests/iotests.py", line
751, in wait_until_completed
+ for event in self.vm.get_qmp_events(wait=wait):
+ File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../python/qemu/machine.py", line
406, in get_qmp_events
+ events = self._qmp.get_events(wait=wait)
+ File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../python/qemu/qmp.py",
line 236, in get_events
+ self.__get_events(wait)
+ File
"/home/thuth/devel/qemu/tests/qemu-iotests/../../python/qemu/qmp.py",
line 124, in __get_events
+ raise QMPTimeoutError("Timeout waiting for event")
+qemu.qmp.QMPTimeoutError: Timeout waiting for event
+
----------------------------------------------------------------------
Ran 30 tests
-OK
+FAILED (errors=1)
Thomas
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/083 | 6 +++---
> tests/qemu-iotests/083.out | 34 +++++++++++++++++-----------------
> 2 files changed, 20 insertions(+), 20 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/140 | 8 ++++----
> tests/qemu-iotests/140.out | 2 +-
> 2 files changed, 5 insertions(+), 5 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/143 | 6 +++---
> tests/qemu-iotests/143.out | 2 +-
> 2 files changed, 4 insertions(+), 4 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/147 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/181 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/182 | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
On 10/10/2019 17.24, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/183 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
[-- Attachment #1.1: Type: text/plain, Size: 1579 bytes --] On 10.10.19 20:18, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Unix sockets generally have a maximum path length. Depending on your >> $TEST_DIR, it may be exceeded and then all tests that create and use >> Unix sockets there may fail. >> >> Circumvent this by adding a new scratch directory specifically for >> Unix socket files. It defaults to a temporary directory (mktemp -d) >> that is completely removed after the iotests are done. >> >> (By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which >> should be short enough for our use cases.) >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/check | 17 +++++++++++++++++ > >> +tmp_sock_dir=false >> +if [ -z "$SOCK_DIR" ]; then >> + SOCK_DIR=$(mktemp -d) >> + tmp_sock_dir=true >> +fi >> + >> +if [ ! -d "$SOCK_DIR" ]; then >> + mkdir "$SOCK_DIR" >> +fi > > Should this use mkdir -p, in case two parallel processes compete with > the same SOCK_DIR? I would have used mkdir -p, but I saw we used this construct for TEST_DIR, so I thought I‘d just go for the same. > What if SOCK_DIR is set to something that is not a directory (say a > file), at which point mkdir fails, but you don't seem to be catching > that failure. Well, the same applies to TEST_DIR. And technically, as long as we don’t use mkdir -p for either, not catching the error at least helps circumvent the potential race. O:-) (I’ll convert both to mkdir -p with error handling.) Max > Otherwise looks good. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 1444 bytes --] On 10.10.19 20:42, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/common.filter | 8 ++++++-- >> 1 file changed, 6 insertions(+), 2 deletions(-) >> >> diff --git a/tests/qemu-iotests/common.filter >> b/tests/qemu-iotests/common.filter >> index 9f418b4881..cd42f5e7e3 100644 >> --- a/tests/qemu-iotests/common.filter >> +++ b/tests/qemu-iotests/common.filter >> @@ -43,7 +43,8 @@ _filter_qom_path() >> # replace occurrences of the actual TEST_DIR value with TEST_DIR >> _filter_testdir() >> { >> - $SED -e "s#$TEST_DIR/#TEST_DIR/#g" >> + $SED -e "s#$TEST_DIR/#TEST_DIR/#g" \ >> + -e "s#$SOCK_DIR/#SOCK_DIR/#g" > > Do we want to output a literal 'SOCK_DIR' (every test that uses it has > to update their expected output), or can we make this also output a > literal 'TEST_DIR' (output is a bit more confusing on which dir to look > in, but fewer files to touch)? Your preference. There’s another advantage to filtering it to be TEST_DIR, and that’s the fact that if $TEST_DIR and $SOCK_DIR are the same, we will always replace $SOCK_DIR by TEST_DIR. But I still preferred filtering it to be SOCK_DIR, because that seemed to me like we would have done it had we had a SOCK_DIR from the start. > Reviewed-by: Eric Blake <eblake@redhat.com> Thanks for reviewing! Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 1550 bytes --] On 10.10.19 21:43, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/194 | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194 >> index d746ab1e21..72e47e8833 100755 >> --- a/tests/qemu-iotests/194 >> +++ b/tests/qemu-iotests/194 >> @@ -26,8 +26,8 @@ iotests.verify_platform(['linux']) >> with iotests.FilePath('source.img') as source_img_path, \ >> iotests.FilePath('dest.img') as dest_img_path, \ > > Any reason these are still two iotests.FilePath(), > >> - iotests.FilePath('migration.sock') as migration_sock_path, \ >> - iotests.FilePath('nbd.sock') as nbd_sock_path, \ >> + iotests.FilePaths(['migration.sock', 'nbd.sock'], >> iotests.sock_dir) as \ >> + [migration_sock_path, nbd_sock_path], \ > > while you joined this into one iotests.FilePaths()? Doesn't affect > correctness, but does raise a consistency issue (I noticed it again in > the untouched part of patch 17). The migration.sock FilePath line would have exceeded 80 characters, so I would’ve had to wrap it. Thus I decided I might as well make it a FilePaths. I didn’t dare touch the surrounding code, because that would have required me to explain myself in the commit message. O:-) Max >> iotests.VM('source') as source_vm, \ >> iotests.VM('dest') as dest_vm: >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 1352 bytes --] On 10.10.19 21:50, Eric Blake wrote: > On 10/10/19 10:24 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/common.filter | 8 ++++++-- >> 1 file changed, 6 insertions(+), 2 deletions(-) >> > >> @@ -218,7 +221,8 @@ _filter_nbd() >> # Filter out the TCP port number since this changes between runs. >> $SED -e '/nbd\/.*\.c:/d' \ >> -e 's#127\.0\.0\.1:[0-9]*#127.0.0.1:PORT#g' \ >> - -e "s#?socket=$TEST_DIR#?socket=TEST_DIR#g" \ >> + -e "s#?socket=$SOCK_DIR#?socket=TEST_DIR#g" \ >> + -e "s#?socket=$SOCK_DIR#?socket=SOCK_DIR#g" \ >> -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#' > > This goes away in 23, but this looks crazy. Don't you really want the > first line to replace $TEST_DIR with TEST_DIR (not $SOCK_DIR with > TEST_DIR)? Otherwise, bisection is likely to break until all the > intermediate patches have made the conversion to stop using TEST_DIR. > > I already gave R-b, but you'll need to fix this one. Oops, yes. I messed it up. I only intended to add the SOCK_DIR replacement line. (Originally I had 23 merged into this one, and then I noticed it would break bisection, so I tried to pull it out. And failed, as you can see.) Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 1636 bytes --] On 11/10/2019 09.54, Max Reitz wrote: > On 10.10.19 20:42, Eric Blake wrote: >> On 10/10/19 10:24 AM, Max Reitz wrote: >>> Signed-off-by: Max Reitz <mreitz@redhat.com> >>> --- >>> tests/qemu-iotests/common.filter | 8 ++++++-- >>> 1 file changed, 6 insertions(+), 2 deletions(-) >>> >>> diff --git a/tests/qemu-iotests/common.filter >>> b/tests/qemu-iotests/common.filter >>> index 9f418b4881..cd42f5e7e3 100644 >>> --- a/tests/qemu-iotests/common.filter >>> +++ b/tests/qemu-iotests/common.filter >>> @@ -43,7 +43,8 @@ _filter_qom_path() >>> # replace occurrences of the actual TEST_DIR value with TEST_DIR >>> _filter_testdir() >>> { >>> - $SED -e "s#$TEST_DIR/#TEST_DIR/#g" >>> + $SED -e "s#$TEST_DIR/#TEST_DIR/#g" \ >>> + -e "s#$SOCK_DIR/#SOCK_DIR/#g" >> >> Do we want to output a literal 'SOCK_DIR' (every test that uses it has >> to update their expected output), or can we make this also output a >> literal 'TEST_DIR' (output is a bit more confusing on which dir to look >> in, but fewer files to touch)? Your preference. > > There’s another advantage to filtering it to be TEST_DIR, and that’s the > fact that if $TEST_DIR and $SOCK_DIR are the same, we will always > replace $SOCK_DIR by TEST_DIR. > > But I still preferred filtering it to be SOCK_DIR, because that seemed > to me like we would have done it had we had a SOCK_DIR from the start. I also think that using SOCK_DIR is the better choice. It's a little bit more churn now, but in the long run, it will help to avoid confusion, and I think that's more important. Thomas [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 886 bytes --] On 11.10.19 09:27, Thomas Huth wrote: > On 10/10/2019 17.24, Max Reitz wrote: >> Hi, >> >> Perhaps the main reason we cannot run important tests such as 041 in CI >> is that when they care Unix sockets in $TEST_DIR, the path may become >> too long to connect to them. >> >> To get by this problem, this series lets the check script create a new >> temporary directory (mktemp -d) and then makes the iotests use it for >> all Unix sockets. > > Thanks a lot for tackling this! > > I gave it a try, and most tests work fine now indeed when I run them in > a directory with a veeeery long file name. > > I still get an error with 028 though: Hm, I didn’t see any error for 028 or 055 myself. 028 makes use of common.qemu, which uses FIFOs, and I thought there were exempt from this problem. And for 055 I have no idea. Maybe just bugs in qemu? :-) Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 1084 bytes --] On 11/10/2019 10.03, Max Reitz wrote: > On 11.10.19 09:27, Thomas Huth wrote: >> On 10/10/2019 17.24, Max Reitz wrote: >>> Hi, >>> >>> Perhaps the main reason we cannot run important tests such as 041 in CI >>> is that when they care Unix sockets in $TEST_DIR, the path may become >>> too long to connect to them. >>> >>> To get by this problem, this series lets the check script create a new >>> temporary directory (mktemp -d) and then makes the iotests use it for >>> all Unix sockets. >> >> Thanks a lot for tackling this! >> >> I gave it a try, and most tests work fine now indeed when I run them in >> a directory with a veeeery long file name. >> >> I still get an error with 028 though: > > Hm, I didn’t see any error for 028 or 055 myself. 028 makes use of > common.qemu, which uses FIFOs, and I thought there were exempt from this > problem. And for 055 I have no idea. > > Maybe just bugs in qemu? :-) Yeah, maybe... anyway, both, 028 and 055, are not in the auto group, so I think we simply could ignore these bugs for now. Thomas [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #1.1: Type: text/plain, Size: 1506 bytes --] On 11.10.19 09:27, Thomas Huth wrote: > On 10/10/2019 17.24, Max Reitz wrote: >> Hi, >> >> Perhaps the main reason we cannot run important tests such as 041 in CI >> is that when they care Unix sockets in $TEST_DIR, the path may become >> too long to connect to them. >> >> To get by this problem, this series lets the check script create a new >> temporary directory (mktemp -d) and then makes the iotests use it for >> all Unix sockets. > > Thanks a lot for tackling this! > > I gave it a try, and most tests work fine now indeed when I run them in > a directory with a veeeery long file name. > > I still get an error with 028 though: I still don’t know about 055, but for 028 it looks like a race. We have this: > _send_qemu_cmd $h "drive_backup disk ${TEST_IMG}.copy" "(qemu)" >/dev/null > _send_qemu_cmd $h "" "Formatting" | _filter_img_create But it looks to me like the “Formatting” line comes earlier when the path is longer. No, I don’t know. What I do know is that this looks wrong altogether. Why would the (qemu) prompt necessarily appear before the “Formatting” message? I think the drive-backup job creates the image before it is guaranteed to yield. So I think the solution is to s/(qemu)/Formatting/ in the expected return value, replace the “"" "Formatting"” line by “"" "(qemu)"”, and drop the “Formatting” output from the reference output. (And add an empty “(qemu)” line to the reference output.) Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]