* [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3
@ 2018-10-15 14:14 Max Reitz
2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz
` (9 more replies)
0 siblings, 10 replies; 48+ messages in thread
From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa
This series prepares the iotests to work with both Python 2 and 3. In
some places, it adds version-specific code and decides what to do based
on the version (for instance, whether to import the StringIO class from
the 'io' or the 'StringIO' module), but most of the time, it just makes
code work for both versions in general.
And when we make the switch to make Python 3 mandatory, we can simply
drop the Python 2 branches.
Max Reitz (9):
iotests: Make nbd-fault-injector flush
iotests: Flush in iotests.py's QemuIoInteractive
iotests: Use Python byte strings where appropriate
iotests: Use // for Python integer division
iotests: Different iterator behavior in Python 3
iotests: Explicitly inherit FDs in Python
iotests: 'new' module replacement in 169
iotests: Modify imports for Python 3
iotests: Unify log outputs between Python 2 and 3
scripts/qemu.py | 13 +-
scripts/qmp/qmp.py | 7 +
scripts/qtest.py | 2 +-
tests/qemu-iotests/040 | 4 +-
tests/qemu-iotests/044 | 20 +-
tests/qemu-iotests/056 | 2 +-
tests/qemu-iotests/065 | 4 +-
tests/qemu-iotests/083.out | 9 +
tests/qemu-iotests/093 | 18 +-
tests/qemu-iotests/124 | 4 +-
tests/qemu-iotests/139 | 2 +-
tests/qemu-iotests/147 | 7 +
tests/qemu-iotests/149 | 14 +-
tests/qemu-iotests/151 | 12 +-
tests/qemu-iotests/163 | 8 +-
tests/qemu-iotests/169 | 3 +-
tests/qemu-iotests/194.out | 22 +-
tests/qemu-iotests/202.out | 12 +-
tests/qemu-iotests/203.out | 14 +-
tests/qemu-iotests/206.out | 144 +++----
tests/qemu-iotests/207 | 4 +-
tests/qemu-iotests/207.out | 52 +--
tests/qemu-iotests/208.out | 8 +-
tests/qemu-iotests/210.out | 72 ++--
tests/qemu-iotests/211.out | 66 +--
tests/qemu-iotests/212.out | 102 ++---
tests/qemu-iotests/213.out | 124 +++---
tests/qemu-iotests/216.out | 4 +-
tests/qemu-iotests/218.out | 20 +-
tests/qemu-iotests/219.out | 526 +++++++++++------------
tests/qemu-iotests/222.out | 24 +-
tests/qemu-iotests/iotests.py | 64 ++-
tests/qemu-iotests/nbd-fault-injector.py | 12 +-
tests/qemu-iotests/qcow2.py | 10 +-
34 files changed, 745 insertions(+), 664 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 19:42 ` Eduardo Habkost ` (2 more replies) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive Max Reitz ` (8 subsequent siblings) 9 siblings, 3 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa When closing a connection, make the nbd-fault-injector flush the socket. Without this, the output is a bit unreliable with Python 3. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/083.out | 9 +++++++++ tests/qemu-iotests/nbd-fault-injector.py | 1 + 2 files changed, 10 insertions(+) diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out index be6079d27e..f9af8bb691 100644 --- a/tests/qemu-iotests/083.out +++ b/tests/qemu-iotests/083.out @@ -41,6 +41,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/foo === Check disconnect after neg2 === +Unable to read from socket: Connection reset by peer Connection closed read failed: Input/output error @@ -54,6 +55,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/foo === Check disconnect before request === +Unable to read from socket: Connection reset by peer Connection closed read failed: Input/output error @@ -116,6 +118,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/ === Check disconnect after neg-classic === +Unable to read from socket: Connection reset by peer Connection closed read failed: Input/output error @@ -161,6 +164,8 @@ can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock === Check disconnect after neg2 === +Unable to read from socket: Connection reset by peer +Connection closed read failed: Input/output error === Check disconnect 8 neg2 === @@ -173,6 +178,8 @@ can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock === Check disconnect before request === +Unable to read from socket: Connection reset by peer +Connection closed read failed: Input/output error === Check disconnect after request === @@ -234,6 +241,8 @@ can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock === Check disconnect after neg-classic === +Unable to read from socket: Connection reset by peer +Connection closed read failed: Input/output error *** done diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py index f9193c0fae..439a090eb6 100755 --- a/tests/qemu-iotests/nbd-fault-injector.py +++ b/tests/qemu-iotests/nbd-fault-injector.py @@ -112,6 +112,7 @@ class FaultInjectionSocket(object): if rule.match(event, io): if rule.when == 0 or bufsize is None: print('Closing connection on rule match %s' % rule.name) + self.sock.flush() sys.exit(0) if rule.when != -1: return rule.when -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush 2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz @ 2018-10-15 19:42 ` Eduardo Habkost 2018-10-15 20:24 ` Cleber Rosa 2018-10-16 18:07 ` Eric Blake 2 siblings, 0 replies; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 19:42 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:45PM +0200, Max Reitz wrote: > When closing a connection, make the nbd-fault-injector flush the socket. > Without this, the output is a bit unreliable with Python 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush 2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz 2018-10-15 19:42 ` Eduardo Habkost @ 2018-10-15 20:24 ` Cleber Rosa 2018-10-16 18:07 ` Eric Blake 2 siblings, 0 replies; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 20:24 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > When closing a connection, make the nbd-fault-injector flush the socket. > Without this, the output is a bit unreliable with Python 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush 2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz 2018-10-15 19:42 ` Eduardo Habkost 2018-10-15 20:24 ` Cleber Rosa @ 2018-10-16 18:07 ` Eric Blake 2018-10-19 9:48 ` Max Reitz 2 siblings, 1 reply; 48+ messages in thread From: Eric Blake @ 2018-10-16 18:07 UTC (permalink / raw) To: Max Reitz, qemu-block Cc: Kevin Wolf, Cleber Rosa, qemu-devel, Eduardo Habkost, Vladimir Sementsov-Ogievskiy On 10/15/18 9:14 AM, Max Reitz wrote: > When closing a connection, make the nbd-fault-injector flush the socket. > Without this, the output is a bit unreliable with Python 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/083.out | 9 +++++++++ > tests/qemu-iotests/nbd-fault-injector.py | 1 + > 2 files changed, 10 insertions(+) I already had a complaint that the error message in 083.out should NOT be printing a message (whether the server is python 2 and auto-flushes, or python 3 and needs an explicit flush, the message itself is pointless, and the test is racy as a result). We may need to revisit this patch when that thread is resolved. https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg01041.html That said, I'm not opposed to this patch, if it gets iotests to be more useful in the meantime. > > diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out > index be6079d27e..f9af8bb691 100644 > --- a/tests/qemu-iotests/083.out > +++ b/tests/qemu-iotests/083.out > @@ -41,6 +41,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/foo > > === Check disconnect after neg2 === > > +Unable to read from socket: Connection reset by peer > Connection closed > read failed: Input/output error > > @@ -54,6 +55,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/foo > > === Check disconnect before request === > > +Unable to read from socket: Connection reset by peer > Connection closed > read failed: Input/output error > > @@ -116,6 +118,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/ > > === Check disconnect after neg-classic === > > +Unable to read from socket: Connection reset by peer > Connection closed > read failed: Input/output error > > @@ -161,6 +164,8 @@ can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock > > === Check disconnect after neg2 === > > +Unable to read from socket: Connection reset by peer > +Connection closed > read failed: Input/output error > > === Check disconnect 8 neg2 === > @@ -173,6 +178,8 @@ can't open device nbd+unix:///foo?socket=TEST_DIR/nbd.sock > > === Check disconnect before request === > > +Unable to read from socket: Connection reset by peer > +Connection closed > read failed: Input/output error > > === Check disconnect after request === > @@ -234,6 +241,8 @@ can't open device nbd+unix:///?socket=TEST_DIR/nbd.sock > > === Check disconnect after neg-classic === > > +Unable to read from socket: Connection reset by peer > +Connection closed > read failed: Input/output error > > *** done > diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py > index f9193c0fae..439a090eb6 100755 > --- a/tests/qemu-iotests/nbd-fault-injector.py > +++ b/tests/qemu-iotests/nbd-fault-injector.py > @@ -112,6 +112,7 @@ class FaultInjectionSocket(object): > if rule.match(event, io): > if rule.when == 0 or bufsize is None: > print('Closing connection on rule match %s' % rule.name) > + self.sock.flush() > sys.exit(0) > if rule.when != -1: > return rule.when > -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush 2018-10-16 18:07 ` Eric Blake @ 2018-10-19 9:48 ` Max Reitz 2018-10-19 14:21 ` Eric Blake 0 siblings, 1 reply; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:48 UTC (permalink / raw) To: Eric Blake, qemu-block Cc: Kevin Wolf, Cleber Rosa, qemu-devel, Eduardo Habkost, Vladimir Sementsov-Ogievskiy [-- Attachment #1: Type: text/plain, Size: 3724 bytes --] On 16.10.18 20:07, Eric Blake wrote: > On 10/15/18 9:14 AM, Max Reitz wrote: >> When closing a connection, make the nbd-fault-injector flush the socket. >> Without this, the output is a bit unreliable with Python 3. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/083.out | 9 +++++++++ >> tests/qemu-iotests/nbd-fault-injector.py | 1 + >> 2 files changed, 10 insertions(+) > > I already had a complaint that the error message in 083.out should NOT > be printing a message You mean the NBD server itself, right? > (whether the server is python 2 and auto-flushes, > or python 3 and needs an explicit flush, the message itself is > pointless, and the test is racy as a result). We may need to revisit > this patch when that thread is resolved. > > https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg01041.html Well, it's not like the flush hurts either way. :-) Max > That said, I'm not opposed to this patch, if it gets iotests to be more > useful in the meantime. > >> >> diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out >> index be6079d27e..f9af8bb691 100644 >> --- a/tests/qemu-iotests/083.out >> +++ b/tests/qemu-iotests/083.out >> @@ -41,6 +41,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/foo >> === Check disconnect after neg2 === >> +Unable to read from socket: Connection reset by peer >> Connection closed >> read failed: Input/output error >> @@ -54,6 +55,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/foo >> === Check disconnect before request === >> +Unable to read from socket: Connection reset by peer >> Connection closed >> read failed: Input/output error >> @@ -116,6 +118,7 @@ can't open device nbd+tcp://127.0.0.1:PORT/ >> === Check disconnect after neg-classic === >> +Unable to read from socket: Connection reset by peer >> Connection closed >> read failed: Input/output error >> @@ -161,6 +164,8 @@ can't open device >> nbd+unix:///foo?socket=TEST_DIR/nbd.sock >> === Check disconnect after neg2 === >> +Unable to read from socket: Connection reset by peer >> +Connection closed >> read failed: Input/output error >> === Check disconnect 8 neg2 === >> @@ -173,6 +178,8 @@ can't open device >> nbd+unix:///foo?socket=TEST_DIR/nbd.sock >> === Check disconnect before request === >> +Unable to read from socket: Connection reset by peer >> +Connection closed >> read failed: Input/output error >> === Check disconnect after request === >> @@ -234,6 +241,8 @@ can't open device >> nbd+unix:///?socket=TEST_DIR/nbd.sock >> === Check disconnect after neg-classic === >> +Unable to read from socket: Connection reset by peer >> +Connection closed >> read failed: Input/output error >> *** done >> diff --git a/tests/qemu-iotests/nbd-fault-injector.py >> b/tests/qemu-iotests/nbd-fault-injector.py >> index f9193c0fae..439a090eb6 100755 >> --- a/tests/qemu-iotests/nbd-fault-injector.py >> +++ b/tests/qemu-iotests/nbd-fault-injector.py >> @@ -112,6 +112,7 @@ class FaultInjectionSocket(object): >> if rule.match(event, io): >> if rule.when == 0 or bufsize is None: >> print('Closing connection on rule match %s' % >> rule.name) >> + self.sock.flush() >> sys.exit(0) >> if rule.when != -1: >> return rule.when >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush 2018-10-19 9:48 ` Max Reitz @ 2018-10-19 14:21 ` Eric Blake 0 siblings, 0 replies; 48+ messages in thread From: Eric Blake @ 2018-10-19 14:21 UTC (permalink / raw) To: Max Reitz, qemu-block Cc: Kevin Wolf, Cleber Rosa, qemu-devel, Eduardo Habkost, Vladimir Sementsov-Ogievskiy On 10/19/18 4:48 AM, Max Reitz wrote: > On 16.10.18 20:07, Eric Blake wrote: >> On 10/15/18 9:14 AM, Max Reitz wrote: >>> When closing a connection, make the nbd-fault-injector flush the socket. >>> Without this, the output is a bit unreliable with Python 3. >>> >>> Signed-off-by: Max Reitz <mreitz@redhat.com> >>> --- >>> tests/qemu-iotests/083.out | 9 +++++++++ >>> tests/qemu-iotests/nbd-fault-injector.py | 1 + >>> 2 files changed, 10 insertions(+) >> >> I already had a complaint that the error message in 083.out should NOT >> be printing a message > > You mean the NBD server itself, right? Yes, the NBD server should not be printing the redundant error messages that 083.out exposes. > >> (whether the server is python 2 and auto-flushes, >> or python 3 and needs an explicit flush, the message itself is >> pointless, and the test is racy as a result). We may need to revisit >> this patch when that thread is resolved. >> >> https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg01041.html > > Well, it's not like the flush hurts either way. :-) True. On that grounds: Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 19:43 ` Eduardo Habkost 2018-10-15 20:49 ` Cleber Rosa 2018-10-15 14:14 ` [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate Max Reitz ` (7 subsequent siblings) 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa After issuing a command, flush the pipe. This does not change anything in Python 2, but it makes a difference in Python 3. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/iotests.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 4e67fbbe96..10f2d17419 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -178,6 +178,7 @@ class QemuIoInteractive: cmd = cmd.strip() assert cmd != 'q' and cmd != 'quit' self._p.stdin.write(cmd + '\n') + self._p.stdin.flush() return self._read_output() -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive 2018-10-15 14:14 ` [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive Max Reitz @ 2018-10-15 19:43 ` Eduardo Habkost 2018-10-15 20:49 ` Cleber Rosa 1 sibling, 0 replies; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 19:43 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:46PM +0200, Max Reitz wrote: > After issuing a command, flush the pipe. This does not change anything > in Python 2, but it makes a difference in Python 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive 2018-10-15 14:14 ` [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive Max Reitz 2018-10-15 19:43 ` Eduardo Habkost @ 2018-10-15 20:49 ` Cleber Rosa 1 sibling, 0 replies; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 20:49 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > After issuing a command, flush the pipe. This does not change anything > in Python 2, but it makes a difference in Python 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 19:53 ` Eduardo Habkost 2018-10-15 22:08 ` Philippe Mathieu-Daudé 2018-10-15 14:14 ` [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division Max Reitz ` (6 subsequent siblings) 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa Since byte strings are no longer the default in Python 3, we have to explicitly use them where we need to, which is mostly when working with structures. It also means that we need to open a file in binary mode when we want to use structures. On the other hand, we have to accomodate for the fact that some functions (still) work with byte strings but we want to use unicode strings (in Python 3 at least, and it does not matter in Python 2). This includes base64 encoding, but it is most notable when working with the subprocess module: Either we set univeral_newlines to True so that the default streams are opened in text mode (hence this parameter is aliased as "text" as of 3.7), or, if that is not possible, we have to decode the output to a normal string. Signed-off-by: Max Reitz <mreitz@redhat.com> --- scripts/qtest.py | 2 +- tests/qemu-iotests/044 | 8 ++++---- tests/qemu-iotests/149 | 8 +++++--- tests/qemu-iotests/207 | 4 ++-- tests/qemu-iotests/iotests.py | 11 +++++++---- tests/qemu-iotests/nbd-fault-injector.py | 4 ++-- tests/qemu-iotests/qcow2.py | 10 +++++----- 7 files changed, 26 insertions(+), 21 deletions(-) diff --git a/scripts/qtest.py b/scripts/qtest.py index df0daf26ca..adf1fe3f26 100644 --- a/scripts/qtest.py +++ b/scripts/qtest.py @@ -64,7 +64,7 @@ class QEMUQtestProtocol(object): @param qtest_cmd: qtest command text to be sent """ - self._sock.sendall(qtest_cmd + "\n") + self._sock.sendall((qtest_cmd + "\n").encode('utf-8')) def close(self): self._sock.close() diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 index 11ea0f4d35..69e736f687 100755 --- a/tests/qemu-iotests/044 +++ b/tests/qemu-iotests/044 @@ -53,21 +53,21 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): fd.seek(off_reftable) for i in xrange(0, h.refcount_table_clusters): - sector = ''.join(struct.pack('>Q', + sector = b''.join(struct.pack('>Q', off_refblock + i * 64 * 512 + j * 512) for j in xrange(0, 64)) fd.write(sector) # Write the refcount blocks assert(fd.tell() == off_refblock) - sector = ''.join(struct.pack('>H', 1) for j in xrange(0, 64 * 256)) + sector = b''.join(struct.pack('>H', 1) for j in range(0, 64 * 256)) for block in xrange(0, h.refcount_table_clusters): fd.write(sector) # Write the L1 table assert(fd.tell() == off_l1) assert(off_l2 + 512 * h.l1_size == off_data) - table = ''.join(struct.pack('>Q', (1 << 63) | off_l2 + 512 * j) + table = b''.join(struct.pack('>Q', (1 << 63) | off_l2 + 512 * j) for j in xrange(0, h.l1_size)) fd.write(table) @@ -85,7 +85,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): remaining = remaining - 1024 * 512 off = off + 1024 * 512 - table = ''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) + table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) for j in xrange(0, remaining / 512)) fd.write(table) diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 index 9e0cad76f9..1225334cb8 100755 --- a/tests/qemu-iotests/149 +++ b/tests/qemu-iotests/149 @@ -79,7 +79,7 @@ class LUKSConfig(object): def first_password_base64(self): (pw, slot) = self.first_password() - return base64.b64encode(pw) + return base64.b64encode(pw.encode('ascii')).decode('ascii') def active_slots(self): slots = [] @@ -98,7 +98,8 @@ def verify_passwordless_sudo(): proc = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) + stderr=subprocess.STDOUT, + universal_newlines=True) msg = proc.communicate()[0] @@ -116,7 +117,8 @@ def cryptsetup(args, password=None): proc = subprocess.Popen(fullargs, stdin=subprocess.PIPE, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) + stderr=subprocess.STDOUT, + universal_newlines=True) msg = proc.communicate(password)[0] diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207 index 444ae233ae..2d86a3da37 100755 --- a/tests/qemu-iotests/207 +++ b/tests/qemu-iotests/207 @@ -109,7 +109,7 @@ with iotests.FilePath('t.img') as disk_path, \ md5_key = subprocess.check_output( 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' + 'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1', - shell=True).rstrip() + shell=True).rstrip().decode('ascii') vm.launch() blockdev_create(vm, { 'driver': 'ssh', @@ -147,7 +147,7 @@ with iotests.FilePath('t.img') as disk_path, \ sha1_key = subprocess.check_output( 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' + 'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1', - shell=True).rstrip() + shell=True).rstrip().decode('ascii') vm.launch() blockdev_create(vm, { 'driver': 'ssh', diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 10f2d17419..7290c0b159 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -104,7 +104,8 @@ def qemu_img_pipe(*args): '''Run qemu-img and return its output''' subp = subprocess.Popen(qemu_img_args + list(args), stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) + stderr=subprocess.STDOUT, + universal_newlines=True) exitcode = subp.wait() if exitcode < 0: sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) @@ -128,7 +129,8 @@ def qemu_io(*args): '''Run qemu-io and return the stdout data''' args = qemu_io_args + list(args) subp = subprocess.Popen(args, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) + stderr=subprocess.STDOUT, + universal_newlines=True) exitcode = subp.wait() if exitcode < 0: sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args))) @@ -149,7 +151,8 @@ class QemuIoInteractive: self.args = qemu_io_args + list(args) self._p = subprocess.Popen(self.args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) + stderr=subprocess.STDOUT, + universal_newlines=True) assert self._p.stdout.read(9) == 'qemu-io> ' def close(self): @@ -193,7 +196,7 @@ def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt): def create_image(name, size): '''Create a fully-allocated raw image with sector markers''' - file = open(name, 'w') + file = open(name, 'wb') i = 0 while i < size: sector = struct.pack('>l504xl', i / 512, i / 512) diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py index 439a090eb6..d45e2e0a6a 100755 --- a/tests/qemu-iotests/nbd-fault-injector.py +++ b/tests/qemu-iotests/nbd-fault-injector.py @@ -86,7 +86,7 @@ def recvall(sock, bufsize): raise Exception('unexpected disconnect') chunks.append(chunk) received += len(chunk) - return ''.join(chunks) + return b''.join(chunks) class Rule(object): def __init__(self, name, event, io, when): @@ -177,7 +177,7 @@ def handle_connection(conn, use_export): req = read_request(conn) if req.type == NBD_CMD_READ: write_reply(conn, 0, req.handle) - conn.send('\0' * req.len, event='data') + conn.send(b'\0' * req.len, event='data') elif req.type == NBD_CMD_WRITE: _ = conn.recv(req.len, event='data') write_reply(conn, 0, req.handle) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index b95a837759..b392972d1b 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -10,7 +10,7 @@ class QcowHeaderExtension: def __init__(self, magic, length, data): if length % 8 != 0: padding = 8 - (length % 8) - data += "\0" * padding + data += b"\0" * padding self.magic = magic self.length = length @@ -103,7 +103,7 @@ class QcowHeader: fd.seek(self.header_length) extensions = self.extensions - extensions.append(QcowHeaderExtension(0, 0, "")) + extensions.append(QcowHeaderExtension(0, 0, b"")) for ex in extensions: buf = struct.pack('>II', ex.magic, ex.length) fd.write(buf) @@ -137,8 +137,8 @@ class QcowHeader: for ex in self.extensions: data = ex.data[:ex.length] - if all(c in string.printable for c in data): - data = "'%s'" % data + if all(c in string.printable.encode('ascii') for c in data): + data = "'%s'" % data.decode('ascii') else: data = "<binary>" @@ -178,7 +178,7 @@ def cmd_add_header_ext(fd, magic, data): sys.exit(1) h = QcowHeader(fd) - h.extensions.append(QcowHeaderExtension.create(magic, data)) + h.extensions.append(QcowHeaderExtension.create(magic, data.encode('ascii'))) h.update(fd) def cmd_add_header_ext_stdio(fd, magic): -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate 2018-10-15 14:14 ` [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate Max Reitz @ 2018-10-15 19:53 ` Eduardo Habkost 2018-10-19 8:46 ` Max Reitz 2018-10-15 22:08 ` Philippe Mathieu-Daudé 1 sibling, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 19:53 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:47PM +0200, Max Reitz wrote: > Since byte strings are no longer the default in Python 3, we have to > explicitly use them where we need to, which is mostly when working with > structures. It also means that we need to open a file in binary mode > when we want to use structures. > > On the other hand, we have to accomodate for the fact that some > functions (still) work with byte strings but we want to use unicode > strings (in Python 3 at least, and it does not matter in Python 2). > This includes base64 encoding, but it is most notable when working with > the subprocess module: Either we set univeral_newlines to True so that > the default streams are opened in text mode (hence this parameter is > aliased as "text" as of 3.7), or, if that is not possible, we have to > decode the output to a normal string. > > Signed-off-by: Max Reitz <mreitz@redhat.com> [...] > diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 > index 9e0cad76f9..1225334cb8 100755 > --- a/tests/qemu-iotests/149 > +++ b/tests/qemu-iotests/149 > @@ -79,7 +79,7 @@ class LUKSConfig(object): > > def first_password_base64(self): > (pw, slot) = self.first_password() > - return base64.b64encode(pw) > + return base64.b64encode(pw.encode('ascii')).decode('ascii') Would we want to have a test case for non-ascii passwords in the future? In that case you would probably need to make self.passwords[] contain byte strings instead of text. In either case, using 'ascii' as the encoding everywhere will ensure the code will not try to be too smart about string encodings if that happens. I like that. Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate 2018-10-15 19:53 ` Eduardo Habkost @ 2018-10-19 8:46 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 8:46 UTC (permalink / raw) To: Eduardo Habkost; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 2284 bytes --] On 15.10.18 21:53, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 04:14:47PM +0200, Max Reitz wrote: >> Since byte strings are no longer the default in Python 3, we have to >> explicitly use them where we need to, which is mostly when working with >> structures. It also means that we need to open a file in binary mode >> when we want to use structures. >> >> On the other hand, we have to accomodate for the fact that some >> functions (still) work with byte strings but we want to use unicode >> strings (in Python 3 at least, and it does not matter in Python 2). >> This includes base64 encoding, but it is most notable when working with >> the subprocess module: Either we set univeral_newlines to True so that >> the default streams are opened in text mode (hence this parameter is >> aliased as "text" as of 3.7), or, if that is not possible, we have to >> decode the output to a normal string. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> > [...] >> diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 >> index 9e0cad76f9..1225334cb8 100755 >> --- a/tests/qemu-iotests/149 >> +++ b/tests/qemu-iotests/149 >> @@ -79,7 +79,7 @@ class LUKSConfig(object): >> >> def first_password_base64(self): >> (pw, slot) = self.first_password() >> - return base64.b64encode(pw) >> + return base64.b64encode(pw.encode('ascii')).decode('ascii') > > Would we want to have a test case for non-ascii passwords in the > future? In that case you would probably need to make > self.passwords[] contain byte strings instead of text. I remember someone once providing a non-ASCII initial password to some system for me. I remember the system was running Windows, and I was running Linux, so the system expected ISO-8859-1, while I was sending UTF-8. The moral of the story is that you probably don't want non-ASCII passwords. And if we do want to test them, well, we'll need to decide on an encoding then (or use byte strings, as you suggest). > In either case, using 'ascii' as the encoding everywhere will > ensure the code will not try to be too smart about string > encodings if that happens. I like that. > > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Thanks for reviewing! Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate 2018-10-15 14:14 ` [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate Max Reitz 2018-10-15 19:53 ` Eduardo Habkost @ 2018-10-15 22:08 ` Philippe Mathieu-Daudé 1 sibling, 0 replies; 48+ messages in thread From: Philippe Mathieu-Daudé @ 2018-10-15 22:08 UTC (permalink / raw) To: Max Reitz, qemu-block Cc: Kevin Wolf, Cleber Rosa, qemu-devel, Eduardo Habkost On 15/10/2018 16:14, Max Reitz wrote: > Since byte strings are no longer the default in Python 3, we have to > explicitly use them where we need to, which is mostly when working with > structures. It also means that we need to open a file in binary mode > when we want to use structures. > > On the other hand, we have to accomodate for the fact that some > functions (still) work with byte strings but we want to use unicode > strings (in Python 3 at least, and it does not matter in Python 2). > This includes base64 encoding, but it is most notable when working with > the subprocess module: Either we set univeral_newlines to True so that 'universal_newlines' > the default streams are opened in text mode (hence this parameter is > aliased as "text" as of 3.7), or, if that is not possible, we have to > decode the output to a normal string. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > scripts/qtest.py | 2 +- > tests/qemu-iotests/044 | 8 ++++---- > tests/qemu-iotests/149 | 8 +++++--- > tests/qemu-iotests/207 | 4 ++-- > tests/qemu-iotests/iotests.py | 11 +++++++---- > tests/qemu-iotests/nbd-fault-injector.py | 4 ++-- > tests/qemu-iotests/qcow2.py | 10 +++++----- > 7 files changed, 26 insertions(+), 21 deletions(-) > > diff --git a/scripts/qtest.py b/scripts/qtest.py > index df0daf26ca..adf1fe3f26 100644 > --- a/scripts/qtest.py > +++ b/scripts/qtest.py > @@ -64,7 +64,7 @@ class QEMUQtestProtocol(object): > > @param qtest_cmd: qtest command text to be sent > """ > - self._sock.sendall(qtest_cmd + "\n") > + self._sock.sendall((qtest_cmd + "\n").encode('utf-8')) > > def close(self): > self._sock.close() > diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 > index 11ea0f4d35..69e736f687 100755 > --- a/tests/qemu-iotests/044 > +++ b/tests/qemu-iotests/044 > @@ -53,21 +53,21 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): > fd.seek(off_reftable) > > for i in xrange(0, h.refcount_table_clusters): > - sector = ''.join(struct.pack('>Q', > + sector = b''.join(struct.pack('>Q', > off_refblock + i * 64 * 512 + j * 512) > for j in xrange(0, 64)) > fd.write(sector) > > # Write the refcount blocks > assert(fd.tell() == off_refblock) > - sector = ''.join(struct.pack('>H', 1) for j in xrange(0, 64 * 256)) > + sector = b''.join(struct.pack('>H', 1) for j in range(0, 64 * 256)) > for block in xrange(0, h.refcount_table_clusters): > fd.write(sector) > > # Write the L1 table > assert(fd.tell() == off_l1) > assert(off_l2 + 512 * h.l1_size == off_data) > - table = ''.join(struct.pack('>Q', (1 << 63) | off_l2 + 512 * j) > + table = b''.join(struct.pack('>Q', (1 << 63) | off_l2 + 512 * j) > for j in xrange(0, h.l1_size)) > fd.write(table) > > @@ -85,7 +85,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): > remaining = remaining - 1024 * 512 > off = off + 1024 * 512 > > - table = ''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) > + table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) > for j in xrange(0, remaining / 512)) > fd.write(table) > > diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 > index 9e0cad76f9..1225334cb8 100755 > --- a/tests/qemu-iotests/149 > +++ b/tests/qemu-iotests/149 > @@ -79,7 +79,7 @@ class LUKSConfig(object): > > def first_password_base64(self): > (pw, slot) = self.first_password() > - return base64.b64encode(pw) > + return base64.b64encode(pw.encode('ascii')).decode('ascii') > > def active_slots(self): > slots = [] > @@ -98,7 +98,8 @@ def verify_passwordless_sudo(): > proc = subprocess.Popen(args, > stdin=subprocess.PIPE, > stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, > + universal_newlines=True) > > msg = proc.communicate()[0] > > @@ -116,7 +117,8 @@ def cryptsetup(args, password=None): > proc = subprocess.Popen(fullargs, > stdin=subprocess.PIPE, > stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, > + universal_newlines=True) > > msg = proc.communicate(password)[0] > > diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207 > index 444ae233ae..2d86a3da37 100755 > --- a/tests/qemu-iotests/207 > +++ b/tests/qemu-iotests/207 > @@ -109,7 +109,7 @@ with iotests.FilePath('t.img') as disk_path, \ > md5_key = subprocess.check_output( > 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' + > 'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1', > - shell=True).rstrip() > + shell=True).rstrip().decode('ascii') > > vm.launch() > blockdev_create(vm, { 'driver': 'ssh', > @@ -147,7 +147,7 @@ with iotests.FilePath('t.img') as disk_path, \ > sha1_key = subprocess.check_output( > 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' + > 'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1', > - shell=True).rstrip() > + shell=True).rstrip().decode('ascii') > > vm.launch() > blockdev_create(vm, { 'driver': 'ssh', > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > index 10f2d17419..7290c0b159 100644 > --- a/tests/qemu-iotests/iotests.py > +++ b/tests/qemu-iotests/iotests.py > @@ -104,7 +104,8 @@ def qemu_img_pipe(*args): > '''Run qemu-img and return its output''' > subp = subprocess.Popen(qemu_img_args + list(args), > stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, > + universal_newlines=True) > exitcode = subp.wait() > if exitcode < 0: > sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) > @@ -128,7 +129,8 @@ def qemu_io(*args): > '''Run qemu-io and return the stdout data''' > args = qemu_io_args + list(args) > subp = subprocess.Popen(args, stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, > + universal_newlines=True) > exitcode = subp.wait() > if exitcode < 0: > sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args))) > @@ -149,7 +151,8 @@ class QemuIoInteractive: > self.args = qemu_io_args + list(args) > self._p = subprocess.Popen(self.args, stdin=subprocess.PIPE, > stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, > + universal_newlines=True) > assert self._p.stdout.read(9) == 'qemu-io> ' > > def close(self): > @@ -193,7 +196,7 @@ def compare_images(img1, img2, fmt1=imgfmt, fmt2=imgfmt): > > def create_image(name, size): > '''Create a fully-allocated raw image with sector markers''' > - file = open(name, 'w') > + file = open(name, 'wb') > i = 0 > while i < size: > sector = struct.pack('>l504xl', i / 512, i / 512) > diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py > index 439a090eb6..d45e2e0a6a 100755 > --- a/tests/qemu-iotests/nbd-fault-injector.py > +++ b/tests/qemu-iotests/nbd-fault-injector.py > @@ -86,7 +86,7 @@ def recvall(sock, bufsize): > raise Exception('unexpected disconnect') > chunks.append(chunk) > received += len(chunk) > - return ''.join(chunks) > + return b''.join(chunks) > > class Rule(object): > def __init__(self, name, event, io, when): > @@ -177,7 +177,7 @@ def handle_connection(conn, use_export): > req = read_request(conn) > if req.type == NBD_CMD_READ: > write_reply(conn, 0, req.handle) > - conn.send('\0' * req.len, event='data') > + conn.send(b'\0' * req.len, event='data') > elif req.type == NBD_CMD_WRITE: > _ = conn.recv(req.len, event='data') > write_reply(conn, 0, req.handle) > diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py > index b95a837759..b392972d1b 100755 > --- a/tests/qemu-iotests/qcow2.py > +++ b/tests/qemu-iotests/qcow2.py > @@ -10,7 +10,7 @@ class QcowHeaderExtension: > def __init__(self, magic, length, data): > if length % 8 != 0: > padding = 8 - (length % 8) > - data += "\0" * padding > + data += b"\0" * padding > > self.magic = magic > self.length = length > @@ -103,7 +103,7 @@ class QcowHeader: > > fd.seek(self.header_length) > extensions = self.extensions > - extensions.append(QcowHeaderExtension(0, 0, "")) > + extensions.append(QcowHeaderExtension(0, 0, b"")) > for ex in extensions: > buf = struct.pack('>II', ex.magic, ex.length) > fd.write(buf) > @@ -137,8 +137,8 @@ class QcowHeader: > for ex in self.extensions: > > data = ex.data[:ex.length] > - if all(c in string.printable for c in data): > - data = "'%s'" % data > + if all(c in string.printable.encode('ascii') for c in data): > + data = "'%s'" % data.decode('ascii') > else: > data = "<binary>" > > @@ -178,7 +178,7 @@ def cmd_add_header_ext(fd, magic, data): > sys.exit(1) > > h = QcowHeader(fd) > - h.extensions.append(QcowHeaderExtension.create(magic, data)) > + h.extensions.append(QcowHeaderExtension.create(magic, data.encode('ascii'))) > h.update(fd) > > def cmd_add_header_ext_stdio(fd, magic): > ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (2 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 19:54 ` Eduardo Habkost 2018-10-15 21:13 ` Cleber Rosa 2018-10-15 14:14 ` [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 Max Reitz ` (5 subsequent siblings) 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa In Python 3, / is always a floating-point division. We usually do not want this, and as Python 2.7 understands // as well, change all integer divisions to use that. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/040 | 4 ++-- tests/qemu-iotests/044 | 2 +- tests/qemu-iotests/093 | 18 +++++++++--------- tests/qemu-iotests/149 | 6 +++--- tests/qemu-iotests/151 | 12 ++++++------ tests/qemu-iotests/163 | 2 +- tests/qemu-iotests/iotests.py | 2 +- 7 files changed, 23 insertions(+), 23 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 1cb1ceeb33..b81133a474 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -195,7 +195,7 @@ class TestSingleDrive(ImageCommitTestCase): self.assert_no_active_block_jobs() result = self.vm.qmp('block-commit', device='drive0', top=mid_img, - base=backing_img, speed=(self.image_len / 4)) + base=backing_img, speed=(self.image_len // 4)) self.assert_qmp(result, 'return', {}) result = self.vm.qmp('device_del', id='scsi0') self.assert_qmp(result, 'return', {}) @@ -225,7 +225,7 @@ class TestSingleDrive(ImageCommitTestCase): self.assert_no_active_block_jobs() result = self.vm.qmp('block-commit', device='drive0', top=mid_img, - base=backing_img, speed=(self.image_len / 4)) + base=backing_img, speed=(self.image_len // 4)) self.assert_qmp(result, 'return', {}) result = self.vm.qmp('query-block') diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 index 69e736f687..7ef5e46fe9 100755 --- a/tests/qemu-iotests/044 +++ b/tests/qemu-iotests/044 @@ -86,7 +86,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): off = off + 1024 * 512 table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) - for j in xrange(0, remaining / 512)) + for j in xrange(0, remaining // 512)) fd.write(table) diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 index 9d1971a56c..d88fbc182e 100755 --- a/tests/qemu-iotests/093 +++ b/tests/qemu-iotests/093 @@ -69,18 +69,18 @@ class ThrottleTestCase(iotests.QMPTestCase): # in. The throttled requests won't be executed until we # advance the virtual clock. rq_size = 512 - rd_nr = max(params['bps'] / rq_size / 2, - params['bps_rd'] / rq_size, - params['iops'] / 2, + rd_nr = max(params['bps'] // rq_size // 2, + params['bps_rd'] // rq_size, + params['iops'] // 2, params['iops_rd']) rd_nr *= seconds * 2 - rd_nr /= ndrives - wr_nr = max(params['bps'] / rq_size / 2, - params['bps_wr'] / rq_size, - params['iops'] / 2, + rd_nr //= ndrives + wr_nr = max(params['bps'] // rq_size // 2, + params['bps_wr'] // rq_size, + params['iops'] // 2, params['iops_wr']) wr_nr *= seconds * 2 - wr_nr /= ndrives + wr_nr //= ndrives # Send I/O requests to all drives for i in range(rd_nr): @@ -196,7 +196,7 @@ class ThrottleTestCase(iotests.QMPTestCase): self.configure_throttle(ndrives, settings) # Wait for the bucket to empty so we can do bursts - wait_ns = nsec_per_sec * burst_length * burst_rate / rate + wait_ns = nsec_per_sec * burst_length * burst_rate // rate self.vm.qtest("clock_step %d" % wait_ns) # Test I/O at the max burst rate diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 index 1225334cb8..4f363f295f 100755 --- a/tests/qemu-iotests/149 +++ b/tests/qemu-iotests/149 @@ -314,13 +314,13 @@ def test_once(config, qemu_img=False): image_size = 4 * oneTB if qemu_img: iotests.log("# Create image") - qemu_img_create(config, image_size / oneMB) + qemu_img_create(config, image_size // oneMB) else: iotests.log("# Create image") - create_image(config, image_size / oneMB) + create_image(config, image_size // oneMB) lowOffsetMB = 100 - highOffsetMB = 3 * oneTB / oneMB + highOffsetMB = 3 * oneTB // oneMB try: if not qemu_img: diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 index fe53b9f446..1bb74d67c4 100755 --- a/tests/qemu-iotests/151 +++ b/tests/qemu-iotests/151 @@ -67,9 +67,9 @@ class TestActiveMirror(iotests.QMPTestCase): 'write -P 1 0 %i' % self.image_len); # Start some background requests - for offset in range(1 * self.image_len / 8, 3 * self.image_len / 8, 1024 * 1024): + for offset in range(1 * self.image_len // 8, 3 * self.image_len // 8, 1024 * 1024): self.vm.hmp_qemu_io('source', 'aio_write -P 2 %i 1M' % offset) - for offset in range(2 * self.image_len / 8, 3 * self.image_len / 8, 1024 * 1024): + for offset in range(2 * self.image_len // 8, 3 * self.image_len // 8, 1024 * 1024): self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) # Start the block job @@ -83,9 +83,9 @@ class TestActiveMirror(iotests.QMPTestCase): self.assert_qmp(result, 'return', {}) # Start some more requests - for offset in range(3 * self.image_len / 8, 5 * self.image_len / 8, 1024 * 1024): + for offset in range(3 * self.image_len // 8, 5 * self.image_len // 8, 1024 * 1024): self.vm.hmp_qemu_io('source', 'aio_write -P 3 %i 1M' % offset) - for offset in range(4 * self.image_len / 8, 5 * self.image_len / 8, 1024 * 1024): + for offset in range(4 * self.image_len // 8, 5 * self.image_len // 8, 1024 * 1024): self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) # Wait for the READY event @@ -95,9 +95,9 @@ class TestActiveMirror(iotests.QMPTestCase): # the source) should be settled using the active mechanism. # The mirror code itself asserts that the source BDS's dirty # bitmap will stay clean between READY and COMPLETED. - for offset in range(5 * self.image_len / 8, 7 * self.image_len / 8, 1024 * 1024): + for offset in range(5 * self.image_len // 8, 7 * self.image_len // 8, 1024 * 1024): self.vm.hmp_qemu_io('source', 'aio_write -P 3 %i 1M' % offset) - for offset in range(6 * self.image_len / 8, 7 * self.image_len / 8, 1024 * 1024): + for offset in range(6 * self.image_len // 8, 7 * self.image_len // 8, 1024 * 1024): self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) if sync_source_and_target: diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163 index 403842354e..5fd424761b 100755 --- a/tests/qemu-iotests/163 +++ b/tests/qemu-iotests/163 @@ -38,7 +38,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): entry_bits = 3 entry_size = 1 << entry_bits l1_mask = 0x00fffffffffffe00 - div_roundup = lambda n, d: (n + d - 1) / d + div_roundup = lambda n, d: (n + d - 1) // d def split_by_n(data, n): for x in xrange(0, len(data), n): diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 7290c0b159..7ca94e9278 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -199,7 +199,7 @@ def create_image(name, size): file = open(name, 'wb') i = 0 while i < size: - sector = struct.pack('>l504xl', i / 512, i / 512) + sector = struct.pack('>l504xl', i // 512, i // 512) file.write(sector) i = i + 512 file.close() -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division 2018-10-15 14:14 ` [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division Max Reitz @ 2018-10-15 19:54 ` Eduardo Habkost 2018-10-15 21:13 ` Cleber Rosa 1 sibling, 0 replies; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 19:54 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:48PM +0200, Max Reitz wrote: > In Python 3, / is always a floating-point division. We usually do not > want this, and as Python 2.7 understands // as well, change all integer > divisions to use that. > > Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division 2018-10-15 14:14 ` [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division Max Reitz 2018-10-15 19:54 ` Eduardo Habkost @ 2018-10-15 21:13 ` Cleber Rosa 2018-10-19 9:06 ` Max Reitz 1 sibling, 1 reply; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 21:13 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > In Python 3, / is always a floating-point division. We usually do not > want this, and as Python 2.7 understands // as well, change all integer > divisions to use that. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/040 | 4 ++-- > tests/qemu-iotests/044 | 2 +- > tests/qemu-iotests/093 | 18 +++++++++--------- > tests/qemu-iotests/149 | 6 +++--- > tests/qemu-iotests/151 | 12 ++++++------ > tests/qemu-iotests/163 | 2 +- > tests/qemu-iotests/iotests.py | 2 +- > 7 files changed, 23 insertions(+), 23 deletions(-) > > diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 > index 1cb1ceeb33..b81133a474 100755 > --- a/tests/qemu-iotests/040 > +++ b/tests/qemu-iotests/040 > @@ -195,7 +195,7 @@ class TestSingleDrive(ImageCommitTestCase): > > self.assert_no_active_block_jobs() > result = self.vm.qmp('block-commit', device='drive0', top=mid_img, > - base=backing_img, speed=(self.image_len / 4)) > + base=backing_img, speed=(self.image_len // 4)) > self.assert_qmp(result, 'return', {}) > result = self.vm.qmp('device_del', id='scsi0') > self.assert_qmp(result, 'return', {}) > @@ -225,7 +225,7 @@ class TestSingleDrive(ImageCommitTestCase): > > self.assert_no_active_block_jobs() > result = self.vm.qmp('block-commit', device='drive0', top=mid_img, > - base=backing_img, speed=(self.image_len / 4)) > + base=backing_img, speed=(self.image_len // 4)) > self.assert_qmp(result, 'return', {}) > > result = self.vm.qmp('query-block') > diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 > index 69e736f687..7ef5e46fe9 100755 > --- a/tests/qemu-iotests/044 > +++ b/tests/qemu-iotests/044 > @@ -86,7 +86,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): > off = off + 1024 * 512 > > table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) > - for j in xrange(0, remaining / 512)) > + for j in xrange(0, remaining // 512)) > fd.write(table) > > > diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 > index 9d1971a56c..d88fbc182e 100755 > --- a/tests/qemu-iotests/093 > +++ b/tests/qemu-iotests/093 > @@ -69,18 +69,18 @@ class ThrottleTestCase(iotests.QMPTestCase): > # in. The throttled requests won't be executed until we > # advance the virtual clock. > rq_size = 512 > - rd_nr = max(params['bps'] / rq_size / 2, > - params['bps_rd'] / rq_size, > - params['iops'] / 2, > + rd_nr = max(params['bps'] // rq_size // 2, > + params['bps_rd'] // rq_size, > + params['iops'] // 2, > params['iops_rd']) > rd_nr *= seconds * 2 > - rd_nr /= ndrives > - wr_nr = max(params['bps'] / rq_size / 2, > - params['bps_wr'] / rq_size, > - params['iops'] / 2, > + rd_nr //= ndrives > + wr_nr = max(params['bps'] // rq_size // 2, > + params['bps_wr'] // rq_size, > + params['iops'] // 2, > params['iops_wr']) > wr_nr *= seconds * 2 > - wr_nr /= ndrives > + wr_nr //= ndrives > > # Send I/O requests to all drives > for i in range(rd_nr): > @@ -196,7 +196,7 @@ class ThrottleTestCase(iotests.QMPTestCase): > self.configure_throttle(ndrives, settings) > > # Wait for the bucket to empty so we can do bursts > - wait_ns = nsec_per_sec * burst_length * burst_rate / rate > + wait_ns = nsec_per_sec * burst_length * burst_rate // rate > self.vm.qtest("clock_step %d" % wait_ns) > > # Test I/O at the max burst rate > diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 > index 1225334cb8..4f363f295f 100755 > --- a/tests/qemu-iotests/149 > +++ b/tests/qemu-iotests/149 > @@ -314,13 +314,13 @@ def test_once(config, qemu_img=False): > image_size = 4 * oneTB > if qemu_img: > iotests.log("# Create image") > - qemu_img_create(config, image_size / oneMB) > + qemu_img_create(config, image_size // oneMB) > else: > iotests.log("# Create image") > - create_image(config, image_size / oneMB) > + create_image(config, image_size // oneMB) > > lowOffsetMB = 100 > - highOffsetMB = 3 * oneTB / oneMB > + highOffsetMB = 3 * oneTB // oneMB > > try: > if not qemu_img: > diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 > index fe53b9f446..1bb74d67c4 100755 > --- a/tests/qemu-iotests/151 > +++ b/tests/qemu-iotests/151 > @@ -67,9 +67,9 @@ class TestActiveMirror(iotests.QMPTestCase): > 'write -P 1 0 %i' % self.image_len); > > # Start some background requests > - for offset in range(1 * self.image_len / 8, 3 * self.image_len / 8, 1024 * 1024): > + for offset in range(1 * self.image_len // 8, 3 * self.image_len // 8, 1024 * 1024): > self.vm.hmp_qemu_io('source', 'aio_write -P 2 %i 1M' % offset) > - for offset in range(2 * self.image_len / 8, 3 * self.image_len / 8, 1024 * 1024): > + for offset in range(2 * self.image_len // 8, 3 * self.image_len // 8, 1024 * 1024): > self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) > > # Start the block job > @@ -83,9 +83,9 @@ class TestActiveMirror(iotests.QMPTestCase): > self.assert_qmp(result, 'return', {}) > > # Start some more requests > - for offset in range(3 * self.image_len / 8, 5 * self.image_len / 8, 1024 * 1024): > + for offset in range(3 * self.image_len // 8, 5 * self.image_len // 8, 1024 * 1024): > self.vm.hmp_qemu_io('source', 'aio_write -P 3 %i 1M' % offset) > - for offset in range(4 * self.image_len / 8, 5 * self.image_len / 8, 1024 * 1024): > + for offset in range(4 * self.image_len // 8, 5 * self.image_len // 8, 1024 * 1024): > self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) > > # Wait for the READY event > @@ -95,9 +95,9 @@ class TestActiveMirror(iotests.QMPTestCase): > # the source) should be settled using the active mechanism. > # The mirror code itself asserts that the source BDS's dirty > # bitmap will stay clean between READY and COMPLETED. > - for offset in range(5 * self.image_len / 8, 7 * self.image_len / 8, 1024 * 1024): > + for offset in range(5 * self.image_len // 8, 7 * self.image_len // 8, 1024 * 1024): > self.vm.hmp_qemu_io('source', 'aio_write -P 3 %i 1M' % offset) > - for offset in range(6 * self.image_len / 8, 7 * self.image_len / 8, 1024 * 1024): > + for offset in range(6 * self.image_len // 8, 7 * self.image_len // 8, 1024 * 1024): > self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) > > if sync_source_and_target: > diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163 > index 403842354e..5fd424761b 100755 > --- a/tests/qemu-iotests/163 > +++ b/tests/qemu-iotests/163 > @@ -38,7 +38,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): > entry_bits = 3 > entry_size = 1 << entry_bits > l1_mask = 0x00fffffffffffe00 > - div_roundup = lambda n, d: (n + d - 1) / d > + div_roundup = lambda n, d: (n + d - 1) // d > > def split_by_n(data, n): > for x in xrange(0, len(data), n): > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > index 7290c0b159..7ca94e9278 100644 > --- a/tests/qemu-iotests/iotests.py > +++ b/tests/qemu-iotests/iotests.py > @@ -199,7 +199,7 @@ def create_image(name, size): > file = open(name, 'wb') > i = 0 > while i < size: > - sector = struct.pack('>l504xl', i / 512, i / 512) > + sector = struct.pack('>l504xl', i // 512, i // 512) > file.write(sector) > i = i + 512 > file.close() > Found a few other occurrences: 030:''' % (event, errno, self.STREAM_BUFFER_SIZE / 512, event, event)) 041:''' % (event, errno, self.MIRROR_GRANULARITY / 512, event, event)) 041:''' % (event, errno, self.MIRROR_GRANULARITY / 512, event, event)) - Cleber. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division 2018-10-15 21:13 ` Cleber Rosa @ 2018-10-19 9:06 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:06 UTC (permalink / raw) To: Cleber Rosa, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 9127 bytes --] On 15.10.18 23:13, Cleber Rosa wrote: > > > On 10/15/18 10:14 AM, Max Reitz wrote: >> In Python 3, / is always a floating-point division. We usually do not >> want this, and as Python 2.7 understands // as well, change all integer >> divisions to use that. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/040 | 4 ++-- >> tests/qemu-iotests/044 | 2 +- >> tests/qemu-iotests/093 | 18 +++++++++--------- >> tests/qemu-iotests/149 | 6 +++--- >> tests/qemu-iotests/151 | 12 ++++++------ >> tests/qemu-iotests/163 | 2 +- >> tests/qemu-iotests/iotests.py | 2 +- >> 7 files changed, 23 insertions(+), 23 deletions(-) >> >> diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 >> index 1cb1ceeb33..b81133a474 100755 >> --- a/tests/qemu-iotests/040 >> +++ b/tests/qemu-iotests/040 >> @@ -195,7 +195,7 @@ class TestSingleDrive(ImageCommitTestCase): >> >> self.assert_no_active_block_jobs() >> result = self.vm.qmp('block-commit', device='drive0', top=mid_img, >> - base=backing_img, speed=(self.image_len / 4)) >> + base=backing_img, speed=(self.image_len // 4)) >> self.assert_qmp(result, 'return', {}) >> result = self.vm.qmp('device_del', id='scsi0') >> self.assert_qmp(result, 'return', {}) >> @@ -225,7 +225,7 @@ class TestSingleDrive(ImageCommitTestCase): >> >> self.assert_no_active_block_jobs() >> result = self.vm.qmp('block-commit', device='drive0', top=mid_img, >> - base=backing_img, speed=(self.image_len / 4)) >> + base=backing_img, speed=(self.image_len // 4)) >> self.assert_qmp(result, 'return', {}) >> >> result = self.vm.qmp('query-block') >> diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 >> index 69e736f687..7ef5e46fe9 100755 >> --- a/tests/qemu-iotests/044 >> +++ b/tests/qemu-iotests/044 >> @@ -86,7 +86,7 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): >> off = off + 1024 * 512 >> >> table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) >> - for j in xrange(0, remaining / 512)) >> + for j in xrange(0, remaining // 512)) >> fd.write(table) >> >> >> diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093 >> index 9d1971a56c..d88fbc182e 100755 >> --- a/tests/qemu-iotests/093 >> +++ b/tests/qemu-iotests/093 >> @@ -69,18 +69,18 @@ class ThrottleTestCase(iotests.QMPTestCase): >> # in. The throttled requests won't be executed until we >> # advance the virtual clock. >> rq_size = 512 >> - rd_nr = max(params['bps'] / rq_size / 2, >> - params['bps_rd'] / rq_size, >> - params['iops'] / 2, >> + rd_nr = max(params['bps'] // rq_size // 2, >> + params['bps_rd'] // rq_size, >> + params['iops'] // 2, >> params['iops_rd']) >> rd_nr *= seconds * 2 >> - rd_nr /= ndrives >> - wr_nr = max(params['bps'] / rq_size / 2, >> - params['bps_wr'] / rq_size, >> - params['iops'] / 2, >> + rd_nr //= ndrives >> + wr_nr = max(params['bps'] // rq_size // 2, >> + params['bps_wr'] // rq_size, >> + params['iops'] // 2, >> params['iops_wr']) >> wr_nr *= seconds * 2 >> - wr_nr /= ndrives >> + wr_nr //= ndrives >> >> # Send I/O requests to all drives >> for i in range(rd_nr): >> @@ -196,7 +196,7 @@ class ThrottleTestCase(iotests.QMPTestCase): >> self.configure_throttle(ndrives, settings) >> >> # Wait for the bucket to empty so we can do bursts >> - wait_ns = nsec_per_sec * burst_length * burst_rate / rate >> + wait_ns = nsec_per_sec * burst_length * burst_rate // rate >> self.vm.qtest("clock_step %d" % wait_ns) >> >> # Test I/O at the max burst rate >> diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 >> index 1225334cb8..4f363f295f 100755 >> --- a/tests/qemu-iotests/149 >> +++ b/tests/qemu-iotests/149 >> @@ -314,13 +314,13 @@ def test_once(config, qemu_img=False): >> image_size = 4 * oneTB >> if qemu_img: >> iotests.log("# Create image") >> - qemu_img_create(config, image_size / oneMB) >> + qemu_img_create(config, image_size // oneMB) >> else: >> iotests.log("# Create image") >> - create_image(config, image_size / oneMB) >> + create_image(config, image_size // oneMB) >> >> lowOffsetMB = 100 >> - highOffsetMB = 3 * oneTB / oneMB >> + highOffsetMB = 3 * oneTB // oneMB >> >> try: >> if not qemu_img: >> diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151 >> index fe53b9f446..1bb74d67c4 100755 >> --- a/tests/qemu-iotests/151 >> +++ b/tests/qemu-iotests/151 >> @@ -67,9 +67,9 @@ class TestActiveMirror(iotests.QMPTestCase): >> 'write -P 1 0 %i' % self.image_len); >> >> # Start some background requests >> - for offset in range(1 * self.image_len / 8, 3 * self.image_len / 8, 1024 * 1024): >> + for offset in range(1 * self.image_len // 8, 3 * self.image_len // 8, 1024 * 1024): >> self.vm.hmp_qemu_io('source', 'aio_write -P 2 %i 1M' % offset) >> - for offset in range(2 * self.image_len / 8, 3 * self.image_len / 8, 1024 * 1024): >> + for offset in range(2 * self.image_len // 8, 3 * self.image_len // 8, 1024 * 1024): >> self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) >> >> # Start the block job >> @@ -83,9 +83,9 @@ class TestActiveMirror(iotests.QMPTestCase): >> self.assert_qmp(result, 'return', {}) >> >> # Start some more requests >> - for offset in range(3 * self.image_len / 8, 5 * self.image_len / 8, 1024 * 1024): >> + for offset in range(3 * self.image_len // 8, 5 * self.image_len // 8, 1024 * 1024): >> self.vm.hmp_qemu_io('source', 'aio_write -P 3 %i 1M' % offset) >> - for offset in range(4 * self.image_len / 8, 5 * self.image_len / 8, 1024 * 1024): >> + for offset in range(4 * self.image_len // 8, 5 * self.image_len // 8, 1024 * 1024): >> self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) >> >> # Wait for the READY event >> @@ -95,9 +95,9 @@ class TestActiveMirror(iotests.QMPTestCase): >> # the source) should be settled using the active mechanism. >> # The mirror code itself asserts that the source BDS's dirty >> # bitmap will stay clean between READY and COMPLETED. >> - for offset in range(5 * self.image_len / 8, 7 * self.image_len / 8, 1024 * 1024): >> + for offset in range(5 * self.image_len // 8, 7 * self.image_len // 8, 1024 * 1024): >> self.vm.hmp_qemu_io('source', 'aio_write -P 3 %i 1M' % offset) >> - for offset in range(6 * self.image_len / 8, 7 * self.image_len / 8, 1024 * 1024): >> + for offset in range(6 * self.image_len // 8, 7 * self.image_len // 8, 1024 * 1024): >> self.vm.hmp_qemu_io('source', 'aio_write -z %i 1M' % offset) >> >> if sync_source_and_target: >> diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163 >> index 403842354e..5fd424761b 100755 >> --- a/tests/qemu-iotests/163 >> +++ b/tests/qemu-iotests/163 >> @@ -38,7 +38,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): >> entry_bits = 3 >> entry_size = 1 << entry_bits >> l1_mask = 0x00fffffffffffe00 >> - div_roundup = lambda n, d: (n + d - 1) / d >> + div_roundup = lambda n, d: (n + d - 1) // d >> >> def split_by_n(data, n): >> for x in xrange(0, len(data), n): >> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py >> index 7290c0b159..7ca94e9278 100644 >> --- a/tests/qemu-iotests/iotests.py >> +++ b/tests/qemu-iotests/iotests.py >> @@ -199,7 +199,7 @@ def create_image(name, size): >> file = open(name, 'wb') >> i = 0 >> while i < size: >> - sector = struct.pack('>l504xl', i / 512, i / 512) >> + sector = struct.pack('>l504xl', i // 512, i // 512) >> file.write(sector) >> i = i + 512 >> file.close() >> > > Found a few other occurrences: > > 030:''' % (event, errno, self.STREAM_BUFFER_SIZE / 512, event, event)) > 041:''' % (event, errno, self.MIRROR_GRANULARITY / 512, event, event)) > 041:''' % (event, errno, self.MIRROR_GRANULARITY / 512, event, event)) Oh, thanks. Right, those just worked because they were converted back to integers thanks to the %d in the format string. Will fix. Thanks for reviewing! Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (3 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 20:07 ` Eduardo Habkost 2018-10-15 22:39 ` Cleber Rosa 2018-10-15 14:14 ` [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python Max Reitz ` (4 subsequent siblings) 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa In Python 3, several functions now return iterators instead of lists. This includes range(), items(), map(), and filter(). This means that if we really want a list, we have to wrap those instances with list(). On the other hand, sometimes we do just want an iterator, in which case we have sometimes used xrange() and iteritems() which no longer exist in Python 3. Just change these calls to be range() and items(), which costs a bit of performance in Python 2, but will do the right thing in Python 3 (which is what is important). In one instance, we only wanted the first instance of the result of a filter() call. Instead of using next(filter()) which would work only in Python 3, or list(filter())[0] which would work everywhere but is a bit weird, this instance is changed to a single-line for with next() wrapped around, which works both in 2.7 and 3. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/044 | 12 ++++++------ tests/qemu-iotests/056 | 2 +- tests/qemu-iotests/065 | 4 ++-- tests/qemu-iotests/124 | 4 ++-- tests/qemu-iotests/139 | 2 +- tests/qemu-iotests/163 | 6 +++--- 6 files changed, 15 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 index 7ef5e46fe9..e2d6c9b189 100755 --- a/tests/qemu-iotests/044 +++ b/tests/qemu-iotests/044 @@ -52,23 +52,23 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): # Write a refcount table fd.seek(off_reftable) - for i in xrange(0, h.refcount_table_clusters): + for i in range(0, h.refcount_table_clusters): sector = b''.join(struct.pack('>Q', off_refblock + i * 64 * 512 + j * 512) - for j in xrange(0, 64)) + for j in range(0, 64)) fd.write(sector) # Write the refcount blocks assert(fd.tell() == off_refblock) sector = b''.join(struct.pack('>H', 1) for j in range(0, 64 * 256)) - for block in xrange(0, h.refcount_table_clusters): + for block in range(0, h.refcount_table_clusters): fd.write(sector) # Write the L1 table assert(fd.tell() == off_l1) assert(off_l2 + 512 * h.l1_size == off_data) table = b''.join(struct.pack('>Q', (1 << 63) | off_l2 + 512 * j) - for j in xrange(0, h.l1_size)) + for j in range(0, h.l1_size)) fd.write(table) # Write the L2 tables @@ -79,14 +79,14 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): off = off_data while remaining > 1024 * 512: pytable = list((1 << 63) | off + 512 * j - for j in xrange(0, 1024)) + for j in range(0, 1024)) table = struct.pack('>1024Q', *pytable) fd.write(table) remaining = remaining - 1024 * 512 off = off + 1024 * 512 table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) - for j in xrange(0, remaining // 512)) + for j in range(0, remaining // 512)) fd.write(table) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index 223292175a..3df323984d 100755 --- a/tests/qemu-iotests/056 +++ b/tests/qemu-iotests/056 @@ -32,7 +32,7 @@ target_img = os.path.join(iotests.test_dir, 'target.img') def img_create(img, fmt=iotests.imgfmt, size='64M', **kwargs): fullname = os.path.join(iotests.test_dir, '%s.%s' % (img, fmt)) optargs = [] - for k,v in kwargs.iteritems(): + for k,v in kwargs.items(): optargs = optargs + ['-o', '%s=%s' % (k,v)] args = ['create', '-f', fmt] + optargs + [fullname, size] iotests.qemu_img(*args) diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065 index 72aa9707c7..a339bf6069 100755 --- a/tests/qemu-iotests/065 +++ b/tests/qemu-iotests/065 @@ -59,7 +59,7 @@ class TestQemuImgInfo(TestImageInfoSpecific): :data.index('')] for field in data: self.assertTrue(re.match('^ {4}[^ ]', field) is not None) - data = map(lambda line: line.strip(), data) + data = list(map(lambda line: line.strip(), data)) self.assertEqual(data, self.human_compare) class TestQMP(TestImageInfoSpecific): @@ -80,7 +80,7 @@ class TestQMP(TestImageInfoSpecific): def test_qmp(self): result = self.vm.qmp('query-block')['return'] - drive = filter(lambda drive: drive['device'] == 'drive0', result)[0] + drive = next(drive for drive in result if drive['device'] == 'drive0') data = drive['inserted']['image']['format-specific'] self.assertEqual(data['type'], iotests.imgfmt) self.assertEqual(data['data'], self.compare) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3ea4ac53f5..9f189e3b54 100755 --- a/tests/qemu-iotests/124 +++ b/tests/qemu-iotests/124 @@ -39,7 +39,7 @@ def try_remove(img): def transaction_action(action, **kwargs): return { 'type': action, - 'data': dict((k.replace('_', '-'), v) for k, v in kwargs.iteritems()) + 'data': dict((k.replace('_', '-'), v) for k, v in kwargs.items()) } @@ -134,7 +134,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): def img_create(self, img, fmt=iotests.imgfmt, size='64M', parent=None, parentFormat=None, **kwargs): optargs = [] - for k,v in kwargs.iteritems(): + for k,v in kwargs.items(): optargs = optargs + ['-o', '%s=%s' % (k,v)] args = ['create', '-f', fmt] + optargs + [img, size] if parent: diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 index cc7fe337f3..e00f10b8c8 100755 --- a/tests/qemu-iotests/139 +++ b/tests/qemu-iotests/139 @@ -51,7 +51,7 @@ class TestBlockdevDel(iotests.QMPTestCase): # Check whether a BlockDriverState exists def checkBlockDriverState(self, node, must_exist = True): result = self.vm.qmp('query-named-block-nodes') - nodes = filter(lambda x: x['node-name'] == node, result['return']) + nodes = list(filter(lambda x: x['node-name'] == node, result['return'])) self.assertLessEqual(len(nodes), 1) self.assertEqual(must_exist, len(nodes) == 1) diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163 index 5fd424761b..35c1a2bafc 100755 --- a/tests/qemu-iotests/163 +++ b/tests/qemu-iotests/163 @@ -41,7 +41,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): div_roundup = lambda n, d: (n + d - 1) // d def split_by_n(data, n): - for x in xrange(0, len(data), n): + for x in range(0, len(data), n): yield struct.unpack('>Q', data[x:x + n])[0] & l1_mask def check_l1_table(h, l1_data): @@ -135,8 +135,8 @@ class ShrinkBaseClass(iotests.QMPTestCase): self.image_verify() def test_random_write(self): - offs_list = range(0, size_to_int(self.image_len), - size_to_int(self.chunk_size)) + offs_list = list(range(0, size_to_int(self.image_len), + size_to_int(self.chunk_size))) random.shuffle(offs_list) for offs in offs_list: qemu_io('-c', 'write -P 0xff %d %s' % (offs, self.chunk_size), -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 2018-10-15 14:14 ` [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 Max Reitz @ 2018-10-15 20:07 ` Eduardo Habkost 2018-10-19 8:52 ` Max Reitz 2018-10-15 22:39 ` Cleber Rosa 1 sibling, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 20:07 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:49PM +0200, Max Reitz wrote: > In Python 3, several functions now return iterators instead of lists. > This includes range(), items(), map(), and filter(). This means that if > we really want a list, we have to wrap those instances with list(). On > the other hand, sometimes we do just want an iterator, in which case we > have sometimes used xrange() and iteritems() which no longer exist in > Python 3. Just change these calls to be range() and items(), which > costs a bit of performance in Python 2, but will do the right thing in > Python 3 (which is what is important). > > In one instance, we only wanted the first instance of the result of a > filter() call. Instead of using next(filter()) which would work only in > Python 3, or list(filter())[0] which would work everywhere but is a bit > weird, this instance is changed to a single-line for with next() wrapped > around, which works both in 2.7 and 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Minor comments below: > --- > tests/qemu-iotests/044 | 12 ++++++------ > tests/qemu-iotests/056 | 2 +- > tests/qemu-iotests/065 | 4 ++-- > tests/qemu-iotests/124 | 4 ++-- > tests/qemu-iotests/139 | 2 +- > tests/qemu-iotests/163 | 6 +++--- > 6 files changed, 15 insertions(+), 15 deletions(-) > > diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044 > index 7ef5e46fe9..e2d6c9b189 100755 > --- a/tests/qemu-iotests/044 > +++ b/tests/qemu-iotests/044 > @@ -52,23 +52,23 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): > # Write a refcount table > fd.seek(off_reftable) > > - for i in xrange(0, h.refcount_table_clusters): > + for i in range(0, h.refcount_table_clusters): > sector = b''.join(struct.pack('>Q', > off_refblock + i * 64 * 512 + j * 512) > - for j in xrange(0, 64)) > + for j in range(0, 64)) > fd.write(sector) > > # Write the refcount blocks > assert(fd.tell() == off_refblock) > sector = b''.join(struct.pack('>H', 1) for j in range(0, 64 * 256)) > - for block in xrange(0, h.refcount_table_clusters): > + for block in range(0, h.refcount_table_clusters): > fd.write(sector) > > # Write the L1 table > assert(fd.tell() == off_l1) > assert(off_l2 + 512 * h.l1_size == off_data) > table = b''.join(struct.pack('>Q', (1 << 63) | off_l2 + 512 * j) > - for j in xrange(0, h.l1_size)) > + for j in range(0, h.l1_size)) > fd.write(table) > > # Write the L2 tables > @@ -79,14 +79,14 @@ class TestRefcountTableGrowth(iotests.QMPTestCase): > off = off_data > while remaining > 1024 * 512: > pytable = list((1 << 63) | off + 512 * j > - for j in xrange(0, 1024)) > + for j in range(0, 1024)) > table = struct.pack('>1024Q', *pytable) > fd.write(table) > remaining = remaining - 1024 * 512 > off = off + 1024 * 512 > > table = b''.join(struct.pack('>Q', (1 << 63) | off + 512 * j) > - for j in xrange(0, remaining // 512)) > + for j in range(0, remaining // 512)) > fd.write(table) > > > diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 > index 223292175a..3df323984d 100755 > --- a/tests/qemu-iotests/056 > +++ b/tests/qemu-iotests/056 > @@ -32,7 +32,7 @@ target_img = os.path.join(iotests.test_dir, 'target.img') > def img_create(img, fmt=iotests.imgfmt, size='64M', **kwargs): > fullname = os.path.join(iotests.test_dir, '%s.%s' % (img, fmt)) > optargs = [] > - for k,v in kwargs.iteritems(): > + for k,v in kwargs.items(): > optargs = optargs + ['-o', '%s=%s' % (k,v)] > args = ['create', '-f', fmt] + optargs + [fullname, size] > iotests.qemu_img(*args) > diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065 > index 72aa9707c7..a339bf6069 100755 > --- a/tests/qemu-iotests/065 > +++ b/tests/qemu-iotests/065 > @@ -59,7 +59,7 @@ class TestQemuImgInfo(TestImageInfoSpecific): > :data.index('')] > for field in data: > self.assertTrue(re.match('^ {4}[^ ]', field) is not None) > - data = map(lambda line: line.strip(), data) > + data = list(map(lambda line: line.strip(), data)) I would find this more readable: data = [line.strip() for line in data] Not a blocker, though. > self.assertEqual(data, self.human_compare) > > class TestQMP(TestImageInfoSpecific): > @@ -80,7 +80,7 @@ class TestQMP(TestImageInfoSpecific): > > def test_qmp(self): > result = self.vm.qmp('query-block')['return'] > - drive = filter(lambda drive: drive['device'] == 'drive0', result)[0] > + drive = next(drive for drive in result if drive['device'] == 'drive0') I didn't understand what you meant by "single-line for" on the commit message, until I saw the generator expression here. :) This will raise an exception if there's no drive0 in the list, but that was already true on the original code. > data = drive['inserted']['image']['format-specific'] > self.assertEqual(data['type'], iotests.imgfmt) > self.assertEqual(data['data'], self.compare) > diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 > index 3ea4ac53f5..9f189e3b54 100755 > --- a/tests/qemu-iotests/124 > +++ b/tests/qemu-iotests/124 > @@ -39,7 +39,7 @@ def try_remove(img): > def transaction_action(action, **kwargs): > return { > 'type': action, > - 'data': dict((k.replace('_', '-'), v) for k, v in kwargs.iteritems()) > + 'data': dict((k.replace('_', '-'), v) for k, v in kwargs.items()) > } > > > @@ -134,7 +134,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): > def img_create(self, img, fmt=iotests.imgfmt, size='64M', > parent=None, parentFormat=None, **kwargs): > optargs = [] > - for k,v in kwargs.iteritems(): > + for k,v in kwargs.items(): > optargs = optargs + ['-o', '%s=%s' % (k,v)] > args = ['create', '-f', fmt] + optargs + [img, size] > if parent: > diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 > index cc7fe337f3..e00f10b8c8 100755 > --- a/tests/qemu-iotests/139 > +++ b/tests/qemu-iotests/139 > @@ -51,7 +51,7 @@ class TestBlockdevDel(iotests.QMPTestCase): > # Check whether a BlockDriverState exists > def checkBlockDriverState(self, node, must_exist = True): > result = self.vm.qmp('query-named-block-nodes') > - nodes = filter(lambda x: x['node-name'] == node, result['return']) > + nodes = list(filter(lambda x: x['node-name'] == node, result['return'])) I'd prefer a list expression: nodes = [x for x in result['return'] if x['node-name'] == node] Also not a blocker. > self.assertLessEqual(len(nodes), 1) > self.assertEqual(must_exist, len(nodes) == 1) > > diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163 > index 5fd424761b..35c1a2bafc 100755 > --- a/tests/qemu-iotests/163 > +++ b/tests/qemu-iotests/163 > @@ -41,7 +41,7 @@ class ShrinkBaseClass(iotests.QMPTestCase): > div_roundup = lambda n, d: (n + d - 1) // d > > def split_by_n(data, n): > - for x in xrange(0, len(data), n): > + for x in range(0, len(data), n): > yield struct.unpack('>Q', data[x:x + n])[0] & l1_mask > > def check_l1_table(h, l1_data): > @@ -135,8 +135,8 @@ class ShrinkBaseClass(iotests.QMPTestCase): > self.image_verify() > > def test_random_write(self): > - offs_list = range(0, size_to_int(self.image_len), > - size_to_int(self.chunk_size)) > + offs_list = list(range(0, size_to_int(self.image_len), > + size_to_int(self.chunk_size))) > random.shuffle(offs_list) > for offs in offs_list: > qemu_io('-c', 'write -P 0xff %d %s' % (offs, self.chunk_size), > -- > 2.17.1 > -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 2018-10-15 20:07 ` Eduardo Habkost @ 2018-10-19 8:52 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 8:52 UTC (permalink / raw) To: Eduardo Habkost; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 4588 bytes --] On 15.10.18 22:07, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 04:14:49PM +0200, Max Reitz wrote: >> In Python 3, several functions now return iterators instead of lists. >> This includes range(), items(), map(), and filter(). This means that if >> we really want a list, we have to wrap those instances with list(). On >> the other hand, sometimes we do just want an iterator, in which case we >> have sometimes used xrange() and iteritems() which no longer exist in >> Python 3. Just change these calls to be range() and items(), which >> costs a bit of performance in Python 2, but will do the right thing in >> Python 3 (which is what is important). >> >> In one instance, we only wanted the first instance of the result of a >> filter() call. Instead of using next(filter()) which would work only in >> Python 3, or list(filter())[0] which would work everywhere but is a bit >> weird, this instance is changed to a single-line for with next() wrapped >> around, which works both in 2.7 and 3. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> > > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> > > Minor comments below: [...] >> diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065 >> index 72aa9707c7..a339bf6069 100755 >> --- a/tests/qemu-iotests/065 >> +++ b/tests/qemu-iotests/065 >> @@ -59,7 +59,7 @@ class TestQemuImgInfo(TestImageInfoSpecific): >> :data.index('')] >> for field in data: >> self.assertTrue(re.match('^ {4}[^ ]', field) is not None) >> - data = map(lambda line: line.strip(), data) >> + data = list(map(lambda line: line.strip(), data)) > > I would find this more readable: > > data = [line.strip() for line in data] > > Not a blocker, though. Yes, I agree, that looks better. >> self.assertEqual(data, self.human_compare) >> >> class TestQMP(TestImageInfoSpecific): >> @@ -80,7 +80,7 @@ class TestQMP(TestImageInfoSpecific): >> >> def test_qmp(self): >> result = self.vm.qmp('query-block')['return'] >> - drive = filter(lambda drive: drive['device'] == 'drive0', result)[0] >> + drive = next(drive for drive in result if drive['device'] == 'drive0') > > I didn't understand what you meant by "single-line for" on the > commit message, until I saw the generator expression here. :) Maybe I should at least put quotes around the "for", because the combination of "for with" really makes it difficult to read... > This will raise an exception if there's no drive0 in the list, > but that was already true on the original code. > > >> data = drive['inserted']['image']['format-specific'] >> self.assertEqual(data['type'], iotests.imgfmt) >> self.assertEqual(data['data'], self.compare) >> diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 >> index 3ea4ac53f5..9f189e3b54 100755 >> --- a/tests/qemu-iotests/124 >> +++ b/tests/qemu-iotests/124 >> @@ -39,7 +39,7 @@ def try_remove(img): >> def transaction_action(action, **kwargs): >> return { >> 'type': action, >> - 'data': dict((k.replace('_', '-'), v) for k, v in kwargs.iteritems()) >> + 'data': dict((k.replace('_', '-'), v) for k, v in kwargs.items()) >> } >> >> >> @@ -134,7 +134,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): >> def img_create(self, img, fmt=iotests.imgfmt, size='64M', >> parent=None, parentFormat=None, **kwargs): >> optargs = [] >> - for k,v in kwargs.iteritems(): >> + for k,v in kwargs.items(): >> optargs = optargs + ['-o', '%s=%s' % (k,v)] >> args = ['create', '-f', fmt] + optargs + [img, size] >> if parent: >> diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139 >> index cc7fe337f3..e00f10b8c8 100755 >> --- a/tests/qemu-iotests/139 >> +++ b/tests/qemu-iotests/139 >> @@ -51,7 +51,7 @@ class TestBlockdevDel(iotests.QMPTestCase): >> # Check whether a BlockDriverState exists >> def checkBlockDriverState(self, node, must_exist = True): >> result = self.vm.qmp('query-named-block-nodes') >> - nodes = filter(lambda x: x['node-name'] == node, result['return']) >> + nodes = list(filter(lambda x: x['node-name'] == node, result['return'])) > > I'd prefer a list expression: > > nodes = [x for x in result['return'] if x['node-name'] == node] > > Also not a blocker. I don't mind either way, so why not. Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 2018-10-15 14:14 ` [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 Max Reitz 2018-10-15 20:07 ` Eduardo Habkost @ 2018-10-15 22:39 ` Cleber Rosa 2018-10-19 9:42 ` Max Reitz 1 sibling, 1 reply; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 22:39 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > In Python 3, several functions now return iterators instead of lists. > This includes range(), items(), map(), and filter(). This means that if > we really want a list, we have to wrap those instances with list(). On > the other hand, sometimes we do just want an iterator, in which case we > have sometimes used xrange() and iteritems() which no longer exist in > Python 3. Just change these calls to be range() and items(), which > costs a bit of performance in Python 2, but will do the right thing in > Python 3 (which is what is important). > > In one instance, we only wanted the first instance of the result of a > filter() call. Instead of using next(filter()) which would work only in > Python 3, or list(filter())[0] which would work everywhere but is a bit > weird, this instance is changed to a single-line for with next() wrapped > around, which works both in 2.7 and 3. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/044 | 12 ++++++------ > tests/qemu-iotests/056 | 2 +- > tests/qemu-iotests/065 | 4 ++-- > tests/qemu-iotests/124 | 4 ++-- > tests/qemu-iotests/139 | 2 +- > tests/qemu-iotests/163 | 6 +++--- > 6 files changed, 15 insertions(+), 15 deletions(-) > You have 2 files here which use xrange (which is a manageable size, and whose occurrences involve a moderate size of items) to also consider: if sys.version_info.major == 2: range = xrange Defaulting to the Python 3 names, but behaving the same across Python 2 and 3. To do the same for dict.iteritems() => dict.items() requires a lot more code, so I'd stay away from it for now. Also, it looks like most of those dicts are small in size (**kwargs and the like). Other than that suggestion, Reviewed-by: Cleber Rosa <crosa@redhat.com> ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 2018-10-15 22:39 ` Cleber Rosa @ 2018-10-19 9:42 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:42 UTC (permalink / raw) To: Cleber Rosa, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 2150 bytes --] On 16.10.18 00:39, Cleber Rosa wrote: > > > On 10/15/18 10:14 AM, Max Reitz wrote: >> In Python 3, several functions now return iterators instead of lists. >> This includes range(), items(), map(), and filter(). This means that if >> we really want a list, we have to wrap those instances with list(). On >> the other hand, sometimes we do just want an iterator, in which case we >> have sometimes used xrange() and iteritems() which no longer exist in >> Python 3. Just change these calls to be range() and items(), which >> costs a bit of performance in Python 2, but will do the right thing in >> Python 3 (which is what is important). >> >> In one instance, we only wanted the first instance of the result of a >> filter() call. Instead of using next(filter()) which would work only in >> Python 3, or list(filter())[0] which would work everywhere but is a bit >> weird, this instance is changed to a single-line for with next() wrapped >> around, which works both in 2.7 and 3. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/044 | 12 ++++++------ >> tests/qemu-iotests/056 | 2 +- >> tests/qemu-iotests/065 | 4 ++-- >> tests/qemu-iotests/124 | 4 ++-- >> tests/qemu-iotests/139 | 2 +- >> tests/qemu-iotests/163 | 6 +++--- >> 6 files changed, 15 insertions(+), 15 deletions(-) >> > > You have 2 files here which use xrange (which is a manageable size, and > whose occurrences involve a moderate size of items) to also consider: > > if sys.version_info.major == 2: > range = xrange I don't think it's necessary, but it's so easy to grep for "sys.version_info" once we want to completely switch to Python 3 that I can't find good arguments against it. O:-) Max > Defaulting to the Python 3 names, but behaving the same across Python 2 > and 3. > > To do the same for dict.iteritems() => dict.items() requires a lot more > code, so I'd stay away from it for now. Also, it looks like most of > those dicts are small in size (**kwargs and the like). > > Other than that suggestion, > > Reviewed-by: Cleber Rosa <crosa@redhat.com> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (4 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 20:30 ` Eduardo Habkost 2018-10-15 23:18 ` Cleber Rosa 2018-10-15 14:14 ` [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 Max Reitz ` (3 subsequent siblings) 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa Python 3.2 introduced the inheritable attribute for FDs. At the same time, it changed the default so that all FDs are not inheritable by default, that only inheritable FDs are inherited to subprocesses, and only if close_fds is explicitly set to False. Adhere to this by setting close_fds to False when working with subprocesses that may want to inherit FDs, and by trying to set_inheritable() on FDs that we do want to bequeath to them. Signed-off-by: Max Reitz <mreitz@redhat.com> --- scripts/qemu.py | 13 +++++++++++-- scripts/qmp/qmp.py | 7 +++++++ tests/qemu-iotests/147 | 7 +++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/scripts/qemu.py b/scripts/qemu.py index f099ce7278..28366c4a67 100644 --- a/scripts/qemu.py +++ b/scripts/qemu.py @@ -142,10 +142,18 @@ class QEMUMachine(object): if opts: options.append(opts) + # This did not exist before 3.2, but since then it is + # mandatory for our purpose + try: + os.set_inheritable(fd, True) + except AttributeError: + pass + self._args.append('-add-fd') self._args.append(','.join(options)) return self + # The caller needs to make sure the FD is inheritable def send_fd_scm(self, fd_file_path): # In iotest.py, the qmp should always use unix socket. assert self._qmp.is_scm_available() @@ -159,7 +167,7 @@ class QEMUMachine(object): "%s" % fd_file_path] devnull = open(os.path.devnull, 'rb') proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) + stderr=subprocess.STDOUT, close_fds=False) output = proc.communicate()[0] if output: LOG.debug(output) @@ -280,7 +288,8 @@ class QEMUMachine(object): stdin=devnull, stdout=self._qemu_log_file, stderr=subprocess.STDOUT, - shell=False) + shell=False, + close_fds=False) self._post_launch() def wait(self): diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py index 5c8cf6a056..009be8345b 100644 --- a/scripts/qmp/qmp.py +++ b/scripts/qmp/qmp.py @@ -10,6 +10,7 @@ import json import errno +import os import socket import logging @@ -253,4 +254,10 @@ class QEMUMonitorProtocol(object): return self.__sock.fileno() def is_scm_available(self): + # This did not exist before 3.2, but since then it is + # mandatory for our purpose + try: + os.set_inheritable(self.get_sock_fd(), True) + except AttributeError: + pass return self.__sock.family == socket.AF_UNIX diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 index d2081df84b..b58455645b 100755 --- a/tests/qemu-iotests/147 +++ b/tests/qemu-iotests/147 @@ -229,6 +229,13 @@ class BuiltinNBD(NBDBlockdevAddBase): sockfd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) sockfd.connect(unix_socket) + # This did not exist before 3.2, but since then it is + # mandatory for our purpose + try: + os.set_inheritable(sockfd.fileno(), True) + except AttributeError: + pass + result = self.vm.send_fd_scm(str(sockfd.fileno())) self.assertEqual(result, 0, 'Failed to send socket FD') -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python 2018-10-15 14:14 ` [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python Max Reitz @ 2018-10-15 20:30 ` Eduardo Habkost 2018-10-19 9:03 ` Max Reitz 2018-10-15 23:18 ` Cleber Rosa 1 sibling, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 20:30 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:50PM +0200, Max Reitz wrote: > Python 3.2 introduced the inheritable attribute for FDs. At the same > time, it changed the default so that all FDs are not inheritable by > default, that only inheritable FDs are inherited to subprocesses, and > only if close_fds is explicitly set to False. > > Adhere to this by setting close_fds to False when working with > subprocesses that may want to inherit FDs, and by trying to > set_inheritable() on FDs that we do want to bequeath to them. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > scripts/qemu.py | 13 +++++++++++-- > scripts/qmp/qmp.py | 7 +++++++ > tests/qemu-iotests/147 | 7 +++++++ > 3 files changed, 25 insertions(+), 2 deletions(-) > > diff --git a/scripts/qemu.py b/scripts/qemu.py > index f099ce7278..28366c4a67 100644 > --- a/scripts/qemu.py > +++ b/scripts/qemu.py > @@ -142,10 +142,18 @@ class QEMUMachine(object): > if opts: > options.append(opts) > > + # This did not exist before 3.2, but since then it is > + # mandatory for our purpose > + try: > + os.set_inheritable(fd, True) > + except AttributeError: > + pass > + This is add_fd(), so calling set_inheritable() automatically here makes sense. > self._args.append('-add-fd') > self._args.append(','.join(options)) > return self > > + # The caller needs to make sure the FD is inheritable > def send_fd_scm(self, fd_file_path): > # In iotest.py, the qmp should always use unix socket. > assert self._qmp.is_scm_available() > @@ -159,7 +167,7 @@ class QEMUMachine(object): > "%s" % fd_file_path] > devnull = open(os.path.devnull, 'rb') > proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, close_fds=False) > output = proc.communicate()[0] > if output: > LOG.debug(output) > @@ -280,7 +288,8 @@ class QEMUMachine(object): > stdin=devnull, > stdout=self._qemu_log_file, > stderr=subprocess.STDOUT, > - shell=False) > + shell=False, > + close_fds=False) > self._post_launch() > > def wait(self): > diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py > index 5c8cf6a056..009be8345b 100644 > --- a/scripts/qmp/qmp.py > +++ b/scripts/qmp/qmp.py > @@ -10,6 +10,7 @@ > > import json > import errno > +import os > import socket > import logging > > @@ -253,4 +254,10 @@ class QEMUMonitorProtocol(object): > return self.__sock.fileno() > > def is_scm_available(self): > + # This did not exist before 3.2, but since then it is > + # mandatory for our purpose > + try: > + os.set_inheritable(self.get_sock_fd(), True) > + except AttributeError: > + pass Why did you decide to place this code inside is_scm_available()? For reference, this is the only caller of is_scm_available(): def send_fd_scm(self, fd_file_path): # In iotest.py, the qmp should always use unix socket. assert self._qmp.is_scm_available() ... In addition to making a method called is_*() have an unexpected side-effect, the method won't be called at all if running with debugging disabled. I suggest simply placing the os.set_inheritable() call inside send_fd_scm(), as close as possible to the subprocess.Popen() call. > return self.__sock.family == socket.AF_UNIX > diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 > index d2081df84b..b58455645b 100755 > --- a/tests/qemu-iotests/147 > +++ b/tests/qemu-iotests/147 > @@ -229,6 +229,13 @@ class BuiltinNBD(NBDBlockdevAddBase): > sockfd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) > sockfd.connect(unix_socket) > > + # This did not exist before 3.2, but since then it is > + # mandatory for our purpose > + try: > + os.set_inheritable(sockfd.fileno(), True) > + except AttributeError: > + pass > + Why not make send_fd_scm() responsible for calling os.set_inheritable(), making this hunk unnecessary? > result = self.vm.send_fd_scm(str(sockfd.fileno())) > self.assertEqual(result, 0, 'Failed to send socket FD') > > -- > 2.17.1 > -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python 2018-10-15 20:30 ` Eduardo Habkost @ 2018-10-19 9:03 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:03 UTC (permalink / raw) To: Eduardo Habkost; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa [-- Attachment #1: Type: text/plain, Size: 6369 bytes --] On 15.10.18 22:30, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 04:14:50PM +0200, Max Reitz wrote: >> Python 3.2 introduced the inheritable attribute for FDs. At the same >> time, it changed the default so that all FDs are not inheritable by >> default, that only inheritable FDs are inherited to subprocesses, and >> only if close_fds is explicitly set to False. >> >> Adhere to this by setting close_fds to False when working with >> subprocesses that may want to inherit FDs, and by trying to >> set_inheritable() on FDs that we do want to bequeath to them. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> scripts/qemu.py | 13 +++++++++++-- >> scripts/qmp/qmp.py | 7 +++++++ >> tests/qemu-iotests/147 | 7 +++++++ >> 3 files changed, 25 insertions(+), 2 deletions(-) >> >> diff --git a/scripts/qemu.py b/scripts/qemu.py >> index f099ce7278..28366c4a67 100644 >> --- a/scripts/qemu.py >> +++ b/scripts/qemu.py >> @@ -142,10 +142,18 @@ class QEMUMachine(object): >> if opts: >> options.append(opts) >> >> + # This did not exist before 3.2, but since then it is >> + # mandatory for our purpose >> + try: >> + os.set_inheritable(fd, True) >> + except AttributeError: >> + pass >> + > > This is add_fd(), so calling set_inheritable() automatically here > makes sense. > >> self._args.append('-add-fd') >> self._args.append(','.join(options)) >> return self >> >> + # The caller needs to make sure the FD is inheritable >> def send_fd_scm(self, fd_file_path): >> # In iotest.py, the qmp should always use unix socket. >> assert self._qmp.is_scm_available() >> @@ -159,7 +167,7 @@ class QEMUMachine(object): >> "%s" % fd_file_path] >> devnull = open(os.path.devnull, 'rb') >> proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE, >> - stderr=subprocess.STDOUT) >> + stderr=subprocess.STDOUT, close_fds=False) >> output = proc.communicate()[0] >> if output: >> LOG.debug(output) >> @@ -280,7 +288,8 @@ class QEMUMachine(object): >> stdin=devnull, >> stdout=self._qemu_log_file, >> stderr=subprocess.STDOUT, >> - shell=False) >> + shell=False, >> + close_fds=False) >> self._post_launch() >> >> def wait(self): >> diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py >> index 5c8cf6a056..009be8345b 100644 >> --- a/scripts/qmp/qmp.py >> +++ b/scripts/qmp/qmp.py >> @@ -10,6 +10,7 @@ >> >> import json >> import errno >> +import os >> import socket >> import logging >> >> @@ -253,4 +254,10 @@ class QEMUMonitorProtocol(object): >> return self.__sock.fileno() >> >> def is_scm_available(self): >> + # This did not exist before 3.2, but since then it is >> + # mandatory for our purpose >> + try: >> + os.set_inheritable(self.get_sock_fd(), True) >> + except AttributeError: >> + pass > > Why did you decide to place this code inside is_scm_available()? > > For reference, this is the only caller of is_scm_available(): > > def send_fd_scm(self, fd_file_path): > # In iotest.py, the qmp should always use unix socket. > assert self._qmp.is_scm_available() > ... > > In addition to making a method called is_*() have an unexpected > side-effect, True. My idea was that a function that asks for SCM to be available might as well make it available. > the method won't be called at all if running with > debugging disabled. Well, I sure hope we don't disable debugging in the iotests. We use assert quite a number of times there. On the other hand, someone might want to use this outside of the iotests but I don't even know whether that works, considering the SCM helper program is part of the iotests. > I suggest simply placing the os.set_inheritable() call inside > send_fd_scm(), as close as possible to the subprocess.Popen() > call. Yes, I think you're right. I didn't want to put it into send_fd_scm(), because that method is only there to send some FD over QMP/SCM; it isn't really supposed to send the QMP socket FD somewhere. But sending something over QMP/SCM is different from bequeathing something to a child process (the socket_scm_helper). And since send_fd_scm() needs to bequeath the QMP socket FD to that helper, it should be responsible for making it inheritable. And maybe even more importantly, whether the socket allows for SCM really has nothing to do with whether it's inheritable. So it's actually just wrong to put it here. >> return self.__sock.family == socket.AF_UNIX >> diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 >> index d2081df84b..b58455645b 100755 >> --- a/tests/qemu-iotests/147 >> +++ b/tests/qemu-iotests/147 >> @@ -229,6 +229,13 @@ class BuiltinNBD(NBDBlockdevAddBase): >> sockfd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) >> sockfd.connect(unix_socket) >> >> + # This did not exist before 3.2, but since then it is >> + # mandatory for our purpose >> + try: >> + os.set_inheritable(sockfd.fileno(), True) >> + except AttributeError: >> + pass >> + > > Why not make send_fd_scm() responsible for calling > os.set_inheritable(), making this hunk unnecessary? My idea was: Because send_fd_scm() takes a string and not an integer. The socket_scm_helper takes either an FD or a path, so send_fd_scm() accepts either. But now I realize that if we pass a path, we don't need to make the FD inheritable. So send_fd_scm() can check whether it's supposed to pass an FD, and if so, make it inheritable, yes. Max >> result = self.vm.send_fd_scm(str(sockfd.fileno())) >> self.assertEqual(result, 0, 'Failed to send socket FD') >> >> -- >> 2.17.1 >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python 2018-10-15 14:14 ` [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python Max Reitz 2018-10-15 20:30 ` Eduardo Habkost @ 2018-10-15 23:18 ` Cleber Rosa 2018-10-19 9:43 ` Max Reitz 1 sibling, 1 reply; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 23:18 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > Python 3.2 introduced the inheritable attribute for FDs. At the same > time, it changed the default so that all FDs are not inheritable by > default, that only inheritable FDs are inherited to subprocesses, and > only if close_fds is explicitly set to False. > It's actually a change that was introduced in 3.4: https://docs.python.org/3/library/os.html#inheritance-of-file-descriptors > Adhere to this by setting close_fds to False when working with > subprocesses that may want to inherit FDs, and by trying to > set_inheritable() on FDs that we do want to bequeath to them. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > scripts/qemu.py | 13 +++++++++++-- > scripts/qmp/qmp.py | 7 +++++++ > tests/qemu-iotests/147 | 7 +++++++ > 3 files changed, 25 insertions(+), 2 deletions(-) > > diff --git a/scripts/qemu.py b/scripts/qemu.py > index f099ce7278..28366c4a67 100644 > --- a/scripts/qemu.py > +++ b/scripts/qemu.py > @@ -142,10 +142,18 @@ class QEMUMachine(object): > if opts: > options.append(opts) > > + # This did not exist before 3.2, but since then it is > + # mandatory for our purpose > + try: Version should be 3.4 here as well. > + os.set_inheritable(fd, True) > + except AttributeError: > + pass > + Doing hasattr(os, "set_inheritable") is cheaper. - Cleber. > self._args.append('-add-fd') > self._args.append(','.join(options)) > return self > > + # The caller needs to make sure the FD is inheritable > def send_fd_scm(self, fd_file_path): > # In iotest.py, the qmp should always use unix socket. > assert self._qmp.is_scm_available() > @@ -159,7 +167,7 @@ class QEMUMachine(object): > "%s" % fd_file_path] > devnull = open(os.path.devnull, 'rb') > proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE, > - stderr=subprocess.STDOUT) > + stderr=subprocess.STDOUT, close_fds=False) > output = proc.communicate()[0] > if output: > LOG.debug(output) > @@ -280,7 +288,8 @@ class QEMUMachine(object): > stdin=devnull, > stdout=self._qemu_log_file, > stderr=subprocess.STDOUT, > - shell=False) > + shell=False, > + close_fds=False) > self._post_launch() > > def wait(self): > diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py > index 5c8cf6a056..009be8345b 100644 > --- a/scripts/qmp/qmp.py > +++ b/scripts/qmp/qmp.py > @@ -10,6 +10,7 @@ > > import json > import errno > +import os > import socket > import logging > > @@ -253,4 +254,10 @@ class QEMUMonitorProtocol(object): > return self.__sock.fileno() > > def is_scm_available(self): > + # This did not exist before 3.2, but since then it is > + # mandatory for our purpose > + try: > + os.set_inheritable(self.get_sock_fd(), True) > + except AttributeError: > + pass > return self.__sock.family == socket.AF_UNIX > diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 > index d2081df84b..b58455645b 100755 > --- a/tests/qemu-iotests/147 > +++ b/tests/qemu-iotests/147 > @@ -229,6 +229,13 @@ class BuiltinNBD(NBDBlockdevAddBase): > sockfd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) > sockfd.connect(unix_socket) > > + # This did not exist before 3.2, but since then it is > + # mandatory for our purpose > + try: > + os.set_inheritable(sockfd.fileno(), True) > + except AttributeError: > + pass > + > result = self.vm.send_fd_scm(str(sockfd.fileno())) > self.assertEqual(result, 0, 'Failed to send socket FD') > > ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python 2018-10-15 23:18 ` Cleber Rosa @ 2018-10-19 9:43 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:43 UTC (permalink / raw) To: Cleber Rosa, qemu-block; +Cc: Kevin Wolf, qemu-devel, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 4473 bytes --] On 16.10.18 01:18, Cleber Rosa wrote: > > > On 10/15/18 10:14 AM, Max Reitz wrote: >> Python 3.2 introduced the inheritable attribute for FDs. At the same >> time, it changed the default so that all FDs are not inheritable by >> default, that only inheritable FDs are inherited to subprocesses, and >> only if close_fds is explicitly set to False. >> > > It's actually a change that was introduced in 3.4: > > https://docs.python.org/3/library/os.html#inheritance-of-file-descriptors Oops, don't know how I got that wrong. >> Adhere to this by setting close_fds to False when working with >> subprocesses that may want to inherit FDs, and by trying to >> set_inheritable() on FDs that we do want to bequeath to them. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> scripts/qemu.py | 13 +++++++++++-- >> scripts/qmp/qmp.py | 7 +++++++ >> tests/qemu-iotests/147 | 7 +++++++ >> 3 files changed, 25 insertions(+), 2 deletions(-) >> >> diff --git a/scripts/qemu.py b/scripts/qemu.py >> index f099ce7278..28366c4a67 100644 >> --- a/scripts/qemu.py >> +++ b/scripts/qemu.py >> @@ -142,10 +142,18 @@ class QEMUMachine(object): >> if opts: >> options.append(opts) >> >> + # This did not exist before 3.2, but since then it is >> + # mandatory for our purpose >> + try: > > Version should be 3.4 here as well. Yes, will fix everywhere. >> + os.set_inheritable(fd, True) >> + except AttributeError: >> + pass >> + > > Doing hasattr(os, "set_inheritable") is cheaper. > > - Cleber. Nice, I'll use that then. Max >> self._args.append('-add-fd') >> self._args.append(','.join(options)) >> return self >> >> + # The caller needs to make sure the FD is inheritable >> def send_fd_scm(self, fd_file_path): >> # In iotest.py, the qmp should always use unix socket. >> assert self._qmp.is_scm_available() >> @@ -159,7 +167,7 @@ class QEMUMachine(object): >> "%s" % fd_file_path] >> devnull = open(os.path.devnull, 'rb') >> proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE, >> - stderr=subprocess.STDOUT) >> + stderr=subprocess.STDOUT, close_fds=False) >> output = proc.communicate()[0] >> if output: >> LOG.debug(output) >> @@ -280,7 +288,8 @@ class QEMUMachine(object): >> stdin=devnull, >> stdout=self._qemu_log_file, >> stderr=subprocess.STDOUT, >> - shell=False) >> + shell=False, >> + close_fds=False) >> self._post_launch() >> >> def wait(self): >> diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py >> index 5c8cf6a056..009be8345b 100644 >> --- a/scripts/qmp/qmp.py >> +++ b/scripts/qmp/qmp.py >> @@ -10,6 +10,7 @@ >> >> import json >> import errno >> +import os >> import socket >> import logging >> >> @@ -253,4 +254,10 @@ class QEMUMonitorProtocol(object): >> return self.__sock.fileno() >> >> def is_scm_available(self): >> + # This did not exist before 3.2, but since then it is >> + # mandatory for our purpose >> + try: >> + os.set_inheritable(self.get_sock_fd(), True) >> + except AttributeError: >> + pass >> return self.__sock.family == socket.AF_UNIX >> diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 >> index d2081df84b..b58455645b 100755 >> --- a/tests/qemu-iotests/147 >> +++ b/tests/qemu-iotests/147 >> @@ -229,6 +229,13 @@ class BuiltinNBD(NBDBlockdevAddBase): >> sockfd = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) >> sockfd.connect(unix_socket) >> >> + # This did not exist before 3.2, but since then it is >> + # mandatory for our purpose >> + try: >> + os.set_inheritable(sockfd.fileno(), True) >> + except AttributeError: >> + pass >> + >> result = self.vm.send_fd_scm(str(sockfd.fileno())) >> self.assertEqual(result, 0, 'Failed to send socket FD') >> >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (5 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 21:13 ` Eduardo Habkost 2018-10-15 23:38 ` Cleber Rosa 2018-10-15 14:14 ` [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 Max Reitz ` (2 subsequent siblings) 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa iotest 169 uses the 'new' module to add methods to a class. This module no longer exists in Python 3. Instead, we can use a lambda. Best of all, this works in 2.7 just as well. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/169 | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index f243db9955..e5614b159d 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -23,7 +23,6 @@ import iotests import time import itertools import operator -import new from iotests import qemu_img @@ -144,7 +143,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): def inject_test_case(klass, name, method, *args, **kwargs): mc = operator.methodcaller(method, *args, **kwargs) - setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) + setattr(klass, 'test_' + name, lambda self: mc(self)) for cmb in list(itertools.product((True, False), repeat=4)): name = ('_' if cmb[0] else '_not_') + 'persistent_' -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-15 14:14 ` [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 Max Reitz @ 2018-10-15 21:13 ` Eduardo Habkost 2018-10-15 23:38 ` Cleber Rosa 1 sibling, 0 replies; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 21:13 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:51PM +0200, Max Reitz wrote: > iotest 169 uses the 'new' module to add methods to a class. This module > no longer exists in Python 3. Instead, we can use a lambda. Best of > all, this works in 2.7 just as well. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/169 | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 > index f243db9955..e5614b159d 100755 > --- a/tests/qemu-iotests/169 > +++ b/tests/qemu-iotests/169 > @@ -23,7 +23,6 @@ import iotests > import time > import itertools > import operator > -import new > from iotests import qemu_img > > > @@ -144,7 +143,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): > > def inject_test_case(klass, name, method, *args, **kwargs): > mc = operator.methodcaller(method, *args, **kwargs) > - setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) > + setattr(klass, 'test_' + name, lambda self: mc(self)) The "lambda self: mc(self)" expression looks weird and unnecessary at first look, but I have just confirmed that this doesn't work: setattr(klass, 'test_' + name, mc) Probably because the methodcaller object won't automatically become a instance method after the TestCase class is instantiated. Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> > > for cmb in list(itertools.product((True, False), repeat=4)): > name = ('_' if cmb[0] else '_not_') + 'persistent_' > -- > 2.17.1 > -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-15 14:14 ` [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 Max Reitz 2018-10-15 21:13 ` Eduardo Habkost @ 2018-10-15 23:38 ` Cleber Rosa 2018-10-15 23:57 ` Eduardo Habkost 1 sibling, 1 reply; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 23:38 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: qemu-devel, Kevin Wolf, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > iotest 169 uses the 'new' module to add methods to a class. This module > no longer exists in Python 3. Instead, we can use a lambda. Best of > all, this works in 2.7 just as well. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/169 | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 > index f243db9955..e5614b159d 100755 > --- a/tests/qemu-iotests/169 > +++ b/tests/qemu-iotests/169 > @@ -23,7 +23,6 @@ import iotests > import time > import itertools > import operator > -import new > from iotests import qemu_img > > > @@ -144,7 +143,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): > > def inject_test_case(klass, name, method, *args, **kwargs): > mc = operator.methodcaller(method, *args, **kwargs) > - setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) > + setattr(klass, 'test_' + name, lambda self: mc(self)) > > for cmb in list(itertools.product((True, False), repeat=4)): > name = ('_' if cmb[0] else '_not_') + 'persistent_' > This can be simplified with: diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index e5614b159d..2199f14ae7 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -22,7 +22,6 @@ import os import iotests import time import itertools -import operator from iotests import qemu_img @@ -141,18 +140,15 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.check_bitmap(self.vm_b, sha256 if persistent else False) -def inject_test_case(klass, name, method, *args, **kwargs): - mc = operator.methodcaller(method, *args, **kwargs) - setattr(klass, 'test_' + name, lambda self: mc(self)) - for cmb in list(itertools.product((True, False), repeat=4)): name = ('_' if cmb[0] else '_not_') + 'persistent_' name += ('_' if cmb[1] else '_not_') + 'migbitmap_' name += '_online' if cmb[2] else '_offline' name += '_shared' if cmb[3] else '_nonshared' - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', - *list(cmb)) + setattr(TestDirtyBitmapMigration, 'test_' + name, + lambda self: TestDirtyBitmapMigration.do_test_migration( + self, *list(cmb))) if __name__ == '__main__': ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-15 23:38 ` Cleber Rosa @ 2018-10-15 23:57 ` Eduardo Habkost 2018-10-16 1:01 ` Cleber Rosa 0 siblings, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 23:57 UTC (permalink / raw) To: Cleber Rosa; +Cc: Max Reitz, qemu-block, qemu-devel, Kevin Wolf On Mon, Oct 15, 2018 at 07:38:45PM -0400, Cleber Rosa wrote: > > > On 10/15/18 10:14 AM, Max Reitz wrote: > > iotest 169 uses the 'new' module to add methods to a class. This module > > no longer exists in Python 3. Instead, we can use a lambda. Best of > > all, this works in 2.7 just as well. > > > > Signed-off-by: Max Reitz <mreitz@redhat.com> > > --- > > tests/qemu-iotests/169 | 3 +-- > > 1 file changed, 1 insertion(+), 2 deletions(-) > > > > diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 > > index f243db9955..e5614b159d 100755 > > --- a/tests/qemu-iotests/169 > > +++ b/tests/qemu-iotests/169 > > @@ -23,7 +23,6 @@ import iotests > > import time > > import itertools > > import operator > > -import new > > from iotests import qemu_img > > > > > > @@ -144,7 +143,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): > > > > def inject_test_case(klass, name, method, *args, **kwargs): > > mc = operator.methodcaller(method, *args, **kwargs) > > - setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) > > + setattr(klass, 'test_' + name, lambda self: mc(self)) > > > > for cmb in list(itertools.product((True, False), repeat=4)): > > name = ('_' if cmb[0] else '_not_') + 'persistent_' > > > > This can be simplified with: > > diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 > index e5614b159d..2199f14ae7 100755 > --- a/tests/qemu-iotests/169 > +++ b/tests/qemu-iotests/169 > @@ -22,7 +22,6 @@ import os > import iotests > import time > import itertools > -import operator > from iotests import qemu_img > > > @@ -141,18 +140,15 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): > self.check_bitmap(self.vm_b, sha256 if persistent else False) > > > -def inject_test_case(klass, name, method, *args, **kwargs): > - mc = operator.methodcaller(method, *args, **kwargs) > - setattr(klass, 'test_' + name, lambda self: mc(self)) > - > for cmb in list(itertools.product((True, False), repeat=4)): > name = ('_' if cmb[0] else '_not_') + 'persistent_' > name += ('_' if cmb[1] else '_not_') + 'migbitmap_' > name += '_online' if cmb[2] else '_offline' > name += '_shared' if cmb[3] else '_nonshared' > > - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', > - *list(cmb)) > + setattr(TestDirtyBitmapMigration, 'test_' + name, > + lambda self: TestDirtyBitmapMigration.do_test_migration( > + self, *list(cmb))) I'm not fond of the long multi-line lambda expression, but I love that you got rid of operator.methodcaller(). :) However, this doesn't seem to work: `*list(cmb)` will be evaluated only when the lambda is called, and `cmb` will be always `(False, False, False, False)`. I was going to suggest defining a nested function inside inject_test_case() to replace operator.methodcaller(), but then I thought it was not worth it. -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-15 23:57 ` Eduardo Habkost @ 2018-10-16 1:01 ` Cleber Rosa 2018-10-19 9:46 ` Max Reitz 0 siblings, 1 reply; 48+ messages in thread From: Cleber Rosa @ 2018-10-16 1:01 UTC (permalink / raw) To: Eduardo Habkost; +Cc: Kevin Wolf, qemu-devel, qemu-block, Max Reitz On 10/15/18 7:57 PM, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 07:38:45PM -0400, Cleber Rosa wrote: >> >> >> On 10/15/18 10:14 AM, Max Reitz wrote: >>> iotest 169 uses the 'new' module to add methods to a class. This module >>> no longer exists in Python 3. Instead, we can use a lambda. Best of >>> all, this works in 2.7 just as well. >>> >>> Signed-off-by: Max Reitz <mreitz@redhat.com> >>> --- >>> tests/qemu-iotests/169 | 3 +-- >>> 1 file changed, 1 insertion(+), 2 deletions(-) >>> >>> diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 >>> index f243db9955..e5614b159d 100755 >>> --- a/tests/qemu-iotests/169 >>> +++ b/tests/qemu-iotests/169 >>> @@ -23,7 +23,6 @@ import iotests >>> import time >>> import itertools >>> import operator >>> -import new >>> from iotests import qemu_img >>> >>> >>> @@ -144,7 +143,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): >>> >>> def inject_test_case(klass, name, method, *args, **kwargs): >>> mc = operator.methodcaller(method, *args, **kwargs) >>> - setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) >>> + setattr(klass, 'test_' + name, lambda self: mc(self)) >>> >>> for cmb in list(itertools.product((True, False), repeat=4)): >>> name = ('_' if cmb[0] else '_not_') + 'persistent_' >>> >> >> This can be simplified with: >> >> diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 >> index e5614b159d..2199f14ae7 100755 >> --- a/tests/qemu-iotests/169 >> +++ b/tests/qemu-iotests/169 >> @@ -22,7 +22,6 @@ import os >> import iotests >> import time >> import itertools >> -import operator >> from iotests import qemu_img >> >> >> @@ -141,18 +140,15 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): >> self.check_bitmap(self.vm_b, sha256 if persistent else False) >> >> >> -def inject_test_case(klass, name, method, *args, **kwargs): >> - mc = operator.methodcaller(method, *args, **kwargs) >> - setattr(klass, 'test_' + name, lambda self: mc(self)) >> - >> for cmb in list(itertools.product((True, False), repeat=4)): >> name = ('_' if cmb[0] else '_not_') + 'persistent_' >> name += ('_' if cmb[1] else '_not_') + 'migbitmap_' >> name += '_online' if cmb[2] else '_offline' >> name += '_shared' if cmb[3] else '_nonshared' >> >> - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', >> - *list(cmb)) >> + setattr(TestDirtyBitmapMigration, 'test_' + name, >> + lambda self: TestDirtyBitmapMigration.do_test_migration( >> + self, *list(cmb))) > > I'm not fond of the long multi-line lambda expression, but I love > that you got rid of operator.methodcaller(). :) > > However, this doesn't seem to work: `*list(cmb)` will be > evaluated only when the lambda is called, and `cmb` will be > always `(False, False, False, False)`. > > I was going to suggest defining a nested function inside > inject_test_case() to replace operator.methodcaller(), but then I > thought it was not worth it. > You're right! I missed that. Anyway, another possibility: diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 index e5614b159d..cd0d9c289c 100755 --- a/tests/qemu-iotests/169 +++ b/tests/qemu-iotests/169 @@ -22,7 +22,7 @@ import os import iotests import time import itertools -import operator +import functools from iotests import qemu_img @@ -140,20 +140,15 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): self.vm_b.launch() self.check_bitmap(self.vm_b, sha256 if persistent else False) - -def inject_test_case(klass, name, method, *args, **kwargs): - mc = operator.methodcaller(method, *args, **kwargs) - setattr(klass, 'test_' + name, lambda self: mc(self)) - -for cmb in list(itertools.product((True, False), repeat=4)): +for cmb in itertools.product((True, False), repeat=4): name = ('_' if cmb[0] else '_not_') + 'persistent_' name += ('_' if cmb[1] else '_not_') + 'migbitmap_' name += '_online' if cmb[2] else '_offline' name += '_shared' if cmb[3] else '_nonshared' - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', - *list(cmb)) - + test = functools.partialmethod(TestDirtyBitmapMigration.do_test_migration, + *cmb) + setattr(TestDirtyBitmapMigration, 'test_' + name, test) if __name__ == '__main__': iotests.main(supported_fmts=['qcow2']) ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-16 1:01 ` Cleber Rosa @ 2018-10-19 9:46 ` Max Reitz 2018-10-19 14:18 ` Eduardo Habkost 0 siblings, 1 reply; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:46 UTC (permalink / raw) To: Cleber Rosa, Eduardo Habkost; +Cc: Kevin Wolf, qemu-devel, qemu-block [-- Attachment #1: Type: text/plain, Size: 4953 bytes --] On 16.10.18 03:01, Cleber Rosa wrote: > > > On 10/15/18 7:57 PM, Eduardo Habkost wrote: >> On Mon, Oct 15, 2018 at 07:38:45PM -0400, Cleber Rosa wrote: >>> >>> >>> On 10/15/18 10:14 AM, Max Reitz wrote: >>>> iotest 169 uses the 'new' module to add methods to a class. This module >>>> no longer exists in Python 3. Instead, we can use a lambda. Best of >>>> all, this works in 2.7 just as well. >>>> >>>> Signed-off-by: Max Reitz <mreitz@redhat.com> >>>> --- >>>> tests/qemu-iotests/169 | 3 +-- >>>> 1 file changed, 1 insertion(+), 2 deletions(-) >>>> >>>> diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 >>>> index f243db9955..e5614b159d 100755 >>>> --- a/tests/qemu-iotests/169 >>>> +++ b/tests/qemu-iotests/169 >>>> @@ -23,7 +23,6 @@ import iotests >>>> import time >>>> import itertools >>>> import operator >>>> -import new >>>> from iotests import qemu_img >>>> >>>> >>>> @@ -144,7 +143,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): >>>> >>>> def inject_test_case(klass, name, method, *args, **kwargs): >>>> mc = operator.methodcaller(method, *args, **kwargs) >>>> - setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) >>>> + setattr(klass, 'test_' + name, lambda self: mc(self)) >>>> >>>> for cmb in list(itertools.product((True, False), repeat=4)): >>>> name = ('_' if cmb[0] else '_not_') + 'persistent_' >>>> >>> >>> This can be simplified with: >>> >>> diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 >>> index e5614b159d..2199f14ae7 100755 >>> --- a/tests/qemu-iotests/169 >>> +++ b/tests/qemu-iotests/169 >>> @@ -22,7 +22,6 @@ import os >>> import iotests >>> import time >>> import itertools >>> -import operator >>> from iotests import qemu_img >>> >>> >>> @@ -141,18 +140,15 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): >>> self.check_bitmap(self.vm_b, sha256 if persistent else False) >>> >>> >>> -def inject_test_case(klass, name, method, *args, **kwargs): >>> - mc = operator.methodcaller(method, *args, **kwargs) >>> - setattr(klass, 'test_' + name, lambda self: mc(self)) >>> - >>> for cmb in list(itertools.product((True, False), repeat=4)): >>> name = ('_' if cmb[0] else '_not_') + 'persistent_' >>> name += ('_' if cmb[1] else '_not_') + 'migbitmap_' >>> name += '_online' if cmb[2] else '_offline' >>> name += '_shared' if cmb[3] else '_nonshared' >>> >>> - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', >>> - *list(cmb)) >>> + setattr(TestDirtyBitmapMigration, 'test_' + name, >>> + lambda self: TestDirtyBitmapMigration.do_test_migration( >>> + self, *list(cmb))) >> >> I'm not fond of the long multi-line lambda expression, but I love >> that you got rid of operator.methodcaller(). :) >> >> However, this doesn't seem to work: `*list(cmb)` will be >> evaluated only when the lambda is called, and `cmb` will be >> always `(False, False, False, False)`. >> >> I was going to suggest defining a nested function inside >> inject_test_case() to replace operator.methodcaller(), but then I >> thought it was not worth it. >> > > You're right! I missed that. Anyway, another possibility: > > diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 > index e5614b159d..cd0d9c289c 100755 > --- a/tests/qemu-iotests/169 > +++ b/tests/qemu-iotests/169 > @@ -22,7 +22,7 @@ import os > import iotests > import time > import itertools > -import operator > +import functools > from iotests import qemu_img > > > @@ -140,20 +140,15 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): > self.vm_b.launch() > self.check_bitmap(self.vm_b, sha256 if persistent else False) > > - > -def inject_test_case(klass, name, method, *args, **kwargs): > - mc = operator.methodcaller(method, *args, **kwargs) > - setattr(klass, 'test_' + name, lambda self: mc(self)) > - > -for cmb in list(itertools.product((True, False), repeat=4)): > +for cmb in itertools.product((True, False), repeat=4): > name = ('_' if cmb[0] else '_not_') + 'persistent_' > name += ('_' if cmb[1] else '_not_') + 'migbitmap_' > name += '_online' if cmb[2] else '_offline' > name += '_shared' if cmb[3] else '_nonshared' > > - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', > - *list(cmb)) > - > + test = > functools.partialmethod(TestDirtyBitmapMigration.do_test_migration, > + *cmb) > + setattr(TestDirtyBitmapMigration, 'test_' + name, test) I'm not sure how that is any simpler, though (apart from the inlining). :-) I mean, my personal goal is not to touch this beyond what I need to because it's black magic to me anyway, so I'm happy with what works. Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 2018-10-19 9:46 ` Max Reitz @ 2018-10-19 14:18 ` Eduardo Habkost 0 siblings, 0 replies; 48+ messages in thread From: Eduardo Habkost @ 2018-10-19 14:18 UTC (permalink / raw) To: Max Reitz; +Cc: Cleber Rosa, Kevin Wolf, qemu-devel, qemu-block On Fri, Oct 19, 2018 at 11:46:59AM +0200, Max Reitz wrote: [...] > I mean, my personal goal is not to touch this beyond what I need to > because it's black magic to me anyway, so I'm happy with what works. I agree with this approach, and your original patch looks good enough to me. -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (6 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 18:59 ` Cleber Rosa 2018-10-15 21:17 ` Eduardo Habkost 2018-10-15 14:14 ` [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 Max Reitz 2018-10-15 22:19 ` [Qemu-devel] [PATCH 0/9] iotests: Make them work for both " Philippe Mathieu-Daudé 9 siblings, 2 replies; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa There are two imports that need to be modified when running the iotests under Python 3: One is StringIO, which no longer exists; instead, the StringIO class comes from the io module, so import it from there. The other is the ConfigParser, which has just been renamed to configparser. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/iotests.py | 8 ++++++-- tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 7ca94e9278..a64ea90fb4 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], # We need to filter out the time taken from the output so that qemu-iotest # can reliably diff the results against master output. - import StringIO + if sys.version_info.major >= 3: + from io import StringIO + else: + from StringIO import StringIO + if debug: output = sys.stdout verbosity = 2 sys.argv.remove('-d') else: - output = StringIO.StringIO() + output = StringIO() logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py index d45e2e0a6a..6b2d659dee 100755 --- a/tests/qemu-iotests/nbd-fault-injector.py +++ b/tests/qemu-iotests/nbd-fault-injector.py @@ -48,7 +48,10 @@ import sys import socket import struct import collections -import ConfigParser +if sys.version_info.major >= 3: + import configparser +else: + import ConfigParser as configparser FAKE_DISK_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB @@ -225,7 +228,7 @@ def parse_config(config): return rules def load_rules(filename): - config = ConfigParser.RawConfigParser() + config = configparser.RawConfigParser() with open(filename, 'rt') as f: config.readfp(f, filename) return parse_config(config) -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-15 14:14 ` [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 Max Reitz @ 2018-10-15 18:59 ` Cleber Rosa 2018-10-15 20:15 ` Eduardo Habkost 2018-10-19 8:44 ` Max Reitz 2018-10-15 21:17 ` Eduardo Habkost 1 sibling, 2 replies; 48+ messages in thread From: Cleber Rosa @ 2018-10-15 18:59 UTC (permalink / raw) To: Max Reitz, qemu-block; +Cc: qemu-devel, Kevin Wolf, Eduardo Habkost On 10/15/18 10:14 AM, Max Reitz wrote: > There are two imports that need to be modified when running the iotests > under Python 3: One is StringIO, which no longer exists; instead, the > StringIO class comes from the io module, so import it from there. The > other is the ConfigParser, which has just been renamed to configparser. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/iotests.py | 8 ++++++-- > tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- > 2 files changed, 11 insertions(+), 4 deletions(-) > > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > index 7ca94e9278..a64ea90fb4 100644 > --- a/tests/qemu-iotests/iotests.py > +++ b/tests/qemu-iotests/iotests.py > @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], > > # We need to filter out the time taken from the output so that qemu-iotest > # can reliably diff the results against master output. > - import StringIO > + if sys.version_info.major >= 3: > + from io import StringIO > + else: > + from StringIO import StringIO > + > if debug: > output = sys.stdout > verbosity = 2 > sys.argv.remove('-d') > else: > - output = StringIO.StringIO() > + output = StringIO() > > logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) > > diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py > index d45e2e0a6a..6b2d659dee 100755 > --- a/tests/qemu-iotests/nbd-fault-injector.py > +++ b/tests/qemu-iotests/nbd-fault-injector.py > @@ -48,7 +48,10 @@ import sys > import socket > import struct > import collections > -import ConfigParser > +if sys.version_info.major >= 3: > + import configparser > +else: > + import ConfigParser as configparser > > FAKE_DISK_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB > > @@ -225,7 +228,7 @@ def parse_config(config): > return rules > > def load_rules(filename): > - config = ConfigParser.RawConfigParser() > + config = configparser.RawConfigParser() > with open(filename, 'rt') as f: > config.readfp(f, filename) > return parse_config(config) > This may be a type of culture clash (on my side, due to not enough QEMU culture), but shouldn't this be applied before anything else on this series? I mean, PATCH 1/9 is supposed to fix the reliability aspects of nbd-fault-injector under Python 3, but without this patch, it won't actually run on Python 3. Regards, - Cleber. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-15 18:59 ` Cleber Rosa @ 2018-10-15 20:15 ` Eduardo Habkost 2018-10-19 8:44 ` Max Reitz 1 sibling, 0 replies; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 20:15 UTC (permalink / raw) To: Cleber Rosa; +Cc: Max Reitz, qemu-block, qemu-devel, Kevin Wolf On Mon, Oct 15, 2018 at 02:59:28PM -0400, Cleber Rosa wrote: > > > On 10/15/18 10:14 AM, Max Reitz wrote: > > There are two imports that need to be modified when running the iotests > > under Python 3: One is StringIO, which no longer exists; instead, the > > StringIO class comes from the io module, so import it from there. The > > other is the ConfigParser, which has just been renamed to configparser. > > > > Signed-off-by: Max Reitz <mreitz@redhat.com> > > --- > > tests/qemu-iotests/iotests.py | 8 ++++++-- > > tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- > > 2 files changed, 11 insertions(+), 4 deletions(-) > > > > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > > index 7ca94e9278..a64ea90fb4 100644 > > --- a/tests/qemu-iotests/iotests.py > > +++ b/tests/qemu-iotests/iotests.py > > @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], > > > > # We need to filter out the time taken from the output so that qemu-iotest > > # can reliably diff the results against master output. > > - import StringIO > > + if sys.version_info.major >= 3: > > + from io import StringIO > > + else: > > + from StringIO import StringIO > > + > > if debug: > > output = sys.stdout > > verbosity = 2 > > sys.argv.remove('-d') > > else: > > - output = StringIO.StringIO() > > + output = StringIO() > > > > logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) > > > > diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py > > index d45e2e0a6a..6b2d659dee 100755 > > --- a/tests/qemu-iotests/nbd-fault-injector.py > > +++ b/tests/qemu-iotests/nbd-fault-injector.py > > @@ -48,7 +48,10 @@ import sys > > import socket > > import struct > > import collections > > -import ConfigParser > > +if sys.version_info.major >= 3: > > + import configparser > > +else: > > + import ConfigParser as configparser > > > > FAKE_DISK_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB > > > > @@ -225,7 +228,7 @@ def parse_config(config): > > return rules > > > > def load_rules(filename): > > - config = ConfigParser.RawConfigParser() > > + config = configparser.RawConfigParser() > > with open(filename, 'rt') as f: > > config.readfp(f, filename) > > return parse_config(config) > > > > This may be a type of culture clash (on my side, due to not enough QEMU > culture), but shouldn't this be applied before anything else on this series? > > I mean, PATCH 1/9 is supposed to fix the reliability aspects of > nbd-fault-injector under Python 3, but without this patch, it won't > actually run on Python 3. Both patches are required to make the code run on Python 3, and they don't depend on each other. So I think the order doesn't matter. But I think the order chosen by Max is slightly more intuitive: having explicit mentions of Python 3 in the code would be confusing if we didn't fix the compatibility issues on patches 1-7 first. -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-15 18:59 ` Cleber Rosa 2018-10-15 20:15 ` Eduardo Habkost @ 2018-10-19 8:44 ` Max Reitz 1 sibling, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 8:44 UTC (permalink / raw) To: Cleber Rosa, qemu-block; +Cc: qemu-devel, Kevin Wolf, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 2944 bytes --] On 15.10.18 20:59, Cleber Rosa wrote: > > > On 10/15/18 10:14 AM, Max Reitz wrote: >> There are two imports that need to be modified when running the iotests >> under Python 3: One is StringIO, which no longer exists; instead, the >> StringIO class comes from the io module, so import it from there. The >> other is the ConfigParser, which has just been renamed to configparser. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/iotests.py | 8 ++++++-- >> tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- >> 2 files changed, 11 insertions(+), 4 deletions(-) >> >> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py >> index 7ca94e9278..a64ea90fb4 100644 >> --- a/tests/qemu-iotests/iotests.py >> +++ b/tests/qemu-iotests/iotests.py >> @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], >> >> # We need to filter out the time taken from the output so that qemu-iotest >> # can reliably diff the results against master output. >> - import StringIO >> + if sys.version_info.major >= 3: >> + from io import StringIO >> + else: >> + from StringIO import StringIO >> + >> if debug: >> output = sys.stdout >> verbosity = 2 >> sys.argv.remove('-d') >> else: >> - output = StringIO.StringIO() >> + output = StringIO() >> >> logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) >> >> diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py >> index d45e2e0a6a..6b2d659dee 100755 >> --- a/tests/qemu-iotests/nbd-fault-injector.py >> +++ b/tests/qemu-iotests/nbd-fault-injector.py >> @@ -48,7 +48,10 @@ import sys >> import socket >> import struct >> import collections >> -import ConfigParser >> +if sys.version_info.major >= 3: >> + import configparser >> +else: >> + import ConfigParser as configparser >> >> FAKE_DISK_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB >> >> @@ -225,7 +228,7 @@ def parse_config(config): >> return rules >> >> def load_rules(filename): >> - config = ConfigParser.RawConfigParser() >> + config = configparser.RawConfigParser() >> with open(filename, 'rt') as f: >> config.readfp(f, filename) >> return parse_config(config) >> > > This may be a type of culture clash (on my side, due to not enough QEMU > culture), but shouldn't this be applied before anything else on this series? > > I mean, PATCH 1/9 is supposed to fix the reliability aspects of > nbd-fault-injector under Python 3, but without this patch, it won't > actually run on Python 3. I don't mind, I followed no specific order. Well, patch 9 is patch 9 because it is the largest one, so I didn't want to discourage people before the end of the series. :-) Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-15 14:14 ` [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 Max Reitz 2018-10-15 18:59 ` Cleber Rosa @ 2018-10-15 21:17 ` Eduardo Habkost 2018-10-16 0:05 ` Cleber Rosa 1 sibling, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 21:17 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf, Cleber Rosa On Mon, Oct 15, 2018 at 04:14:52PM +0200, Max Reitz wrote: > There are two imports that need to be modified when running the iotests > under Python 3: One is StringIO, which no longer exists; instead, the > StringIO class comes from the io module, so import it from there. The > other is the ConfigParser, which has just been renamed to configparser. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/iotests.py | 8 ++++++-- > tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- > 2 files changed, 11 insertions(+), 4 deletions(-) > > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > index 7ca94e9278..a64ea90fb4 100644 > --- a/tests/qemu-iotests/iotests.py > +++ b/tests/qemu-iotests/iotests.py > @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], > > # We need to filter out the time taken from the output so that qemu-iotest > # can reliably diff the results against master output. > - import StringIO > + if sys.version_info.major >= 3: > + from io import StringIO > + else: > + from StringIO import StringIO Considering that io.StringIO exists on Python 2.7, a comment explaining why exactly it doesn't work would be nice. But this shouldn't block this workaround, so: Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> > + > if debug: > output = sys.stdout > verbosity = 2 > sys.argv.remove('-d') > else: > - output = StringIO.StringIO() > + output = StringIO() > > logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) > > diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py > index d45e2e0a6a..6b2d659dee 100755 > --- a/tests/qemu-iotests/nbd-fault-injector.py > +++ b/tests/qemu-iotests/nbd-fault-injector.py > @@ -48,7 +48,10 @@ import sys > import socket > import struct > import collections > -import ConfigParser > +if sys.version_info.major >= 3: > + import configparser > +else: > + import ConfigParser as configparser > > FAKE_DISK_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB > > @@ -225,7 +228,7 @@ def parse_config(config): > return rules > > def load_rules(filename): > - config = ConfigParser.RawConfigParser() > + config = configparser.RawConfigParser() > with open(filename, 'rt') as f: > config.readfp(f, filename) > return parse_config(config) > -- > 2.17.1 > -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-15 21:17 ` Eduardo Habkost @ 2018-10-16 0:05 ` Cleber Rosa 2018-10-16 0:12 ` Eduardo Habkost 0 siblings, 1 reply; 48+ messages in thread From: Cleber Rosa @ 2018-10-16 0:05 UTC (permalink / raw) To: Eduardo Habkost, Max Reitz; +Cc: qemu-block, qemu-devel, Kevin Wolf On 10/15/18 5:17 PM, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 04:14:52PM +0200, Max Reitz wrote: >> There are two imports that need to be modified when running the iotests >> under Python 3: One is StringIO, which no longer exists; instead, the >> StringIO class comes from the io module, so import it from there. The >> other is the ConfigParser, which has just been renamed to configparser. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/iotests.py | 8 ++++++-- >> tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- >> 2 files changed, 11 insertions(+), 4 deletions(-) >> >> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py >> index 7ca94e9278..a64ea90fb4 100644 >> --- a/tests/qemu-iotests/iotests.py >> +++ b/tests/qemu-iotests/iotests.py >> @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], >> >> # We need to filter out the time taken from the output so that qemu-iotest >> # can reliably diff the results against master output. >> - import StringIO >> + if sys.version_info.major >= 3: >> + from io import StringIO >> + else: >> + from StringIO import StringIO > > Considering that io.StringIO exists on Python 2.7, a comment > explaining why exactly it doesn't work would be nice. > Another possibility, that I find self explanatory: import io if sys.version_info.major >= 3: output = io.StringIO() else: output = io.BytesIO() - Cleber. > But this shouldn't block this workaround, so: > > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> > > >> + >> if debug: >> output = sys.stdout >> verbosity = 2 >> sys.argv.remove('-d') >> else: >> - output = StringIO.StringIO() >> + output = StringIO() >> >> logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) >> >> diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py >> index d45e2e0a6a..6b2d659dee 100755 >> --- a/tests/qemu-iotests/nbd-fault-injector.py >> +++ b/tests/qemu-iotests/nbd-fault-injector.py >> @@ -48,7 +48,10 @@ import sys >> import socket >> import struct >> import collections >> -import ConfigParser >> +if sys.version_info.major >= 3: >> + import configparser >> +else: >> + import ConfigParser as configparser >> >> FAKE_DISK_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB >> >> @@ -225,7 +228,7 @@ def parse_config(config): >> return rules >> >> def load_rules(filename): >> - config = ConfigParser.RawConfigParser() >> + config = configparser.RawConfigParser() >> with open(filename, 'rt') as f: >> config.readfp(f, filename) >> return parse_config(config) >> -- >> 2.17.1 >> > ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-16 0:05 ` Cleber Rosa @ 2018-10-16 0:12 ` Eduardo Habkost 2018-10-19 9:25 ` Max Reitz 0 siblings, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-16 0:12 UTC (permalink / raw) To: Cleber Rosa; +Cc: Max Reitz, qemu-block, qemu-devel, Kevin Wolf On Mon, Oct 15, 2018 at 08:05:02PM -0400, Cleber Rosa wrote: > > > On 10/15/18 5:17 PM, Eduardo Habkost wrote: > > On Mon, Oct 15, 2018 at 04:14:52PM +0200, Max Reitz wrote: > >> There are two imports that need to be modified when running the iotests > >> under Python 3: One is StringIO, which no longer exists; instead, the > >> StringIO class comes from the io module, so import it from there. The > >> other is the ConfigParser, which has just been renamed to configparser. > >> > >> Signed-off-by: Max Reitz <mreitz@redhat.com> > >> --- > >> tests/qemu-iotests/iotests.py | 8 ++++++-- > >> tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- > >> 2 files changed, 11 insertions(+), 4 deletions(-) > >> > >> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > >> index 7ca94e9278..a64ea90fb4 100644 > >> --- a/tests/qemu-iotests/iotests.py > >> +++ b/tests/qemu-iotests/iotests.py > >> @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], > >> > >> # We need to filter out the time taken from the output so that qemu-iotest > >> # can reliably diff the results against master output. > >> - import StringIO > >> + if sys.version_info.major >= 3: > >> + from io import StringIO > >> + else: > >> + from StringIO import StringIO > > > > Considering that io.StringIO exists on Python 2.7, a comment > > explaining why exactly it doesn't work would be nice. > > > > Another possibility, that I find self explanatory: > > import io > > if sys.version_info.major >= 3: > output = io.StringIO() > else: > output = io.BytesIO() Looks nice and clean. But I'm not sure all output sent to `output` when running with Python 2 will be byte strings. What if `unittest.TextTestRunner` tries to write unicode strings to `output`? -- Eduardo ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 2018-10-16 0:12 ` Eduardo Habkost @ 2018-10-19 9:25 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:25 UTC (permalink / raw) To: Eduardo Habkost, Cleber Rosa; +Cc: qemu-block, qemu-devel, Kevin Wolf [-- Attachment #1: Type: text/plain, Size: 2496 bytes --] On 16.10.18 02:12, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 08:05:02PM -0400, Cleber Rosa wrote: >> >> >> On 10/15/18 5:17 PM, Eduardo Habkost wrote: >>> On Mon, Oct 15, 2018 at 04:14:52PM +0200, Max Reitz wrote: >>>> There are two imports that need to be modified when running the iotests >>>> under Python 3: One is StringIO, which no longer exists; instead, the >>>> StringIO class comes from the io module, so import it from there. The >>>> other is the ConfigParser, which has just been renamed to configparser. >>>> >>>> Signed-off-by: Max Reitz <mreitz@redhat.com> >>>> --- >>>> tests/qemu-iotests/iotests.py | 8 ++++++-- >>>> tests/qemu-iotests/nbd-fault-injector.py | 7 +++++-- >>>> 2 files changed, 11 insertions(+), 4 deletions(-) >>>> >>>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py >>>> index 7ca94e9278..a64ea90fb4 100644 >>>> --- a/tests/qemu-iotests/iotests.py >>>> +++ b/tests/qemu-iotests/iotests.py >>>> @@ -683,13 +683,17 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[], >>>> >>>> # We need to filter out the time taken from the output so that qemu-iotest >>>> # can reliably diff the results against master output. >>>> - import StringIO >>>> + if sys.version_info.major >= 3: >>>> + from io import StringIO >>>> + else: >>>> + from StringIO import StringIO >>> >>> Considering that io.StringIO exists on Python 2.7, a comment >>> explaining why exactly it doesn't work would be nice. Oh, it does exist? I didn't know. O:-) So I suppose it's because the test runner emits just normal strings, which in 2.x are byte strings; but io's StringIO always expects Unicode strings. StringIO, OTOH, accepts both (and returns Unicode strings once you put a Unicode string into it). >> Another possibility, that I find self explanatory: >> >> import io >> >> if sys.version_info.major >= 3: >> output = io.StringIO() >> else: >> output = io.BytesIO() > > Looks nice and clean. > > But I'm not sure all output sent to `output` when running with > Python 2 will be byte strings. What if `unittest.TextTestRunner` > tries to write unicode strings to `output`? Hm. It doesn't look this way to me in practice. And considering that it only really prints a test summary, I don't think there are edge cases where it behaves differently. So I think using BytesIO() in 2.x is better. Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (7 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 Max Reitz @ 2018-10-15 14:14 ` Max Reitz 2018-10-15 22:26 ` Eduardo Habkost 2018-10-15 22:19 ` [Qemu-devel] [PATCH 0/9] iotests: Make them work for both " Philippe Mathieu-Daudé 9 siblings, 1 reply; 48+ messages in thread From: Max Reitz @ 2018-10-15 14:14 UTC (permalink / raw) To: qemu-block Cc: qemu-devel, Max Reitz, Kevin Wolf, Eduardo Habkost, Cleber Rosa When dumping an object into the log, there are differences between Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2 (they are no longer in 3, because unicode strings are the default there). Second, the order of keys in dicts may differ. Third, especially long numbers are longs in Python 2 and thus get an 'L' suffix, which does not happen in Python 3. To get around these differences, this patch introduces functions to convert an object to a string that looks the same regardless of the Python version: In Python 2, they decode unicode strings to byte strings (normal str); in Python 3, they encode byte strings to unicode strings (normal str). They also manually convert lists and dicts to strings, which allows sorting the dicts by key, so we are no longer at the mercy of the internal implementation when it comes to how the keys appear in the output. This changes the output of all tests that use these logging functions. Signed-off-by: Max Reitz <mreitz@redhat.com> --- tests/qemu-iotests/194.out | 22 +- tests/qemu-iotests/202.out | 12 +- tests/qemu-iotests/203.out | 14 +- tests/qemu-iotests/206.out | 144 +++++----- tests/qemu-iotests/207.out | 52 ++-- tests/qemu-iotests/208.out | 8 +- tests/qemu-iotests/210.out | 72 ++--- tests/qemu-iotests/211.out | 66 ++--- tests/qemu-iotests/212.out | 102 +++---- tests/qemu-iotests/213.out | 124 ++++---- tests/qemu-iotests/216.out | 4 +- tests/qemu-iotests/218.out | 20 +- tests/qemu-iotests/219.out | 526 +++++++++++++++++----------------- tests/qemu-iotests/222.out | 24 +- tests/qemu-iotests/iotests.py | 42 ++- 15 files changed, 634 insertions(+), 598 deletions(-) diff --git a/tests/qemu-iotests/194.out b/tests/qemu-iotests/194.out index 50ac50da5e..722af888f6 100644 --- a/tests/qemu-iotests/194.out +++ b/tests/qemu-iotests/194.out @@ -1,18 +1,18 @@ Launching VMs... Launching NBD server on destination... -{u'return': {}} -{u'return': {}} +{'return': {}} +{'return': {}} Starting `drive-mirror` on source... -{u'return': {}} +{'return': {}} Waiting for `drive-mirror` to complete... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror-job0', u'type': u'mirror', u'speed': 0, u'len': 1073741824, u'offset': 1073741824}, u'event': u'BLOCK_JOB_READY'} +{'data': {'device': 'mirror-job0', 'len': 1073741824, 'offset': 1073741824, 'speed': 0, 'type': 'mirror'}, 'event': 'BLOCK_JOB_READY', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Starting migration... -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'setup'}, u'event': u'MIGRATION'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'active'}, u'event': u'MIGRATION'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'completed'}, u'event': u'MIGRATION'} +{'return': {}} +{'data': {'status': 'setup'}, 'event': 'MIGRATION', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'status': 'active'}, 'event': 'MIGRATION', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'status': 'completed'}, 'event': 'MIGRATION', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Gracefully ending the `drive-mirror` job on source... -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror-job0', u'type': u'mirror', u'speed': 0, u'len': 1073741824, u'offset': 1073741824}, u'event': u'BLOCK_JOB_COMPLETED'} +{'return': {}} +{'data': {'device': 'mirror-job0', 'len': 1073741824, 'offset': 1073741824, 'speed': 0, 'type': 'mirror'}, 'event': 'BLOCK_JOB_COMPLETED', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Stopping the NBD server on destination... -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/202.out b/tests/qemu-iotests/202.out index d5ea374e17..d133f09fe2 100644 --- a/tests/qemu-iotests/202.out +++ b/tests/qemu-iotests/202.out @@ -1,11 +1,11 @@ Launching VM... Adding IOThread... -{u'return': {}} +{'return': {}} Adding blockdevs... -{u'return': {}} -{u'return': {}} +{'return': {}} +{'return': {}} Setting iothread... -{u'return': {}} -{u'return': {}} +{'return': {}} +{'return': {}} Creating external snapshots... -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/203.out b/tests/qemu-iotests/203.out index 1a11f0975c..b156388a0b 100644 --- a/tests/qemu-iotests/203.out +++ b/tests/qemu-iotests/203.out @@ -1,11 +1,11 @@ Launching VM... Setting IOThreads... -{u'return': {}} -{u'return': {}} +{'return': {}} +{'return': {}} Enabling migration QMP events... -{u'return': {}} +{'return': {}} Starting migration... -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'setup'}, u'event': u'MIGRATION'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'active'}, u'event': u'MIGRATION'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'completed'}, u'event': u'MIGRATION'} +{'return': {}} +{'data': {'status': 'setup'}, 'event': 'MIGRATION', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'status': 'active'}, 'event': 'MIGRATION', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'status': 'completed'}, 'event': 'MIGRATION', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out index 789eebe57b..8fb3228cac 100644 --- a/tests/qemu-iotests/206.out +++ b/tests/qemu-iotests/206.out @@ -1,16 +1,16 @@ === Successful image creation (defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}} -{u'return': {}} +{'execute': 'blockdev-add', 'arguments': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2', 'node_name': 'imgfile'}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'imgfile', 'size': 134217728}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -24,15 +24,15 @@ Format specific information: === Successful image creation (inline blockdev-add, explicit defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'nocow': False, 'preallocation': 'off', 'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2', 'nocow': False, 'preallocation': 'off', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'refcount-bits': 16, 'version': 'v3', 'preallocation': 'off', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': False, 'driver': 'qcow2', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'driver': 'qcow2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': False, 'preallocation': 'off', 'refcount-bits': 16, 'size': 67108864, 'version': 'v3'}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -46,15 +46,15 @@ Format specific information: === Successful image creation (v3 non-default options) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'nocow': True, 'preallocation': 'falloc', 'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2', 'nocow': True, 'preallocation': 'falloc', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 2097152, 'refcount-bits': 1, 'version': 'v3', 'preallocation': 'metadata', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': True, 'driver': 'qcow2', 'size': 33554432}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 2097152, 'driver': 'qcow2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': True, 'preallocation': 'metadata', 'refcount-bits': 1, 'size': 33554432, 'version': 'v3'}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -68,15 +68,15 @@ Format specific information: === Successful image creation (v2 non-default options) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'backing-fmt': 'qcow2', 'driver': 'qcow2', 'version': 'v2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'backing-file': 'TEST_DIR/PID-t.qcow2.base', 'size': 33554432}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'backing-file': 'TEST_DIR/PID-t.qcow2.base', 'backing-fmt': 'qcow2', 'cluster-size': 512, 'driver': 'qcow2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'size': 33554432, 'version': 'v2'}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -90,10 +90,10 @@ Format specific information: === Successful image creation (encrypted) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'encrypt': {'key-secret': 'keysec0', 'iter-time': 10, 'cipher-mode': 'ctr', 'ivgen-hash-alg': 'md5', 'cipher-alg': 'twofish-128', 'format': 'luks', 'ivgen-alg': 'plain64', 'hash-alg': 'sha1'}, 'driver': 'qcow2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'size': 33554432}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'encrypt': {'cipher-alg': 'twofish-128', 'cipher-mode': 'ctr', 'format': 'luks', 'hash-alg': 'sha1', 'iter-time': 10, 'ivgen-alg': 'plain64', 'ivgen-hash-alg': 'md5', 'key-secret': 'keysec0'}, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'size': 33554432}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -145,112 +145,112 @@ Format specific information: === Invalid BlockdevRef === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': "this doesn't exist", 'size': 33554432}}} -{u'return': {}} +{'return': {}} Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid sizes === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 1234}}} -{u'return': {}} +{'return': {}} Job failed: Image size must be a multiple of 512 bytes {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 18446744073709551104L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 18446744073709551104}}} +{'return': {}} Job failed: Could not resize image: Image size cannot be negative {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775808L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775808}}} +{'return': {}} Job failed: Could not resize image: Image size cannot be negative {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775296}}} -{u'return': {}} +{'return': {}} Job failed: Could not resize image: Failed to grow the L1 table: File too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid version === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'version': 'v1', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'error': {u'class': u'GenericError', u'desc': u"Invalid parameter 'v1'"}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 67108864, 'version': 'v1'}}} +{'error': {'class': 'GenericError', 'desc': "Invalid parameter 'v1'"}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'lazy-refcounts': True, 'version': 'v2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'lazy-refcounts': True, 'size': 67108864, 'version': 'v2'}}} +{'return': {}} Job failed: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater) {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 8, 'version': 'v2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'refcount-bits': 8, 'size': 67108864, 'version': 'v2'}}} +{'return': {}} Job failed: Different refcount widths than 16 bits require compatibility level 1.1 or above (use version=v3 or greater) {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid backing file options === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'full', 'driver': 'qcow2', 'backing-file': '/dev/null', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'backing-file': '/dev/null', 'driver': 'qcow2', 'file': 'node0', 'preallocation': 'full', 'size': 67108864}}} +{'return': {}} Job failed: Backing file and preallocation cannot be used at the same time {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'backing-fmt': 'qcow2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Backing format cannot be used without backing file {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid cluster size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1234, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size must be a power of two between 512 and 2048k {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 128, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size must be a power of two between 512 and 2048k {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 4194304, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size must be a power of two between 512 and 2048k {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 0, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size must be a power of two between 512 and 2048k {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'driver': 'qcow2', 'file': 'node0', 'size': 281474976710656}}} -{u'return': {}} +{'return': {}} Job failed: Could not resize image: Failed to grow the L1 table: File too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid refcount width === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 128, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'refcount-bits': 128, 'size': 67108864}}} +{'return': {}} Job failed: Refcount width must be a power of two and may not exceed 64 bits {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 0, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'refcount-bits': 0, 'size': 67108864}}} +{'return': {}} Job failed: Refcount width must be a power of two and may not exceed 64 bits {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 7, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'refcount-bits': 7, 'size': 67108864}}} +{'return': {}} Job failed: Refcount width must be a power of two and may not exceed 64 bits {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out index 078b7e63cb..e18ff4e26a 100644 --- a/tests/qemu-iotests/207.out +++ b/tests/qemu-iotests/207.out @@ -1,9 +1,9 @@ === Successful image creation (defaults) === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} file format: IMGFMT @@ -16,49 +16,49 @@ virtual size: 4.0M (4194304 bytes) === Test host-key-check options === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'mode': 'none'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} file format: IMGFMT virtual size: 8.0M (8388608 bytes) -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'known_hosts'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'mode': 'known_hosts'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} file format: IMGFMT virtual size: 4.0M (4194304 bytes) -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'hash': 'wrong', 'mode': 'hash', 'type': 'md5'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} +{'return': {}} Job failed: remote host key does not match host_key_check 'wrong' {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'hash': HASH, 'mode': 'hash', 'type': 'md5'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} file format: IMGFMT virtual size: 8.0M (8388608 bytes) -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'hash': 'wrong', 'mode': 'hash', 'type': 'sha1'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} +{'return': {}} Job failed: remote host key does not match host_key_check 'wrong' {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'hash': HASH, 'mode': 'hash', 'type': 'sha1'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} file format: IMGFMT @@ -66,15 +66,15 @@ virtual size: 4.0M (4194304 bytes) === Invalid path and user === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': '/this/is/not/an/existing/path', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'mode': 'none'}, 'path': '/this/is/not/an/existing/path', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} +{'return': {}} Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31) {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'user': 'invalid user', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'host-key-check': {'mode': 'none'}, 'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}, 'user': 'invalid user'}, 'size': 4194304}}} +{'return': {}} Job failed: failed to authenticate using publickey authentication and the identities held by your ssh-agent {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/208.out b/tests/qemu-iotests/208.out index 3687e9d0dd..38c26457eb 100644 --- a/tests/qemu-iotests/208.out +++ b/tests/qemu-iotests/208.out @@ -1,9 +1,9 @@ Launching VM... Starting NBD server... -{u'return': {}} +{'return': {}} Adding NBD export... -{u'return': {}} +{'return': {}} Creating external snapshot... -{u'return': {}} +{'return': {}} Stopping NBD server... -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/210.out b/tests/qemu-iotests/210.out index 078ba544a1..49a8dcf268 100644 --- a/tests/qemu-iotests/210.out +++ b/tests/qemu-iotests/210.out @@ -1,16 +1,16 @@ === Successful image creation (defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}} -{u'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'iter-time': 10, 'driver': 'luks', 'file': 'imgfile', 'size': 134217728}}} -{u'return': {}} +{'execute': 'blockdev-add', 'arguments': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks', 'node_name': 'imgfile'}} +{'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': 'imgfile', 'iter-time': 10, 'key-secret': 'keysec0', 'size': 134217728}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_IMG"}, "key-secret": "keysec0"} file format: IMGFMT @@ -54,15 +54,15 @@ Format specific information: === Successful image creation (with non-default options) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'hash-alg': 'sha1', 'cipher-mode': 'ctr', 'cipher-alg': 'twofish-128', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}, 'iter-time': 10, 'ivgen-alg': 'plain64', 'ivgen-hash-alg': 'md5', 'driver': 'luks', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cipher-alg': 'twofish-128', 'cipher-mode': 'ctr', 'driver': 'luks', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}, 'hash-alg': 'sha1', 'iter-time': 10, 'ivgen-alg': 'plain64', 'ivgen-hash-alg': 'md5', 'key-secret': 'keysec0', 'size': 67108864}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_IMG"}, "key-secret": "keysec0"} file format: IMGFMT @@ -107,17 +107,17 @@ Format specific information: === Invalid BlockdevRef === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': "this doesn't exist", 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Zero size === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'iter-time': 10, 'driver': 'luks', 'file': 'node0', 'size': 0}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': 'node0', 'iter-time': 10, 'key-secret': 'keysec0', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: json:{"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_IMG"}, "key-secret": "keysec0"} file format: IMGFMT @@ -161,34 +161,34 @@ Format specific information: === Invalid sizes === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 18446744073709551104L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': 'node0', 'key-secret': 'keysec0', 'size': 18446744073709551104}}} +{'return': {}} Job failed: The requested file size is too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 9223372036854775808L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': 'node0', 'key-secret': 'keysec0', 'size': 9223372036854775808}}} +{'return': {}} Job failed: The requested file size is too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 9223372036854775296}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': 'node0', 'key-secret': 'keysec0', 'size': 9223372036854775296}}} +{'return': {}} Job failed: The requested file size is too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Resize image with invalid sizes === -{'execute': 'block_resize', 'arguments': {'size': 9223372036854775296, 'node_name': 'node1'}} -{u'error': {u'class': u'GenericError', u'desc': u'The requested file size is too large'}} -{'execute': 'block_resize', 'arguments': {'size': 9223372036854775808L, 'node_name': 'node1'}} -{u'error': {u'class': u'GenericError', u'desc': u"Invalid parameter type for 'size', expected: integer"}} -{'execute': 'block_resize', 'arguments': {'size': 18446744073709551104L, 'node_name': 'node1'}} -{u'error': {u'class': u'GenericError', u'desc': u"Invalid parameter type for 'size', expected: integer"}} -{'execute': 'block_resize', 'arguments': {'size': -9223372036854775808, 'node_name': 'node1'}} -{u'error': {u'class': u'GenericError', u'desc': u"Parameter 'size' expects a >0 size"}} +{'execute': 'block_resize', 'arguments': {'node_name': 'node1', 'size': 9223372036854775296}} +{'error': {'class': 'GenericError', 'desc': 'The requested file size is too large'}} +{'execute': 'block_resize', 'arguments': {'node_name': 'node1', 'size': 9223372036854775808}} +{'error': {'class': 'GenericError', 'desc': "Invalid parameter type for 'size', expected: integer"}} +{'execute': 'block_resize', 'arguments': {'node_name': 'node1', 'size': 18446744073709551104}} +{'error': {'class': 'GenericError', 'desc': "Invalid parameter type for 'size', expected: integer"}} +{'execute': 'block_resize', 'arguments': {'node_name': 'node1', 'size': -9223372036854775808}} +{'error': {'class': 'GenericError', 'desc': "Parameter 'size' expects a >0 size"}} image: json:{"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_IMG"}, "key-secret": "keysec0"} file format: IMGFMT virtual size: 0 (0 bytes) diff --git a/tests/qemu-iotests/211.out b/tests/qemu-iotests/211.out index 6feaea3978..3816d89b40 100644 --- a/tests/qemu-iotests/211.out +++ b/tests/qemu-iotests/211.out @@ -1,16 +1,16 @@ === Successful image creation (defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}} -{u'return': {}} +{'execute': 'blockdev-add', 'arguments': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi', 'node_name': 'imgfile'}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'imgfile', 'size': 134217728}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -21,15 +21,15 @@ cluster_size: 1048576 === Successful image creation (explicit defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'off', 'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'preallocation': 'off', 'size': 67108864}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -40,15 +40,15 @@ cluster_size: 1048576 === Successful image creation (with non-default options) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'metadata', 'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'size': 33554432}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'preallocation': 'metadata', 'size': 33554432}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -61,17 +61,17 @@ cluster_size: 1048576 === Invalid BlockdevRef === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': "this doesn't exist", 'size': 33554432}}} -{u'return': {}} +{'return': {}} Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Zero size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 0}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -81,9 +81,9 @@ cluster_size: 1048576 === Maximum size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 562949819203584}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -92,21 +92,21 @@ cluster_size: 1048576 === Invalid sizes === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 18446744073709551104L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 18446744073709551104}}} +{'return': {}} Job failed: Unsupported VDI image size (size is 0xfffffffffffffe00, max supported is 0x1fffff8000000) {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 9223372036854775808L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 9223372036854775808}}} +{'return': {}} Job failed: Unsupported VDI image size (size is 0x8000000000000000, max supported is 0x1fffff8000000) {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 562949819203585}}} -{u'return': {}} +{'return': {}} Job failed: Unsupported VDI image size (size is 0x1fffff8000001, max supported is 0x1fffff8000000) {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/212.out b/tests/qemu-iotests/212.out index 9150da7a2c..9779d3a4cb 100644 --- a/tests/qemu-iotests/212.out +++ b/tests/qemu-iotests/212.out @@ -1,16 +1,16 @@ === Successful image creation (defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}} -{u'return': {}} +{'execute': 'blockdev-add', 'arguments': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels', 'node_name': 'imgfile'}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'imgfile', 'size': 134217728}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -18,15 +18,15 @@ virtual size: 128M (134217728 bytes) === Successful image creation (explicit defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1048576, 'driver': 'parallels', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}, 'size': 67108864}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -34,15 +34,15 @@ virtual size: 64M (67108864 bytes) === Successful image creation (with non-default options) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'driver': 'parallels', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}, 'size': 33554432}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -51,17 +51,17 @@ virtual size: 32M (33554432 bytes) === Invalid BlockdevRef === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': "this doesn't exist", 'size': 33554432}}} -{u'return': {}} +{'return': {}} Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Zero size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 0}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -70,9 +70,9 @@ virtual size: 0 (0 bytes) === Maximum size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 4503599627369984}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -81,76 +81,76 @@ virtual size: 4096T (4503599627369984 bytes) === Invalid sizes === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 1234}}} -{u'return': {}} +{'return': {}} Job failed: Image size must be a multiple of 512 bytes {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 18446744073709551104L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 18446744073709551104}}} +{'return': {}} Job failed: Image size is too large for this cluster size {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775808L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775808}}} +{'return': {}} Job failed: Image size is too large for this cluster size {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775296}}} -{u'return': {}} +{'return': {}} Job failed: Image size is too large for this cluster size {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 4503599627370497}}} -{u'return': {}} +{'return': {}} Job failed: Image size is too large for this cluster size {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid cluster size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1234, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size must be a multiple of 512 bytes {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 128, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size must be a multiple of 512 bytes {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 4294967296, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Cluster size is too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 9223372036854775808L, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 9223372036854775808, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Cluster size is too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 18446744073709551104L, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 18446744073709551104, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Cluster size is too large {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 0, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'return': {}} Job failed: Image size is too large for this cluster size {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'driver': 'parallels', 'file': 'node0', 'size': 281474976710656}}} -{u'return': {}} +{'return': {}} Job failed: Image size is too large for this cluster size {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/213.out b/tests/qemu-iotests/213.out index e1dcd47201..9e214d4fc5 100644 --- a/tests/qemu-iotests/213.out +++ b/tests/qemu-iotests/213.out @@ -1,16 +1,16 @@ === Successful image creation (defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}} -{u'return': {}} +{'execute': 'blockdev-add', 'arguments': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx', 'node_name': 'imgfile'}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'imgfile', 'size': 134217728}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -19,15 +19,15 @@ cluster_size: 8388608 === Successful image creation (explicit defaults) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 8388608, 'driver': 'vhdx', 'subformat': 'dynamic', 'log-size': 1048576, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'block-state-zero': True, 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 8388608, 'block-state-zero': True, 'driver': 'vhdx', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'log-size': 1048576, 'size': 67108864, 'subformat': 'dynamic'}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -36,15 +36,15 @@ cluster_size: 8388608 === Successful image creation (with non-default options) === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx', 'size': 0}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 268435456, 'driver': 'vhdx', 'subformat': 'fixed', 'log-size': 8388608, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'block-state-zero': False, 'size': 33554432}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 268435456, 'block-state-zero': False, 'driver': 'vhdx', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'log-size': 8388608, 'size': 33554432, 'subformat': 'fixed'}}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -54,17 +54,17 @@ cluster_size: 268435456 === Invalid BlockdevRef === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': "this doesn't exist", 'size': 33554432}}} -{u'return': {}} +{'return': {}} Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Zero size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 0}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -74,9 +74,9 @@ cluster_size: 8388608 === Maximum size === {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 70368744177664}}} -{u'return': {}} +{'return': {}} {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} image: TEST_IMG file format: IMGFMT @@ -85,85 +85,85 @@ cluster_size: 67108864 === Invalid sizes === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 18446744073709551104L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 18446744073709551104}}} +{'return': {}} Job failed: Image size too large; max of 64TB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775808L}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775808}}} +{'return': {}} Job failed: Image size too large; max of 64TB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775296}}} -{u'return': {}} +{'return': {}} Job failed: Image size too large; max of 64TB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} {'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 70368744177665}}} -{u'return': {}} +{'return': {}} Job failed: Image size too large; max of 64TB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid block size === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 1234567, 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 1234567, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Block size must be a multiple of 1 MB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 128, 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 128, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Block size must be a multiple of 1 MB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 3145728, 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 3145728, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Block size must be a power of two {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 536870912, 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 536870912, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Block size must not exceed 268435456 {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 0, 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 0, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} +{'return': {}} Job failed: Block size must be a multiple of 1 MB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} === Invalid log size === -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 1234567, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'log-size': 1234567, 'size': 67108864}}} +{'return': {}} Job failed: Log size must be a multiple of 1 MB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 128, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'log-size': 128, 'size': 67108864}}} +{'return': {}} Job failed: Log size must be a multiple of 1 MB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 4294967296, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'log-size': 4294967296, 'size': 67108864}}} +{'return': {}} Job failed: Log size must be smaller than 4 GB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} -{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 0, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} -{u'return': {}} +{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'log-size': 0, 'size': 67108864}}} +{'return': {}} Job failed: Log size must be a multiple of 1 MB {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} -{u'return': {}} +{'return': {}} diff --git a/tests/qemu-iotests/216.out b/tests/qemu-iotests/216.out index 45ea857ee1..908f9314ac 100644 --- a/tests/qemu-iotests/216.out +++ b/tests/qemu-iotests/216.out @@ -7,8 +7,8 @@ Done --- Doing COR --- -{u'return': {}} -{u'return': u''} +{'return': {}} +{'return': ''} --- Checking COR result --- diff --git a/tests/qemu-iotests/218.out b/tests/qemu-iotests/218.out index 7dbf78e682..9d270e4834 100644 --- a/tests/qemu-iotests/218.out +++ b/tests/qemu-iotests/218.out @@ -4,27 +4,27 @@ --- force=false --- Cancelling job -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 65536, u'len': 1048576, u'offset': 65536}, u'event': u'BLOCK_JOB_CANCELLED'} +{'return': {}} +{'data': {'device': 'mirror', 'len': 1048576, 'offset': 65536, 'speed': 65536, 'type': 'mirror'}, 'event': 'BLOCK_JOB_CANCELLED', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} --- force=true --- Cancelling job -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 65536, u'len': 1048576, u'offset': 65536}, u'event': u'BLOCK_JOB_CANCELLED'} +{'return': {}} +{'data': {'device': 'mirror', 'len': 1048576, 'offset': 65536, 'speed': 65536, 'type': 'mirror'}, 'event': 'BLOCK_JOB_CANCELLED', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} === Cancel mirror job after convergence === --- force=false --- -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_READY'} +{'data': {'device': 'mirror', 'len': 1048576, 'offset': 1048576, 'speed': 0, 'type': 'mirror'}, 'event': 'BLOCK_JOB_READY', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Cancelling job -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_COMPLETED'} +{'return': {}} +{'data': {'device': 'mirror', 'len': 1048576, 'offset': 1048576, 'speed': 0, 'type': 'mirror'}, 'event': 'BLOCK_JOB_COMPLETED', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} --- force=true --- -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_READY'} +{'data': {'device': 'mirror', 'len': 1048576, 'offset': 1048576, 'speed': 0, 'type': 'mirror'}, 'event': 'BLOCK_JOB_READY', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Cancelling job -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_CANCELLED'} +{'return': {}} +{'data': {'device': 'mirror', 'len': 1048576, 'offset': 1048576, 'speed': 0, 'type': 'mirror'}, 'event': 'BLOCK_JOB_CANCELLED', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} diff --git a/tests/qemu-iotests/219.out b/tests/qemu-iotests/219.out index 6dc07bc41e..9acf1e89ac 100644 --- a/tests/qemu-iotests/219.out +++ b/tests/qemu-iotests/219.out @@ -2,326 +2,326 @@ Launching VM... Starting block job: drive-mirror (auto-finalize: True; auto-dismiss: True) -{u'return': {}} -{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'mirror'}]} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} +{'return': {}} +{'return': [{'current-progress': 'FILTERED', 'id': 'job0', 'status': 'running', 'total-progress': 'FILTERED', 'type': 'mirror'}]} +{'data': {'id': 'job0', 'status': 'created'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Pause/resume in RUNNING === Testing block-job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 65536, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'mirror'}]} === Testing block-job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'mirror'}]} === Testing job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'mirror'}]} === Testing job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'return': {}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 327680, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'mirror'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'return': {}} Waiting for READY state... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'data': {'id': 'job0', 'status': 'ready'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'ready', 'total-progress': 4194304, 'type': 'mirror'}]} Pause/resume in READY === Testing block-job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'standby'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'standby', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'ready'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'ready', 'total-progress': 4194304, 'type': 'mirror'}]} === Testing block-job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'standby'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'standby', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'ready'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'ready', 'total-progress': 4194304, 'type': 'mirror'}]} === Testing job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'standby'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'standby', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'ready'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'ready', 'total-progress': 4194304, 'type': 'mirror'}]} === Testing job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'dismiss'"}} -{u'return': {}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'standby'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'standby', 'total-progress': 4194304, 'type': 'mirror'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'ready'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'ready', 'total-progress': 4194304, 'type': 'mirror'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'ready' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'ready' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'ready' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'ready' cannot accept command verb 'dismiss'"}} +{'return': {}} Waiting for PENDING state... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': []} +{'data': {'id': 'job0', 'status': 'waiting'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'pending'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'concluded'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'null'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': []} Starting block job: drive-backup (auto-finalize: True; auto-dismiss: True) -{u'return': {}} -{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} +{'return': {}} +{'return': [{'current-progress': 'FILTERED', 'id': 'job0', 'status': 'running', 'total-progress': 'FILTERED', 'type': 'backup'}]} +{'data': {'id': 'job0', 'status': 'created'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Pause/resume in RUNNING === Testing block-job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 65536, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing block-job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'return': {}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 327680, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'return': {}} Waiting for PENDING state... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': []} +{'data': {'id': 'job0', 'status': 'waiting'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'pending'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'concluded'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'null'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': []} Starting block job: drive-backup (auto-finalize: True; auto-dismiss: False) -{u'return': {}} -{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} +{'return': {}} +{'return': [{'current-progress': 'FILTERED', 'id': 'job0', 'status': 'running', 'total-progress': 'FILTERED', 'type': 'backup'}]} +{'data': {'id': 'job0', 'status': 'created'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Pause/resume in RUNNING === Testing block-job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 65536, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing block-job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'return': {}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 327680, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'return': {}} Waiting for PENDING state... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'concluded', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': []} +{'data': {'id': 'job0', 'status': 'waiting'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'pending'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'concluded'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'concluded', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'null'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': []} Starting block job: drive-backup (auto-finalize: False; auto-dismiss: True) -{u'return': {}} -{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} +{'return': {}} +{'return': [{'current-progress': 'FILTERED', 'id': 'job0', 'status': 'running', 'total-progress': 'FILTERED', 'type': 'backup'}]} +{'data': {'id': 'job0', 'status': 'created'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Pause/resume in RUNNING === Testing block-job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 65536, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing block-job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'return': {}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 327680, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'return': {}} Waiting for PENDING state... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'pending', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': []} +{'data': {'id': 'job0', 'status': 'waiting'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'pending'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'pending', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'concluded'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'null'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': []} Starting block job: drive-backup (auto-finalize: False; auto-dismiss: False) -{u'return': {}} -{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} +{'return': {}} +{'return': [{'current-progress': 'FILTERED', 'id': 'job0', 'status': 'running', 'total-progress': 'FILTERED', 'type': 'backup'}]} +{'data': {'id': 'job0', 'status': 'created'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} Pause/resume in RUNNING === Testing block-job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 65536, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing block-job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 131072, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/block-job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 196608, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} === Testing job-pause/job-resume === -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} -{u'return': {}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'paused'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 262144, 'id': 'job0', 'status': 'paused', 'total-progress': 4194304, 'type': 'backup'}]} +{'return': {}} +{'data': {'id': 'job0', 'status': 'running'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 327680, 'id': 'job0', 'status': 'running', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} +{'return': {}} Waiting for PENDING state... -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'pending', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': [{u'status': u'concluded', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} -{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} -{u'return': []} +{'data': {'id': 'job0', 'status': 'waiting'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'data': {'id': 'job0', 'status': 'pending'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'pending', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'concluded'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': [{'current-progress': 4194304, 'id': 'job0', 'status': 'concluded', 'total-progress': 4194304, 'type': 'backup'}]} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} +{'error': {'class': 'GenericError', 'desc': "Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} +{'return': {}} +{'data': {'id': 'job0', 'status': 'null'}, 'event': 'JOB_STATUS_CHANGE', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': []} diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/222.out index 48f336a02b..37455f6c2d 100644 --- a/tests/qemu-iotests/222.out +++ b/tests/qemu-iotests/222.out @@ -8,13 +8,13 @@ Done --- Setting up Fleecing Graph --- -{u'return': {}} -{u'return': {}} +{'return': {}} +{'return': {}} --- Setting up NBD Export --- -{u'return': {}} -{u'return': {}} +{'return': {}} +{'return': {}} --- Sanity Check --- @@ -29,13 +29,13 @@ read -P0 0x3fe0000 64k --- Testing COW --- write -P0xab 0 64k -{u'return': u''} +{'return': ''} write -P0xad 0x00f8000 64k -{u'return': u''} +{'return': ''} write -P0x1d 0x2008000 64k -{u'return': u''} +{'return': ''} write -P0xea 0x3fe0000 64k -{u'return': u''} +{'return': ''} --- Verifying Data --- @@ -49,10 +49,10 @@ read -P0 0x3fe0000 64k --- Cleanup --- -{u'return': {}} -{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'drive0', u'type': u'backup', u'speed': 0, u'len': 67108864, u'offset': 393216}, u'event': u'BLOCK_JOB_CANCELLED'} -{u'return': {}} -{u'return': {}} +{'return': {}} +{'data': {'device': 'drive0', 'len': 67108864, 'offset': 393216, 'speed': 0, 'type': 'backup'}, 'event': 'BLOCK_JOB_CANCELLED', 'timestamp': {'microseconds': 'USECS', 'seconds': 'SECS'}} +{'return': {}} +{'return': {}} --- Confirming writes --- diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index a64ea90fb4..f8dbc8cc71 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -250,10 +250,45 @@ def filter_img_info(output, filename): lines.append(line) return '\n'.join(lines) +def log_to_string_repr(obj): + # Normal Python 3 strings are the unicode strings from Python 2; + # and normal Python 2 strings are byte strings in Python 3. Thus, + # we convert bytes -> str in py3 and unicode -> str in py2. + if sys.version_info.major >= 3: + if type(obj) is bytes: + return repr(obj.decode('utf-8')) + else: + if type(obj) is unicode: + return repr(obj.encode('utf-8')) + elif type(obj) is long: + return str(obj) # repr() would include an 'L' suffix + + if type(obj) is list: + return '[' + ', '.join(map(log_to_string_repr, obj)) + ']' + elif type(obj) is dict: + return '{' + ', '.join(map(lambda k: log_to_string_repr(k) + ': ' + + log_to_string_repr(obj[k]), + sorted(obj.keys()))) + '}' + else: + return repr(obj) + +def log_to_string(obj): + if type(obj) is str: + return obj + + if sys.version_info.major >= 3: + if type(obj) is bytes: + return obj.decode('utf-8') + else: + if type(obj) is unicode: + return obj.encode('utf-8') + + return log_to_string_repr(obj) + def log(msg, filters=[]): for flt in filters: msg = flt(msg) - print(msg) + print(log_to_string(msg)) class Timeout: def __init__(self, seconds, errmsg = "Timeout"): @@ -441,10 +476,11 @@ class VM(qtest.QEMUQtestMachine): return result def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): - logmsg = "{'execute': '%s', 'arguments': %s}" % (cmd, kwargs) + logmsg = "{'execute': '%s', 'arguments': %s}" % \ + (cmd, log_to_string(kwargs)) log(logmsg, filters) result = self.qmp(cmd, **kwargs) - log(str(result), filters) + log(log_to_string(result), filters) return result def run_job(self, job, auto_finalize=True, auto_dismiss=False): -- 2.17.1 ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 2018-10-15 14:14 ` [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 Max Reitz @ 2018-10-15 22:26 ` Eduardo Habkost 2018-10-19 9:33 ` Max Reitz 0 siblings, 1 reply; 48+ messages in thread From: Eduardo Habkost @ 2018-10-15 22:26 UTC (permalink / raw) To: Max Reitz; +Cc: qemu-block, Kevin Wolf, Cleber Rosa, qemu-devel On Mon, Oct 15, 2018 at 04:14:53PM +0200, Max Reitz wrote: > When dumping an object into the log, there are differences between > Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2 > (they are no longer in 3, because unicode strings are the default > there). [...] This could be addressed using JSON. See below[1]. > [...] Second, the order of keys in dicts may differ. [...] Can be addressed using json.dumps(..., sort_keys=True). > [...] Third, > especially long numbers are longs in Python 2 and thus get an 'L' > suffix, which does not happen in Python 3. print() doesn't add a L suffix on Python 2.7 on my system: Python 2.7.15 (default, May 15 2018, 15:37:31) [GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print(99999999999999999999999999999999999999999999999999999999999999999999999999999999999L) 99999999999999999999999999999999999999999999999999999999999999999999999999999999999 So I assume this happens only for QMP commands. It would be addressed if using JSON, too. > > To get around these differences, this patch introduces functions to > convert an object to a string that looks the same regardless of the > Python version: In Python 2, they decode unicode strings to byte strings > (normal str); in Python 3, they encode byte strings to unicode strings > (normal str). They also manually convert lists and dicts to strings, > which allows sorting the dicts by key, so we are no longer at the mercy > of the internal implementation when it comes to how the keys appear in > the output. > > This changes the output of all tests that use these logging functions. > > Signed-off-by: Max Reitz <mreitz@redhat.com> > --- > tests/qemu-iotests/194.out | 22 +- > tests/qemu-iotests/202.out | 12 +- > tests/qemu-iotests/203.out | 14 +- > tests/qemu-iotests/206.out | 144 +++++----- > tests/qemu-iotests/207.out | 52 ++-- > tests/qemu-iotests/208.out | 8 +- > tests/qemu-iotests/210.out | 72 ++--- > tests/qemu-iotests/211.out | 66 ++--- > tests/qemu-iotests/212.out | 102 +++---- > tests/qemu-iotests/213.out | 124 ++++---- > tests/qemu-iotests/216.out | 4 +- > tests/qemu-iotests/218.out | 20 +- > tests/qemu-iotests/219.out | 526 +++++++++++++++++----------------- > tests/qemu-iotests/222.out | 24 +- > tests/qemu-iotests/iotests.py | 42 ++- > 15 files changed, 634 insertions(+), 598 deletions(-) [...] > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > index a64ea90fb4..f8dbc8cc71 100644 > --- a/tests/qemu-iotests/iotests.py > +++ b/tests/qemu-iotests/iotests.py > @@ -250,10 +250,45 @@ def filter_img_info(output, filename): > lines.append(line) > return '\n'.join(lines) > > +def log_to_string_repr(obj): > + # Normal Python 3 strings are the unicode strings from Python 2; > + # and normal Python 2 strings are byte strings in Python 3. Thus, > + # we convert bytes -> str in py3 and unicode -> str in py2. > + if sys.version_info.major >= 3: > + if type(obj) is bytes: > + return repr(obj.decode('utf-8')) > + else: > + if type(obj) is unicode: > + return repr(obj.encode('utf-8')) > + elif type(obj) is long: > + return str(obj) # repr() would include an 'L' suffix > + > + if type(obj) is list: > + return '[' + ', '.join(map(log_to_string_repr, obj)) + ']' > + elif type(obj) is dict: > + return '{' + ', '.join(map(lambda k: log_to_string_repr(k) + ': ' + > + log_to_string_repr(obj[k]), > + sorted(obj.keys()))) + '}' > + else: > + return repr(obj) I assume this function exists only because of QMP logging, correct? I would just use json.dumps() at qmp_log(), see below[1]. > + > +def log_to_string(obj): > + if type(obj) is str: > + return obj > + > + if sys.version_info.major >= 3: > + if type(obj) is bytes: > + return obj.decode('utf-8') Do you know how many of existing log() calls actually pass a byte string as argument? > + else: > + if type(obj) is unicode: > + return obj.encode('utf-8') Is this really necessary? The existing code just calls print(msg) and it works, doesn't it? > + > + return log_to_string_repr(obj) > + > def log(msg, filters=[]): > for flt in filters: > msg = flt(msg) > - print(msg) > + print(log_to_string(msg)) > > class Timeout: > def __init__(self, seconds, errmsg = "Timeout"): > @@ -441,10 +476,11 @@ class VM(qtest.QEMUQtestMachine): > return result > > def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): > - logmsg = "{'execute': '%s', 'arguments': %s}" % (cmd, kwargs) > + logmsg = "{'execute': '%s', 'arguments': %s}" % \ > + (cmd, log_to_string(kwargs)) > log(logmsg, filters) > result = self.qmp(cmd, **kwargs) > - log(str(result), filters) > + log(log_to_string(result), filters) [1] If we're being forced to regenerate all the QMP output in the .out files, what about always using JSON instead of the hack at log_to_string_repr()? i.e.: diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index a64ea90fb4..0b28dc2a65 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -441,10 +441,12 @@ class VM(qtest.QEMUQtestMachine): return result def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): - logmsg = "{'execute': '%s', 'arguments': %s}" % (cmd, kwargs) + logmsg = '{"execute": %s, "arguments": %s}' % \ + (json.dumps(cmd, sort_keys=True), + json.dumps(kwargs, sort_keys=True)) log(logmsg, filters) result = self.qmp(cmd, **kwargs) - log(str(result), filters) + log(json.dumps(result, sort_keys=True), filters) return result def run_job(self, job, auto_finalize=True, auto_dismiss=False): > return result > > def run_job(self, job, auto_finalize=True, auto_dismiss=False): > -- > 2.17.1 > > -- Eduardo ^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 2018-10-15 22:26 ` Eduardo Habkost @ 2018-10-19 9:33 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:33 UTC (permalink / raw) To: Eduardo Habkost; +Cc: qemu-block, Kevin Wolf, Cleber Rosa, qemu-devel [-- Attachment #1: Type: text/plain, Size: 7620 bytes --] On 16.10.18 00:26, Eduardo Habkost wrote: > On Mon, Oct 15, 2018 at 04:14:53PM +0200, Max Reitz wrote: >> When dumping an object into the log, there are differences between >> Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2 >> (they are no longer in 3, because unicode strings are the default >> there). [...] > > This could be addressed using JSON. See below[1]. > >> [...] Second, the order of keys in dicts may differ. [...] > > Can be addressed using json.dumps(..., sort_keys=True). Ah. Nice. :-) >> [...] Third, >> especially long numbers are longs in Python 2 and thus get an 'L' >> suffix, which does not happen in Python 3. > > print() doesn't add a L suffix on Python 2.7 on my system: > > Python 2.7.15 (default, May 15 2018, 15:37:31) > [GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> print(99999999999999999999999999999999999999999999999999999999999999999999999999999999999L) > 99999999999999999999999999999999999999999999999999999999999999999999999999999999999 > > So I assume this happens only for QMP commands. It happens for dicts: >>> print 99999999999999999999L 99999999999999999999 >>> print {'foo':99999999999999999999L} {'foo': 99999999999999999999L} > It would be addressed if using JSON, too. OK. >> To get around these differences, this patch introduces functions to >> convert an object to a string that looks the same regardless of the >> Python version: In Python 2, they decode unicode strings to byte strings >> (normal str); in Python 3, they encode byte strings to unicode strings >> (normal str). They also manually convert lists and dicts to strings, >> which allows sorting the dicts by key, so we are no longer at the mercy >> of the internal implementation when it comes to how the keys appear in >> the output. >> >> This changes the output of all tests that use these logging functions. >> >> Signed-off-by: Max Reitz <mreitz@redhat.com> >> --- >> tests/qemu-iotests/194.out | 22 +- >> tests/qemu-iotests/202.out | 12 +- >> tests/qemu-iotests/203.out | 14 +- >> tests/qemu-iotests/206.out | 144 +++++----- >> tests/qemu-iotests/207.out | 52 ++-- >> tests/qemu-iotests/208.out | 8 +- >> tests/qemu-iotests/210.out | 72 ++--- >> tests/qemu-iotests/211.out | 66 ++--- >> tests/qemu-iotests/212.out | 102 +++---- >> tests/qemu-iotests/213.out | 124 ++++---- >> tests/qemu-iotests/216.out | 4 +- >> tests/qemu-iotests/218.out | 20 +- >> tests/qemu-iotests/219.out | 526 +++++++++++++++++----------------- >> tests/qemu-iotests/222.out | 24 +- >> tests/qemu-iotests/iotests.py | 42 ++- >> 15 files changed, 634 insertions(+), 598 deletions(-) > [...] >> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py >> index a64ea90fb4..f8dbc8cc71 100644 >> --- a/tests/qemu-iotests/iotests.py >> +++ b/tests/qemu-iotests/iotests.py >> @@ -250,10 +250,45 @@ def filter_img_info(output, filename): >> lines.append(line) >> return '\n'.join(lines) >> >> +def log_to_string_repr(obj): >> + # Normal Python 3 strings are the unicode strings from Python 2; >> + # and normal Python 2 strings are byte strings in Python 3. Thus, >> + # we convert bytes -> str in py3 and unicode -> str in py2. >> + if sys.version_info.major >= 3: >> + if type(obj) is bytes: >> + return repr(obj.decode('utf-8')) >> + else: >> + if type(obj) is unicode: >> + return repr(obj.encode('utf-8')) >> + elif type(obj) is long: >> + return str(obj) # repr() would include an 'L' suffix >> + >> + if type(obj) is list: >> + return '[' + ', '.join(map(log_to_string_repr, obj)) + ']' >> + elif type(obj) is dict: >> + return '{' + ', '.join(map(lambda k: log_to_string_repr(k) + ': ' + >> + log_to_string_repr(obj[k]), >> + sorted(obj.keys()))) + '}' >> + else: >> + return repr(obj) > > I assume this function exists only because of QMP logging, > correct? In practice probably yes. > I would just use json.dumps() at qmp_log(), see below[1]. However, there is not just qmp_log(), there are places that dump objects directly into log(). >> + >> +def log_to_string(obj): >> + if type(obj) is str: >> + return obj >> + >> + if sys.version_info.major >= 3: >> + if type(obj) is bytes: >> + return obj.decode('utf-8') > > Do you know how many of existing log() calls actually pass a byte > string as argument? Frankly I hope none. >> + else: >> + if type(obj) is unicode: >> + return obj.encode('utf-8') > > Is this really necessary? The existing code just calls > print(msg) and it works, doesn't it? True. But the main issue is that we still need to return immediately for byte strings and Unicode strings, but the type "bytes" only exists in 3.x and "unicode" only exists in 2.x, i.e. "if type(obj) is unicode" throws an exception in 3.x. So I have to distinguish between 2.x and 3.x anyway, and I thought to myself that if I distinguished anyway, I might do the conversion to the native type at the same time. >> + >> + return log_to_string_repr(obj) >> + >> def log(msg, filters=[]): >> for flt in filters: >> msg = flt(msg) >> - print(msg) >> + print(log_to_string(msg)) >> >> class Timeout: >> def __init__(self, seconds, errmsg = "Timeout"): >> @@ -441,10 +476,11 @@ class VM(qtest.QEMUQtestMachine): >> return result >> >> def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): >> - logmsg = "{'execute': '%s', 'arguments': %s}" % (cmd, kwargs) >> + logmsg = "{'execute': '%s', 'arguments': %s}" % \ >> + (cmd, log_to_string(kwargs)) >> log(logmsg, filters) >> result = self.qmp(cmd, **kwargs) >> - log(str(result), filters) >> + log(log_to_string(result), filters) > > [1] > > If we're being forced to regenerate all the QMP output in the > .out files, what about always using JSON instead of the hack at > log_to_string_repr()? i.e.: > > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py > index a64ea90fb4..0b28dc2a65 100644 > --- a/tests/qemu-iotests/iotests.py > +++ b/tests/qemu-iotests/iotests.py > @@ -441,10 +441,12 @@ class VM(qtest.QEMUQtestMachine): > return result > > def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): > - logmsg = "{'execute': '%s', 'arguments': %s}" % (cmd, kwargs) > + logmsg = '{"execute": %s, "arguments": %s}' % \ > + (json.dumps(cmd, sort_keys=True), > + json.dumps(kwargs, sort_keys=True)) > log(logmsg, filters) > result = self.qmp(cmd, **kwargs) > - log(str(result), filters) > + log(json.dumps(result, sort_keys=True), filters) > return result Sure, sounds good, thanks. :-) The only thing is I would allow log() to still accept objects, it should do the conversion to JSON itself (if the object to be logged is a list or dict). Max > def run_job(self, job, auto_finalize=True, auto_dismiss=False): > >> return result >> >> def run_job(self, job, auto_finalize=True, auto_dismiss=False): >> -- >> 2.17.1 >> >> > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz ` (8 preceding siblings ...) 2018-10-15 14:14 ` [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 Max Reitz @ 2018-10-15 22:19 ` Philippe Mathieu-Daudé 2018-10-19 9:08 ` Max Reitz 9 siblings, 1 reply; 48+ messages in thread From: Philippe Mathieu-Daudé @ 2018-10-15 22:19 UTC (permalink / raw) To: Max Reitz, qemu-block Cc: Kevin Wolf, Cleber Rosa, qemu-devel, Eduardo Habkost Hi Max, On 15/10/2018 16:14, Max Reitz wrote: > This series prepares the iotests to work with both Python 2 and 3. In > some places, it adds version-specific code and decides what to do based > on the version (for instance, whether to import the StringIO class from > the 'io' or the 'StringIO' module), but most of the time, it just makes > code work for both versions in general. > > And when we make the switch to make Python 3 mandatory, we can simply > drop the Python 2 branches. > > > Max Reitz (9): > iotests: Make nbd-fault-injector flush > iotests: Flush in iotests.py's QemuIoInteractive > iotests: Use Python byte strings where appropriate > iotests: Use // for Python integer division > iotests: Different iterator behavior in Python 3 > iotests: Explicitly inherit FDs in Python > iotests: 'new' module replacement in 169 > iotests: Modify imports for Python 3 > iotests: Unify log outputs between Python 2 and 3 You forgot: MAINTAINERS: Add myself in the Python scripts section ;) > > scripts/qemu.py | 13 +- > scripts/qmp/qmp.py | 7 + > scripts/qtest.py | 2 +- > tests/qemu-iotests/040 | 4 +- > tests/qemu-iotests/044 | 20 +- > tests/qemu-iotests/056 | 2 +- > tests/qemu-iotests/065 | 4 +- > tests/qemu-iotests/083.out | 9 + > tests/qemu-iotests/093 | 18 +- > tests/qemu-iotests/124 | 4 +- > tests/qemu-iotests/139 | 2 +- > tests/qemu-iotests/147 | 7 + > tests/qemu-iotests/149 | 14 +- > tests/qemu-iotests/151 | 12 +- > tests/qemu-iotests/163 | 8 +- > tests/qemu-iotests/169 | 3 +- > tests/qemu-iotests/194.out | 22 +- > tests/qemu-iotests/202.out | 12 +- > tests/qemu-iotests/203.out | 14 +- > tests/qemu-iotests/206.out | 144 +++---- > tests/qemu-iotests/207 | 4 +- > tests/qemu-iotests/207.out | 52 +-- > tests/qemu-iotests/208.out | 8 +- > tests/qemu-iotests/210.out | 72 ++-- > tests/qemu-iotests/211.out | 66 +-- > tests/qemu-iotests/212.out | 102 ++--- > tests/qemu-iotests/213.out | 124 +++--- > tests/qemu-iotests/216.out | 4 +- > tests/qemu-iotests/218.out | 20 +- > tests/qemu-iotests/219.out | 526 +++++++++++------------ > tests/qemu-iotests/222.out | 24 +- > tests/qemu-iotests/iotests.py | 64 ++- > tests/qemu-iotests/nbd-fault-injector.py | 12 +- > tests/qemu-iotests/qcow2.py | 10 +- > 34 files changed, 745 insertions(+), 664 deletions(-) > ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 2018-10-15 22:19 ` [Qemu-devel] [PATCH 0/9] iotests: Make them work for both " Philippe Mathieu-Daudé @ 2018-10-19 9:08 ` Max Reitz 0 siblings, 0 replies; 48+ messages in thread From: Max Reitz @ 2018-10-19 9:08 UTC (permalink / raw) To: Philippe Mathieu-Daudé, qemu-block Cc: Kevin Wolf, Cleber Rosa, qemu-devel, Eduardo Habkost [-- Attachment #1: Type: text/plain, Size: 1214 bytes --] On 16.10.18 00:19, Philippe Mathieu-Daudé wrote: > Hi Max, > > On 15/10/2018 16:14, Max Reitz wrote: >> This series prepares the iotests to work with both Python 2 and 3. In >> some places, it adds version-specific code and decides what to do based >> on the version (for instance, whether to import the StringIO class from >> the 'io' or the 'StringIO' module), but most of the time, it just makes >> code work for both versions in general. >> >> And when we make the switch to make Python 3 mandatory, we can simply >> drop the Python 2 branches. >> >> >> Max Reitz (9): >> iotests: Make nbd-fault-injector flush >> iotests: Flush in iotests.py's QemuIoInteractive >> iotests: Use Python byte strings where appropriate >> iotests: Use // for Python integer division >> iotests: Different iterator behavior in Python 3 >> iotests: Explicitly inherit FDs in Python >> iotests: 'new' module replacement in 169 >> iotests: Modify imports for Python 3 >> iotests: Unify log outputs between Python 2 and 3 > > You forgot: > > MAINTAINERS: Add myself in the Python scripts section > > ;) Aw, please, no. You do know I don't really know Python? ;-) Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 48+ messages in thread
end of thread, other threads:[~2018-10-19 14:21 UTC | newest] Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-10-15 14:14 [Qemu-devel] [PATCH 0/9] iotests: Make them work for both Python 2 and 3 Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 1/9] iotests: Make nbd-fault-injector flush Max Reitz 2018-10-15 19:42 ` Eduardo Habkost 2018-10-15 20:24 ` Cleber Rosa 2018-10-16 18:07 ` Eric Blake 2018-10-19 9:48 ` Max Reitz 2018-10-19 14:21 ` Eric Blake 2018-10-15 14:14 ` [Qemu-devel] [PATCH 2/9] iotests: Flush in iotests.py's QemuIoInteractive Max Reitz 2018-10-15 19:43 ` Eduardo Habkost 2018-10-15 20:49 ` Cleber Rosa 2018-10-15 14:14 ` [Qemu-devel] [PATCH 3/9] iotests: Use Python byte strings where appropriate Max Reitz 2018-10-15 19:53 ` Eduardo Habkost 2018-10-19 8:46 ` Max Reitz 2018-10-15 22:08 ` Philippe Mathieu-Daudé 2018-10-15 14:14 ` [Qemu-devel] [PATCH 4/9] iotests: Use // for Python integer division Max Reitz 2018-10-15 19:54 ` Eduardo Habkost 2018-10-15 21:13 ` Cleber Rosa 2018-10-19 9:06 ` Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 5/9] iotests: Different iterator behavior in Python 3 Max Reitz 2018-10-15 20:07 ` Eduardo Habkost 2018-10-19 8:52 ` Max Reitz 2018-10-15 22:39 ` Cleber Rosa 2018-10-19 9:42 ` Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 6/9] iotests: Explicitly inherit FDs in Python Max Reitz 2018-10-15 20:30 ` Eduardo Habkost 2018-10-19 9:03 ` Max Reitz 2018-10-15 23:18 ` Cleber Rosa 2018-10-19 9:43 ` Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 7/9] iotests: 'new' module replacement in 169 Max Reitz 2018-10-15 21:13 ` Eduardo Habkost 2018-10-15 23:38 ` Cleber Rosa 2018-10-15 23:57 ` Eduardo Habkost 2018-10-16 1:01 ` Cleber Rosa 2018-10-19 9:46 ` Max Reitz 2018-10-19 14:18 ` Eduardo Habkost 2018-10-15 14:14 ` [Qemu-devel] [PATCH 8/9] iotests: Modify imports for Python 3 Max Reitz 2018-10-15 18:59 ` Cleber Rosa 2018-10-15 20:15 ` Eduardo Habkost 2018-10-19 8:44 ` Max Reitz 2018-10-15 21:17 ` Eduardo Habkost 2018-10-16 0:05 ` Cleber Rosa 2018-10-16 0:12 ` Eduardo Habkost 2018-10-19 9:25 ` Max Reitz 2018-10-15 14:14 ` [Qemu-devel] [PATCH 9/9] iotests: Unify log outputs between Python 2 and 3 Max Reitz 2018-10-15 22:26 ` Eduardo Habkost 2018-10-19 9:33 ` Max Reitz 2018-10-15 22:19 ` [Qemu-devel] [PATCH 0/9] iotests: Make them work for both " Philippe Mathieu-Daudé 2018-10-19 9:08 ` Max Reitz
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.