* [Qemu-devel] [PATCH 0/4] add "make check"
@ 2011-09-01 15:42 Gerd Hoffmann
2011-09-01 15:42 ` [Qemu-devel] [PATCH 1/4] Probe for libcheck by default Gerd Hoffmann
` (5 more replies)
0 siblings, 6 replies; 24+ messages in thread
From: Gerd Hoffmann @ 2011-09-01 15:42 UTC (permalink / raw)
To: qemu-devel; +Cc: Gerd Hoffmann
Hi,
This patch series intends to make unit testing easier. It adds a new
"make check" target which can be used to run all unit tests which are
currently in the tree. It also enables the unit tests by default, so
you don't have to re-run configure with a special switch.
cheers,
Gerd
Gerd Hoffmann (4):
Probe for libcheck by default.
move checks to separate variable
add "make check" target
add test-coroutine to checks
Makefile | 10 ++++++++--
configure | 7 ++++---
2 files changed, 12 insertions(+), 5 deletions(-)
^ permalink raw reply [flat|nested] 24+ messages in thread* [Qemu-devel] [PATCH 1/4] Probe for libcheck by default. 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann @ 2011-09-01 15:42 ` Gerd Hoffmann 2011-09-01 19:37 ` Anthony Liguori 2011-09-01 15:42 ` [Qemu-devel] [PATCH 2/4] move checks to separate variable Gerd Hoffmann ` (4 subsequent siblings) 5 siblings, 1 reply; 24+ messages in thread From: Gerd Hoffmann @ 2011-09-01 15:42 UTC (permalink / raw) To: qemu-devel; +Cc: Gerd Hoffmann Probe for libcheck and build checks (if found) by default. Can be explicitly disabled using --disable-check-utests. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> --- configure | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configure b/configure index 1340c33..920de9e 100755 --- a/configure +++ b/configure @@ -170,7 +170,7 @@ mixemu="no" aix="no" blobs="yes" pkgversion="" -check_utests="no" +check_utests="" user_pie="no" zero_malloc="" trace_backend="nop" -- 1.7.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 1/4] Probe for libcheck by default. 2011-09-01 15:42 ` [Qemu-devel] [PATCH 1/4] Probe for libcheck by default Gerd Hoffmann @ 2011-09-01 19:37 ` Anthony Liguori 2011-09-02 7:42 ` Gerd Hoffmann 0 siblings, 1 reply; 24+ messages in thread From: Anthony Liguori @ 2011-09-01 19:37 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: qemu-devel On 09/01/2011 10:42 AM, Gerd Hoffmann wrote: > Probe for libcheck and build checks (if found) by default. > Can be explicitly disabled using --disable-check-utests. > > Signed-off-by: Gerd Hoffmann<kraxel@redhat.com> > --- > configure | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) I think we should convert the check tests to gtest, and then have make check use gtester and gtester-report generate a single report of all of the test cases. We could then have build bot run make check and post the output. I don't want to end up with a bunch of non gtest unit tests... Regards, Anthony Liguori > > diff --git a/configure b/configure > index 1340c33..920de9e 100755 > --- a/configure > +++ b/configure > @@ -170,7 +170,7 @@ mixemu="no" > aix="no" > blobs="yes" > pkgversion="" > -check_utests="no" > +check_utests="" > user_pie="no" > zero_malloc="" > trace_backend="nop" ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 1/4] Probe for libcheck by default. 2011-09-01 19:37 ` Anthony Liguori @ 2011-09-02 7:42 ` Gerd Hoffmann 2011-09-05 7:39 ` Markus Armbruster 0 siblings, 1 reply; 24+ messages in thread From: Gerd Hoffmann @ 2011-09-02 7:42 UTC (permalink / raw) To: Anthony Liguori; +Cc: qemu-devel On 09/01/11 21:37, Anthony Liguori wrote: > On 09/01/2011 10:42 AM, Gerd Hoffmann wrote: >> Probe for libcheck and build checks (if found) by default. >> Can be explicitly disabled using --disable-check-utests. >> >> Signed-off-by: Gerd Hoffmann<kraxel@redhat.com> >> --- >> configure | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) > > I think we should convert the check tests to gtest, and then have make > check use gtester and gtester-report generate a single report of all of > the test cases. I wouldn't object, that is *way* beyond the scope of this little patch series though. > We could then have build bot run make check and post the output. Running "make check" in buildbot is indeed the motivation to do this ;) > I don't > want to end up with a bunch of non gtest unit tests... This patch series doesn't add any. It just adds some build system glue so "make check" runs the existing stuff. cheers, Gerd ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 1/4] Probe for libcheck by default. 2011-09-02 7:42 ` Gerd Hoffmann @ 2011-09-05 7:39 ` Markus Armbruster 0 siblings, 0 replies; 24+ messages in thread From: Markus Armbruster @ 2011-09-05 7:39 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: qemu-devel Gerd Hoffmann <kraxel@redhat.com> writes: > On 09/01/11 21:37, Anthony Liguori wrote: >> On 09/01/2011 10:42 AM, Gerd Hoffmann wrote: >>> Probe for libcheck and build checks (if found) by default. >>> Can be explicitly disabled using --disable-check-utests. >>> >>> Signed-off-by: Gerd Hoffmann<kraxel@redhat.com> >>> --- >>> configure | 2 +- >>> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> I think we should convert the check tests to gtest, and then have make >> check use gtester and gtester-report generate a single report of all of >> the test cases. > > I wouldn't object, that is *way* beyond the scope of this little patch > series though. One step at a time. >> We could then have build bot run make check and post the output. > > Running "make check" in buildbot is indeed the motivation to do this ;) > >> I don't >> want to end up with a bunch of non gtest unit tests... > > This patch series doesn't add any. Yep. If you want to standardize on gtest, you get to convert or remove the existing tests. Until then, we use what we have. > It just adds some build system > glue so "make check" runs the existing stuff. Running self-tests shouldn't be made any harder than that. ^ permalink raw reply [flat|nested] 24+ messages in thread
* [Qemu-devel] [PATCH 2/4] move checks to separate variable 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 1/4] Probe for libcheck by default Gerd Hoffmann @ 2011-09-01 15:42 ` Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 3/4] add "make check" target Gerd Hoffmann ` (3 subsequent siblings) 5 siblings, 0 replies; 24+ messages in thread From: Gerd Hoffmann @ 2011-09-01 15:42 UTC (permalink / raw) To: qemu-devel; +Cc: Gerd Hoffmann Create a new CHECKS variable. Put the checks there instead of adding them to the TOOLS variable. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> --- Makefile | 4 ++-- configure | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index 8606849..2963c8b 100644 --- a/Makefile +++ b/Makefile @@ -71,7 +71,7 @@ defconfig: -include config-all-devices.mak -build-all: $(DOCS) $(TOOLS) recurse-all +build-all: $(DOCS) $(TOOLS) $(CHECKS) recurse-all config-host.h: config-host.h-timestamp config-host.h-timestamp: config-host.mak @@ -205,7 +205,7 @@ clean: # avoid old build problems by removing potentially incorrect old files rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h rm -f qemu-options.def - rm -f *.o *.d *.a *.lo $(TOOLS) qemu-ga TAGS cscope.* *.pod *~ */*~ + rm -f *.o *.d *.a *.lo $(TOOLS) $(CHECKS) qemu-ga TAGS cscope.* *.pod *~ */*~ rm -Rf .libs rm -f slirp/*.o slirp/*.d audio/*.o audio/*.d block/*.o block/*.d net/*.o net/*.d fsdev/*.o fsdev/*.d ui/*.o ui/*.d qapi/*.o qapi/*.d qga/*.o qga/*.d rm -f qemu-img-cmds.h diff --git a/configure b/configure index 920de9e..44b3295 100755 --- a/configure +++ b/configure @@ -2630,8 +2630,8 @@ if test "$softmmu" = yes ; then tools="qemu-ga\$(EXESUF) $tools" fi if [ "$check_utests" = "yes" ]; then - tools="check-qint check-qstring check-qdict check-qlist $tools" - tools="check-qfloat check-qjson $tools" + checks="check-qint check-qstring check-qdict check-qlist" + checks="check-qfloat check-qjson $checks" fi fi fi @@ -3078,6 +3078,7 @@ fi echo "CONFIG_TRACE_FILE=$trace_file" >> $config_host_mak echo "TOOLS=$tools" >> $config_host_mak +echo "CHECKS=$checks" >> $config_host_mak echo "ROMS=$roms" >> $config_host_mak echo "MAKE=$make" >> $config_host_mak echo "INSTALL=$install" >> $config_host_mak -- 1.7.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [PATCH 3/4] add "make check" target 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 1/4] Probe for libcheck by default Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 2/4] move checks to separate variable Gerd Hoffmann @ 2011-09-01 15:42 ` Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 4/4] add test-coroutine to checks Gerd Hoffmann ` (2 subsequent siblings) 5 siblings, 0 replies; 24+ messages in thread From: Gerd Hoffmann @ 2011-09-01 15:42 UTC (permalink / raw) To: qemu-devel; +Cc: Gerd Hoffmann Add some Makefile glue so we have a simple "make check" to run the unit tests. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> --- Makefile | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/Makefile b/Makefile index 2963c8b..80c0bc2 100644 --- a/Makefile +++ b/Makefile @@ -289,6 +289,12 @@ endif test speed: all $(MAKE) -C tests $@ +.PHONY: check +check: $(patsubst %,run-check-%,$(CHECKS)) + +run-check-%: % + ./$< + .PHONY: TAGS TAGS: find "$(SRC_PATH)" -name '*.[hc]' -print0 | xargs -0 etags -- 1.7.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [PATCH 4/4] add test-coroutine to checks 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann ` (2 preceding siblings ...) 2011-09-01 15:42 ` [Qemu-devel] [PATCH 3/4] add "make check" target Gerd Hoffmann @ 2011-09-01 15:42 ` Gerd Hoffmann 2011-09-05 7:55 ` [Qemu-devel] [PATCH 0/4] add "make check" Markus Armbruster 2011-11-01 18:03 ` Anthony Liguori 5 siblings, 0 replies; 24+ messages in thread From: Gerd Hoffmann @ 2011-09-01 15:42 UTC (permalink / raw) To: qemu-devel; +Cc: Gerd Hoffmann There is a "test-coroutine" which isn't in the list. Add it so "make check" runs it too. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> --- configure | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/configure b/configure index 44b3295..b677801 100755 --- a/configure +++ b/configure @@ -2631,7 +2631,7 @@ if test "$softmmu" = yes ; then fi if [ "$check_utests" = "yes" ]; then checks="check-qint check-qstring check-qdict check-qlist" - checks="check-qfloat check-qjson $checks" + checks="check-qfloat check-qjson test-coroutine $checks" fi fi fi -- 1.7.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann ` (3 preceding siblings ...) 2011-09-01 15:42 ` [Qemu-devel] [PATCH 4/4] add test-coroutine to checks Gerd Hoffmann @ 2011-09-05 7:55 ` Markus Armbruster 2011-10-24 18:43 ` Eduardo Habkost 2011-11-01 18:03 ` Anthony Liguori 5 siblings, 1 reply; 24+ messages in thread From: Markus Armbruster @ 2011-09-05 7:55 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: qemu-devel Gerd Hoffmann <kraxel@redhat.com> writes: > Hi, > > This patch series intends to make unit testing easier. It adds a new > "make check" target which can be used to run all unit tests which are > currently in the tree. It also enables the unit tests by default, so > you don't have to re-run configure with a special switch. Reviewed-by: Markus Armbruster <armbru@redhat.com> One test fails, but Luiz has a fix in his tree. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-09-05 7:55 ` [Qemu-devel] [PATCH 0/4] add "make check" Markus Armbruster @ 2011-10-24 18:43 ` Eduardo Habkost 2011-10-24 18:57 ` Anthony Liguori 0 siblings, 1 reply; 24+ messages in thread From: Eduardo Habkost @ 2011-10-24 18:43 UTC (permalink / raw) To: Markus Armbruster; +Cc: Gerd Hoffmann, qemu-devel On Mon, Sep 05, 2011 at 09:55:20AM +0200, Markus Armbruster wrote: > Gerd Hoffmann <kraxel@redhat.com> writes: > > > Hi, > > > > This patch series intends to make unit testing easier. It adds a new > > "make check" target which can be used to run all unit tests which are > > currently in the tree. It also enables the unit tests by default, so > > you don't have to re-run configure with a special switch. > > Reviewed-by: Markus Armbruster <armbru@redhat.com> > > One test fails, but Luiz has a fix in his tree. Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Now all tests are passing. Why this was not applied yet? -- Eduardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-24 18:43 ` Eduardo Habkost @ 2011-10-24 18:57 ` Anthony Liguori 2011-10-25 12:21 ` Kevin Wolf 0 siblings, 1 reply; 24+ messages in thread From: Anthony Liguori @ 2011-10-24 18:57 UTC (permalink / raw) To: Markus Armbruster, Gerd Hoffmann, qemu-devel On 10/24/2011 01:43 PM, Eduardo Habkost wrote: > On Mon, Sep 05, 2011 at 09:55:20AM +0200, Markus Armbruster wrote: >> Gerd Hoffmann<kraxel@redhat.com> writes: >> >>> Hi, >>> >>> This patch series intends to make unit testing easier. It adds a new >>> "make check" target which can be used to run all unit tests which are >>> currently in the tree. It also enables the unit tests by default, so >>> you don't have to re-run configure with a special switch. >> >> Reviewed-by: Markus Armbruster<armbru@redhat.com> >> >> One test fails, but Luiz has a fix in his tree. > > Reviewed-by: Eduardo Habkost<ehabkost@redhat.com> > > Now all tests are passing. Why this was not applied yet? I was hoping for more, but maybe we just need to start here and grow organically, I'll queue it again. Regards, Anthony Liguori > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-24 18:57 ` Anthony Liguori @ 2011-10-25 12:21 ` Kevin Wolf 2011-10-25 13:27 ` Gerd Hoffmann 2011-10-25 15:03 ` Anthony Liguori 0 siblings, 2 replies; 24+ messages in thread From: Kevin Wolf @ 2011-10-25 12:21 UTC (permalink / raw) To: Anthony Liguori; +Cc: qemu-devel, Avi Kivity, Markus Armbruster, Gerd Hoffmann Am 24.10.2011 20:57, schrieb Anthony Liguori: > On 10/24/2011 01:43 PM, Eduardo Habkost wrote: >> On Mon, Sep 05, 2011 at 09:55:20AM +0200, Markus Armbruster wrote: >>> Gerd Hoffmann<kraxel@redhat.com> writes: >>> >>>> Hi, >>>> >>>> This patch series intends to make unit testing easier. It adds a new >>>> "make check" target which can be used to run all unit tests which are >>>> currently in the tree. It also enables the unit tests by default, so >>>> you don't have to re-run configure with a special switch. >>> >>> Reviewed-by: Markus Armbruster<armbru@redhat.com> >>> >>> One test fails, but Luiz has a fix in his tree. >> >> Reviewed-by: Eduardo Habkost<ehabkost@redhat.com> >> >> Now all tests are passing. Why this was not applied yet? > > I was hoping for more, but maybe we just need to start here and grow > organically, I'll queue it again. A while ago I played with some simple IDE tests. It basically was a small x86 kernel with an empty image that sends IDE commands and prints some results, and a script that invokes the guest and checks whether the test has passed or failed. So at first I started with my own multiboot kernel and copied over some parts of kvm-unittest's libc. Clearly not the best idea once it's more than a couple of lines, so at some point I took the code and integrated with my real kvm-unittests repository. Now I don't have to duplicate code any more, but at the same time there's no chance that a 'make check' in an upstream qemu tree could run this. Tests for other devices will have exactly the same problem. Any suggestions on how to go forward with this kind of tests? Should this go into qemu or into kvm-unittests? Or should kvm-unittests be merged into the qemu tree? Or is the approach completely wrong? Kevin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 12:21 ` Kevin Wolf @ 2011-10-25 13:27 ` Gerd Hoffmann 2011-10-25 14:16 ` Kevin Wolf 2011-10-25 15:03 ` Eduardo Habkost 2011-10-25 15:03 ` Anthony Liguori 1 sibling, 2 replies; 24+ messages in thread From: Gerd Hoffmann @ 2011-10-25 13:27 UTC (permalink / raw) To: Kevin Wolf; +Cc: Avi Kivity, Markus Armbruster, qemu-devel [-- Attachment #1: Type: text/plain, Size: 1598 bytes --] Hi, >> I was hoping for more, but maybe we just need to start here and grow >> organically, I'll queue it again. > > A while ago I played with some simple IDE tests. It basically was a > small x86 kernel with an empty image that sends IDE commands and prints > some results, and a script that invokes the guest and checks whether the > test has passed or failed. That reminds me that I've started toying with running tests inside a guest too. Stopped working on it a while back due to other priorities. Attached what I have so far. > So at first I started with my own multiboot kernel and copied over some > parts of kvm-unittest's libc. Clearly not the best idea once it's more > than a couple of lines, so at some point I took the code and integrated > with my real kvm-unittests repository. > > Now I don't have to duplicate code any more, but at the same time > there's no chance that a 'make check' in an upstream qemu tree could run > this. Tests for other devices will have exactly the same problem. > > Any suggestions on how to go forward with this kind of tests? Should > this go into qemu or into kvm-unittests? Or should kvm-unittests be > merged into the qemu tree? Or is the approach completely wrong? I think we should have some framework to run tests inside the guest in the qemu source tree. I'm not sure kvm-unittests is the right tool for the job though. It is quite low-level and mainly targets the kvm bits inside the linux kernel. Testing -- for example -- usb device emulation would pretty much require writing a usb stack for kvm-unitests ... cheers, Gerd [-- Attachment #2: 0001-initramfs-test-framework.patch --] [-- Type: text/plain, Size: 16053 bytes --] From 096f68ea08c3c4baf1bbdc549b257a67ecc87e25 Mon Sep 17 00:00:00 2001 From: Gerd Hoffmann <kraxel@redhat.com> Date: Tue, 13 Sep 2011 17:38:37 +0200 Subject: [PATCH] initramfs test framework Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> --- initramfs/.gitignore | 3 + initramfs/10-qemu-udev.rules | 5 + initramfs/Makefile | 36 +++++++ initramfs/README | 44 ++++++++ initramfs/init.c | 225 ++++++++++++++++++++++++++++++++++++++++++ initramfs/initramfs-boot | 32 ++++++ initramfs/initramfs-create | 111 +++++++++++++++++++++ initramfs/test-ehci | 3 + initramfs/test-ehci.good | 8 ++ initramfs/test-hello.c | 7 ++ initramfs/test-hello.good | 1 + initramfs/test-uhci | 3 + initramfs/test-uhci.good | 3 + 13 files changed, 481 insertions(+), 0 deletions(-) create mode 100644 initramfs/.gitignore create mode 100644 initramfs/10-qemu-udev.rules create mode 100644 initramfs/Makefile create mode 100644 initramfs/README create mode 100644 initramfs/init.c create mode 100755 initramfs/initramfs-boot create mode 100755 initramfs/initramfs-create create mode 100755 initramfs/test-ehci create mode 100644 initramfs/test-ehci.good create mode 100644 initramfs/test-hello.c create mode 100644 initramfs/test-hello.good create mode 100755 initramfs/test-uhci create mode 100644 initramfs/test-uhci.good diff --git a/initramfs/.gitignore b/initramfs/.gitignore new file mode 100644 index 0000000..8ece42c --- /dev/null +++ b/initramfs/.gitignore @@ -0,0 +1,3 @@ +initramfs.cpio.gz +init +test-hello diff --git a/initramfs/10-qemu-udev.rules b/initramfs/10-qemu-udev.rules new file mode 100644 index 0000000..fb5cc0a --- /dev/null +++ b/initramfs/10-qemu-udev.rules @@ -0,0 +1,5 @@ +# load modules +DRIVER!="?*", ENV{MODALIAS}=="?*", RUN+="/sbin/modprobe -b $env{MODALIAS}" + +# virtio console +KERNEL=="vport*", ATTR{name}=="?*", SYMLINK+="virtio-ports/$attr{name}" diff --git a/initramfs/Makefile b/initramfs/Makefile new file mode 100644 index 0000000..2db2b76 --- /dev/null +++ b/initramfs/Makefile @@ -0,0 +1,36 @@ + +CC := gcc +CFLAGS := -O2 -g -Wall +LDFLAGS := -lutil + +TOOLS := init +TEST_C := test-hello +TEST_SH := test-uhci test-ehci + +TARGETS := $(TOOLS) $(TEST_C) initramfs.cpio.gz +TESTS := $(TEST_C) $(TEST_SH) + +run-test-uhci : QEMU_ARGS := -usb -device usb-tablet +run-test-ehci : QEMU_ARGS := -readconfig ../docs/ich9-ehci-uhci.cfg + +default all: $(TARGETS) + +clean: + rm -f $(TARGETS) *.o *~ + rm -f org.qemu.initramfs.log + +init: init.o +test-hello: test-hello.o + +initramfs.cpio.gz: $(TOOLS) $(TESTS) initramfs-create + ./initramfs-create $@ $(TESTS) + +boot shell: $(TARGETS) + ./initramfs-boot + +run-test-%: $(TARGETS) + ./initramfs-boot /tests/test-$* $(QEMU_ARGS) + diff -u org.qemu.initramfs.log test-$*.good + +run-tests: $(patsubst %,run-%,$(TESTS)) + diff --git a/initramfs/README b/initramfs/README new file mode 100644 index 0000000..a9504ad --- /dev/null +++ b/initramfs/README @@ -0,0 +1,44 @@ + +This is an experimental test framework. + +Design goals +------------ + + * Allow running tests within a guest. + * Be small enougth that it can easily be included in + the qemu source tree. + * Don't require setup and/or downloading stuff + (i.e. guest images) from the internet. + * Be easy to use. + + +How it works +------------ + +It creates a linux initramfs from the bits found on the host machine, +then boots the host kernel with the initramfs just created within +qemu. A special init handles the setup (create core devices, mount +filesystems, start udev), command execution and shutdown. + +Obviously requires a linux host. It also needs udev for device setup +and module loading and virtio-serial support for logging. + + +Getting started +--------------- + +Just type "make boot", a few seconds later you should be greeted by +the guests bash prompt. You can look around now. Exiting the shell +will shutdown the guest. You'll find the shell output logged in the +"org.qemu.initramfs.log" file. + + +Run tests +--------- + +Type "make run-test-$name" for a single test or "make run-tests" to +run all of them. This will run the test program instead of the shell, +then compare the actual output with the expected output. + +This can probably be refined ... + diff --git a/initramfs/init.c b/initramfs/init.c new file mode 100644 index 0000000..94a8764 --- /dev/null +++ b/initramfs/init.c @@ -0,0 +1,225 @@ +#include <stdio.h> +#include <stdlib.h> +#include <stdarg.h> +#include <stdbool.h> +#include <unistd.h> +#include <string.h> +#include <errno.h> +#include <fcntl.h> +#include <pty.h> + +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/wait.h> +#include <sys/mount.h> +#include <sys/reboot.h> +#include <sys/select.h> + +#define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0])) + +static const char *logdev = "/dev/virtio-ports/org.qemu.initramfs.log"; + +static struct { + mode_t mode; + const char *path; + int major; + int minor; +} cdevs[] = { + { 0666, "/dev/null", 1, 3 }, + { 0600, "/dev/console", 5, 1 }, + { 0666, "/dev/ptmx", 5, 2 }, + { 0660, "/dev/kmsg", 1, 11 }, +}; + +static struct { + const char *dest; + const char *type; +} fs[] = { + { "/proc", "proc" }, + { "/sys", "sysfs" }, + { "/sys/kernel/debug", "debugfs" }, + { "/dev/pts", "devpts" }, + { "/dev/shm", "tmpfs" }, +}; + +int run(const char *cmd, ...) +{ + va_list args; + char *argv[16], *arg; + int i, pid; + + va_start(args, cmd); + argv[0] = (char*)cmd; + for (i = 1; i < ARRAY_SIZE(argv)-1; i++) { + arg = va_arg(args, char*); + if (arg == NULL) { + break; + } + argv[i] = arg; + } + argv[i] = NULL; + va_end(args); + + pid = fork(); + if (pid != 0) { + /* parent */ + if (pid < 0) { + perror("fork"); + } + return pid; + } + /* child */ + execv(cmd, argv); + fprintf(stderr, "exec %s: %s\n", cmd, strerror(errno)); + exit(1); +} + +int forward(int from_fd, int to_fd, int log_fd) +{ + int len, pos, rc; + char buf[512]; + + len = read(from_fd, buf, sizeof(buf)); + if (len < 0) { + return len; + } + for (pos = 0; pos < len; pos += rc) { + rc = write(to_fd, buf + pos, len - pos); + if (rc < 0) { + return rc; + } + } + if (log_fd != -1) { + for (pos = 0; pos < len; pos += rc) { + rc = write(log_fd, buf + pos, len - pos); + if (rc < 0) { + return rc; + } + } + } + return len; +} + +static void +tty_raw(int fd) +{ + struct termios tattr; + + tcgetattr(fd, &tattr); + tattr.c_lflag &= ~(ICANON|ECHO); + tattr.c_cc[VMIN] = 1; + tattr.c_cc[VTIME] = 0; + tcsetattr(fd, TCSAFLUSH, &tattr); +} + +int main(int argc, char *argv[]) +{ + const char *command; + pid_t cmdpid, pid; + int status, i, rc, pty, log; + bool cmd_exit, eof_seen; + + /* say hi */ + fprintf(stderr, "-*- qemu initramfs starting -*-\n"); + + /* mount filesystems */ + for (i = 0; i < ARRAY_SIZE(fs); i++) { + rc = mount(fs[i].type, fs[i].dest, fs[i].type, 0, NULL); + if (rc < 0) { + fprintf(stderr, "mount %s at %s: %s\n", + fs[i].type, fs[i].dest, strerror(errno)); + } + } + + /* create basic chardevs */ + for (i = 0; i < ARRAY_SIZE(cdevs); i++) { + rc = mknod(cdevs[i].path, cdevs[i].mode | S_IFCHR, + makedev(cdevs[i].major, cdevs[i].minor)); + if (rc < 0 && errno != EEXIST) { + fprintf(stderr, "mknod %s : %s\n", + cdevs[i].path, strerror(errno)); + } + } + + /* start udev, let it create devices */ + fprintf(stderr, "-*- starting udev -*-\n"); + run("/sbin/depmod", "-a", NULL); + run("/sbin/udevd", "--daemon", "--resolve-names=never", NULL); + run("/sbin/udevadm", "trigger", "--action=add", NULL); + + /* wait until virtio-serial is up'n'running */ + for (log = -1, i = 0; log == -1 && i < 32; i++) { + sleep(1); + run("/sbin/udevadm", "settle", NULL); + log = open(logdev, O_WRONLY); + } + if (log == -1) { + fprintf(stderr, "open %s: %s\n", logdev, strerror(errno)); + } + + /* setup environment */ + setenv("PATH", "/sbin:/bin", 1); + + /* run our command in a pseuto tty */ + command = getenv("QEMU_RUN"); + if (command == NULL) { + command = "/bin/bash"; + } + fprintf(stderr, "-*- running %s -*-\n", command); + + tty_raw(0); + cmdpid = forkpty(&pty, NULL, NULL, NULL); + if (cmdpid == 0) { + /* child */ + execl(command, command, NULL); + fprintf(stderr, "exec %s: %s\n", command, strerror(errno)); + exit(1); + } + + /* main loop */ + cmd_exit = false; + eof_seen = false; + do { + fd_set rd; + + /* reap children and zombies */ + for (;;) { + pid = waitpid(-1, &status, WNOHANG); + if (pid <= 0) { + break; + } + if (pid == cmdpid) { + cmd_exit = true; + } + } + + /* forward data between stdio (aka /dev/console) and pseudo tty */ + if (!eof_seen) { + FD_ZERO(&rd); + FD_SET(0, &rd); + FD_SET(pty, &rd); + rc = select(pty+1, &rd, NULL, NULL, NULL); + if (rc > 0) { + if (FD_ISSET(0, &rd)) { + /* stdin -> pseudo tty */ + if (forward(0, pty, -1) <= 0) { + eof_seen = 1; + } + } + if (FD_ISSET(pty, &rd)) { + /* pseudo tty -> stdout + log */ + if (forward(pty, 1, log) <= 0) { + eof_seen = 1; + } + } + } + } + } while (!cmd_exit || !eof_seen); + + /* powerdown vm */ + fprintf(stderr, "-*- qemu initramfs done -*-\n"); + reboot(RB_POWER_OFF); + + /* keep gcc happy ;) */ + return 0; +} diff --git a/initramfs/initramfs-boot b/initramfs/initramfs-boot new file mode 100755 index 0000000..4884a6d --- /dev/null +++ b/initramfs/initramfs-boot @@ -0,0 +1,32 @@ +#!/bin/sh + +base="$(dirname $0)" +command="$1" +shift; + +case "$(uname -m)" in + x86_64) qemu="x86_64-softmmu/qemu-system-x86_64" + ;; + i?86) qemu="i386-softmmu/qemu-system-i386" + ;; + *) echo "unknown arch: $(uname -m)" + exit 1 + ;; +esac + +kernel="$(echo /boot/vmlinu*$(uname -r)*)" +initrd="initramfs.cpio.gz" +append="console=ttyS0" +if test "$command" != ""; then + append="$append QEMU_RUN=$command" +fi + +exec ../$qemu -nographic -no-reboot \ + -machine accel=kvm:tcg \ + -kernel "$kernel" \ + -initrd "$initrd" \ + -append "$append" \ + -chardev file,id=cmdlog,path=org.qemu.initramfs.log \ + -device virtio-serial \ + -device virtserialport,name=org.qemu.initramfs.log,chardev=cmdlog \ + "$@" diff --git a/initramfs/initramfs-create b/initramfs/initramfs-create new file mode 100755 index 0000000..4525c11 --- /dev/null +++ b/initramfs/initramfs-create @@ -0,0 +1,111 @@ +#!/bin/bash +# +# Create a simple linux initramfs with the stuff found on the host. +# Intented to be used for a quick boot test with the host kernel. +# + +base="$(dirname $0)" +file="${1-initramfs.cpio.gz}"; shift +tests="$*" + +# create work dir +WORK="${TMPDIR-/tmp}/${0##*/}-$$" +mkdir "$WORK" || exit 1 +trap 'rm -rf "$WORK"' EXIT + + +############################################################################## +# helper functions + +function add_dirs() { + local dest="$WORK/fs" + local dir + for dir in dev dev/pts dev/shm etc proc sys \ + var var/tmp /var/log tmp + do + mkdir -p "$dest/$dir" + done +} + +function add_binary() { + local dest="$WORK/fs$1"; shift + local item + + mkdir -p "$dest" + for item in $*; do + cp -L "$(which $item)" "$dest" || exit 1 + done +} + +function add_symlink() { + local target="$1" + local name="$2" + ln -s "$target" "$WORK/fs$name" +} + +function add_data() { + local dest="$WORK/fs" + local item + + for item in $*; do + mkdir -p $(dirname "$dest/$item") + cp -L "$item" "$dest/$item" || exit 1 + done +} + +function add_data_to_dir() { + local dest="$WORK/fs$1"; shift + local item + + mkdir -p "$dest" + for item in $*; do + cp -L "$item" "$dest" || exit 1 + done +} + +function add_libs() { + local item dest + for item in $(ldd $WORK/fs/init $WORK/fs/bin/* $WORK/fs/sbin/* \ + $WORK/fs/tests/* $WORK/fs/lib/udev/*id); do + test -f "$item" || continue + test -f "$WORK/fs$item" && continue + dest=$(dirname "$WORK/fs$item") + mkdir -p "$dest" + cp -L "$item" "$dest" + done +} + +function add_modules() { + local item + + echo -n > "$WORK/modules" + for item in $*; do + modprobe --show-depends $item 2>/dev/null \ + | awk '{ print $2 }' >> "$WORK/modules" + done + add_data $(sort "$WORK/modules" | uniq) +} + + +############################################################################## +# main + +add_dirs +add_binary / $base/init +for t in $tests; do + add_binary /tests $base/$t; +done +add_binary /bin bash ls cat more dmesg ps uname find sort grep uniq +add_binary /sbin lspci lsusb mount umount udevd udevadm blkid +add_binary /sbin depmod insmod lsmod modinfo modprobe rmmod ip +add_symlink bash /bin/sh +add_data /usr/share/hwdata/*.ids +add_data /lib/udev/*id +add_data /lib/udev/rules.d/60-persistent-storage.rules +add_data_to_dir /lib/udev/rules.d $base/10-qemu-udev.rules +add_libs +add_modules virtio_pci virtio_blk virtio_net virtio_console \ + virtio_balloon virtio-rng 9pnet_virtio 9p \ + ata_piix ahci sd_mod sr_mod sg e1000 8139cp + +(cd $WORK/fs; find -print | cpio -o -R 0:0 -H newc) | gzip > "$file" diff --git a/initramfs/test-ehci b/initramfs/test-ehci new file mode 100755 index 0000000..bd4bb5c --- /dev/null +++ b/initramfs/test-ehci @@ -0,0 +1,3 @@ +#!/bin/sh +/sbin/lspci -s1d +/sbin/lsusb | sort diff --git a/initramfs/test-ehci.good b/initramfs/test-ehci.good new file mode 100644 index 0000000..dd9c6b8 --- /dev/null +++ b/initramfs/test-ehci.good @@ -0,0 +1,8 @@ +00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03) +00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03) +00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03) +00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03) +Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub +Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub +Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub +Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub diff --git a/initramfs/test-hello.c b/initramfs/test-hello.c new file mode 100644 index 0000000..e5ce2d2 --- /dev/null +++ b/initramfs/test-hello.c @@ -0,0 +1,7 @@ +#include <stdio.h> + +int main(int argc, char *argv[]) +{ + printf("Hello world!\n"); + return 0; +} diff --git a/initramfs/test-hello.good b/initramfs/test-hello.good new file mode 100644 index 0000000..dfd6895 --- /dev/null +++ b/initramfs/test-hello.good @@ -0,0 +1 @@ +Hello world! diff --git a/initramfs/test-uhci b/initramfs/test-uhci new file mode 100755 index 0000000..0af70e0 --- /dev/null +++ b/initramfs/test-uhci @@ -0,0 +1,3 @@ +#!/bin/sh +/sbin/lspci -s1.2 +/sbin/lsusb | sort \ No newline at end of file diff --git a/initramfs/test-uhci.good b/initramfs/test-uhci.good new file mode 100644 index 0000000..c87271e --- /dev/null +++ b/initramfs/test-uhci.good @@ -0,0 +1,3 @@ +00:01.2 USB Controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01) +Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub +Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd -- 1.7.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 13:27 ` Gerd Hoffmann @ 2011-10-25 14:16 ` Kevin Wolf 2011-10-25 15:03 ` Eduardo Habkost 1 sibling, 0 replies; 24+ messages in thread From: Kevin Wolf @ 2011-10-25 14:16 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: Avi Kivity, Markus Armbruster, qemu-devel Am 25.10.2011 15:27, schrieb Gerd Hoffmann: > Hi, > >>> I was hoping for more, but maybe we just need to start here and grow >>> organically, I'll queue it again. >> >> A while ago I played with some simple IDE tests. It basically was a >> small x86 kernel with an empty image that sends IDE commands and prints >> some results, and a script that invokes the guest and checks whether the >> test has passed or failed. > > That reminds me that I've started toying with running tests inside a > guest too. Stopped working on it a while back due to other priorities. > Attached what I have so far. > >> So at first I started with my own multiboot kernel and copied over some >> parts of kvm-unittest's libc. Clearly not the best idea once it's more >> than a couple of lines, so at some point I took the code and integrated >> with my real kvm-unittests repository. >> >> Now I don't have to duplicate code any more, but at the same time >> there's no chance that a 'make check' in an upstream qemu tree could run >> this. Tests for other devices will have exactly the same problem. >> >> Any suggestions on how to go forward with this kind of tests? Should >> this go into qemu or into kvm-unittests? Or should kvm-unittests be >> merged into the qemu tree? Or is the approach completely wrong? > > I think we should have some framework to run tests inside the guest in > the qemu source tree. I'm not sure kvm-unittests is the right tool for > the job though. It is quite low-level and mainly targets the kvm bits > inside the linux kernel. Testing -- for example -- usb device emulation > would pretty much require writing a usb stack for kvm-unitests ... Which is more or less the only way to do a reasonably complete test (particularly including the error handling paths that a current Linux might never exercise) at least for the host controllers. I'm not sure if you can send arbitrary USB packets from Linux (is USB passthrough really direct passthrough or does the kernel mess with it?), so maybe for everything above host controllers you could use that. If you can't, you're back to your own USB stack. Kevin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 13:27 ` Gerd Hoffmann 2011-10-25 14:16 ` Kevin Wolf @ 2011-10-25 15:03 ` Eduardo Habkost 2011-10-25 15:22 ` Kevin Wolf ` (2 more replies) 1 sibling, 3 replies; 24+ messages in thread From: Eduardo Habkost @ 2011-10-25 15:03 UTC (permalink / raw) To: Gerd Hoffmann Cc: Kevin Wolf, Lucas Meneghel Rodrigues, qemu-devel, Markus Armbruster, Avi Kivity, Cleber Rosa On Tue, Oct 25, 2011 at 03:27:35PM +0200, Gerd Hoffmann wrote: [...] > > A while ago I played with some simple IDE tests. It basically was a > > small x86 kernel with an empty image that sends IDE commands and prints > > some results, and a script that invokes the guest and checks whether the > > test has passed or failed. > > That reminds me that I've started toying with running tests inside a > guest too. Stopped working on it a while back due to other priorities. > Attached what I have so far. > > > So at first I started with my own multiboot kernel and copied over some > > parts of kvm-unittest's libc. Clearly not the best idea once it's more > > than a couple of lines, so at some point I took the code and integrated > > with my real kvm-unittests repository. > > > > Now I don't have to duplicate code any more, but at the same time > > there's no chance that a 'make check' in an upstream qemu tree could run > > this. Tests for other devices will have exactly the same problem. > > > > Any suggestions on how to go forward with this kind of tests? Should > > this go into qemu or into kvm-unittests? Or should kvm-unittests be > > merged into the qemu tree? Or is the approach completely wrong? > > I think we should have some framework to run tests inside the guest in > the qemu source tree. I'm not sure kvm-unittests is the right tool for > the job though. It is quite low-level and mainly targets the kvm bits > inside the linux kernel. Testing -- for example -- usb device emulation > would pretty much require writing a usb stack for kvm-unitests ... We have a framework to run tests inside a fully-installed guest, that's KVM-Autotest. But maybe it's too much for the kind of tests you need, I don't know. There are different "levels" of testing, with different reequirements, and we need to have good tools for all levels. Just trying to enumerate the kind of tests somebody may need: A) Simple unit tests for internal qemu C functions - 'make check' can run them, using either libcheck or gtest. B) Functional tests that tests actual virtualization/emulation, but only of some specific subsystems, not using a fully-featured qemu process. - We don't have anything like that, today, right? I am not sure we need it. C) Functional tests that just need to run a small binary with no OS installed in the guest, but running a fully-feature qemu process. - The tests in the 'tests' directory do this, right? kvm-unittests does this, right? D) Functional tests that need a minimal OS installed, with, e.g., at least a Linux kernel and a shell. - This is what Gerd's patch below does, right? Also, KVM-Autotest can be used for this. E) Functional tests that need a full OS installed and configured. - Today we use KVM-Autotest for this. Does the above model look correct/complete, or is there some case I missed? > > cheers, > Gerd > From 096f68ea08c3c4baf1bbdc549b257a67ecc87e25 Mon Sep 17 00:00:00 2001 > From: Gerd Hoffmann <kraxel@redhat.com> > Date: Tue, 13 Sep 2011 17:38:37 +0200 > Subject: [PATCH] initramfs test framework > > Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> > --- > initramfs/.gitignore | 3 + > initramfs/10-qemu-udev.rules | 5 + > initramfs/Makefile | 36 +++++++ > initramfs/README | 44 ++++++++ > initramfs/init.c | 225 ++++++++++++++++++++++++++++++++++++++++++ > initramfs/initramfs-boot | 32 ++++++ > initramfs/initramfs-create | 111 +++++++++++++++++++++ > initramfs/test-ehci | 3 + > initramfs/test-ehci.good | 8 ++ > initramfs/test-hello.c | 7 ++ > initramfs/test-hello.good | 1 + > initramfs/test-uhci | 3 + > initramfs/test-uhci.good | 3 + > 13 files changed, 481 insertions(+), 0 deletions(-) > create mode 100644 initramfs/.gitignore > create mode 100644 initramfs/10-qemu-udev.rules > create mode 100644 initramfs/Makefile > create mode 100644 initramfs/README > create mode 100644 initramfs/init.c > create mode 100755 initramfs/initramfs-boot > create mode 100755 initramfs/initramfs-create > create mode 100755 initramfs/test-ehci > create mode 100644 initramfs/test-ehci.good > create mode 100644 initramfs/test-hello.c > create mode 100644 initramfs/test-hello.good > create mode 100755 initramfs/test-uhci > create mode 100644 initramfs/test-uhci.good > -- Eduardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 15:03 ` Eduardo Habkost @ 2011-10-25 15:22 ` Kevin Wolf 2011-10-26 20:49 ` Anthony Liguori 2011-10-25 15:54 ` Gerd Hoffmann 2011-10-25 16:30 ` Lucas Meneghel Rodrigues 2 siblings, 1 reply; 24+ messages in thread From: Kevin Wolf @ 2011-10-25 15:22 UTC (permalink / raw) To: Gerd Hoffmann, Avi Kivity, Markus Armbruster, qemu-devel, Lucas Meneghel Rodrigues, Cleber Rosa Am 25.10.2011 17:03, schrieb Eduardo Habkost: > On Tue, Oct 25, 2011 at 03:27:35PM +0200, Gerd Hoffmann wrote: > [...] >>> A while ago I played with some simple IDE tests. It basically was a >>> small x86 kernel with an empty image that sends IDE commands and prints >>> some results, and a script that invokes the guest and checks whether the >>> test has passed or failed. >> >> That reminds me that I've started toying with running tests inside a >> guest too. Stopped working on it a while back due to other priorities. >> Attached what I have so far. >> >>> So at first I started with my own multiboot kernel and copied over some >>> parts of kvm-unittest's libc. Clearly not the best idea once it's more >>> than a couple of lines, so at some point I took the code and integrated >>> with my real kvm-unittests repository. >>> >>> Now I don't have to duplicate code any more, but at the same time >>> there's no chance that a 'make check' in an upstream qemu tree could run >>> this. Tests for other devices will have exactly the same problem. >>> >>> Any suggestions on how to go forward with this kind of tests? Should >>> this go into qemu or into kvm-unittests? Or should kvm-unittests be >>> merged into the qemu tree? Or is the approach completely wrong? >> >> I think we should have some framework to run tests inside the guest in >> the qemu source tree. I'm not sure kvm-unittests is the right tool for >> the job though. It is quite low-level and mainly targets the kvm bits >> inside the linux kernel. Testing -- for example -- usb device emulation >> would pretty much require writing a usb stack for kvm-unitests ... > > We have a framework to run tests inside a fully-installed guest, that's > KVM-Autotest. But maybe it's too much for the kind of tests you need, I > don't know. There are different "levels" of testing, with different > reequirements, and we need to have good tools for all levels. Autotest works for an entirely different use case. Autotest is the right tool when you can let the tests run over night or over the weekend. Good to run regularly on the current git tree, the right thing to do to test release candidates. It's the wrong tool for testing the patch that I'm currently working on. The focus here is on being able to run very quick tests. I don't want to wait for a guest installation to complete, and I don't even necessarily want to wait for a full-blown Linux guest to boot up. > Just trying to enumerate the kind of tests somebody may need: > > A) Simple unit tests for internal qemu C functions > - 'make check' can run them, using either libcheck or gtest. > B) Functional tests that tests actual virtualization/emulation, but only > of some specific subsystems, not using a fully-featured qemu process. > - We don't have anything like that, today, right? I am not sure we > need it. I think qemu-iotests could be considered an instance of B) > C) Functional tests that just need to run a small binary with no OS > installed in the guest, but running a fully-feature qemu process. > - The tests in the 'tests' directory do this, right? kvm-unittests > does this, right? Not sure what test/ does, but for kvm-unittests yes. And this is also what I was talking about. > D) Functional tests that need a minimal OS installed, with, e.g., at > least a Linux kernel and a shell. > - This is what Gerd's patch below does, right? Also, KVM-Autotest can > be used for this. > E) Functional tests that need a full OS installed and configured. > - Today we use KVM-Autotest for this. > > > Does the above model look correct/complete, or is there some case I > missed? I think it covers what we need. Kevin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 15:22 ` Kevin Wolf @ 2011-10-26 20:49 ` Anthony Liguori 2011-10-27 8:20 ` Kevin Wolf 0 siblings, 1 reply; 24+ messages in thread From: Anthony Liguori @ 2011-10-26 20:49 UTC (permalink / raw) To: Kevin Wolf Cc: Lucas Meneghel Rodrigues, qemu-devel, Markus Armbruster, Gerd Hoffmann, Cleber Rosa, Avi Kivity On 10/25/2011 10:22 AM, Kevin Wolf wrote: > Am 25.10.2011 17:03, schrieb Eduardo Habkost: > I think qemu-iotests could be considered an instance of B) > >> C) Functional tests that just need to run a small binary with no OS >> installed in the guest, but running a fully-feature qemu process. >> - The tests in the 'tests' directory do this, right? kvm-unittests >> does this, right? > > Not sure what test/ does, but for kvm-unittests yes. And this is also > what I was talking about. Thinking more about this... We could add a new '-x-test-server CHR' option. When this option is added, it would do the following: 1) Open CHR character device 2) Use /dev/shm for guest memory 3) Listen for connections on CHR 4) When something connects to CHR a) reset device model b) send /dev/shm fd over CHR c) register CPU physical memory client 1. upon CPU physical memory changes, send the change info over CHR d) instead of doing [kvm_]cpu_exec(), block reading on CHR So when you launch qemu with -x-test-server, it'll sit there doing nothing terribly useful. But this lets you write a program that connects to CHR, and then by mapping {out,in}[bwl] to RPCs over the connection, and accessing RAM via mmap()'ing the passed fd using the client mapping table, you can essentially write kvm-unittest style tests while still having full access to libc. And since each test program can reset QEMU after running, you could very nicely tie into something like gtest as a unit test framework. I think it's pretty appealing from a debugability perspective too. It also means that it's possible to have 100% C test cases such that you could still build something like ppc64-softmmu and run it against the written test cases without having to really understand ppc64 assembly or have a ppc64 build environment (to generate native binaries to run under ppc64 TCG). I think this could work out fairly well as a unit test framework. Regards, Anthony Liguori > >> D) Functional tests that need a minimal OS installed, with, e.g., at >> least a Linux kernel and a shell. >> - This is what Gerd's patch below does, right? Also, KVM-Autotest can >> be used for this. >> E) Functional tests that need a full OS installed and configured. >> - Today we use KVM-Autotest for this. >> >> >> Does the above model look correct/complete, or is there some case I >> missed? > > I think it covers what we need. > > Kevin > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-26 20:49 ` Anthony Liguori @ 2011-10-27 8:20 ` Kevin Wolf 2011-10-27 17:58 ` Anthony Liguori 0 siblings, 1 reply; 24+ messages in thread From: Kevin Wolf @ 2011-10-27 8:20 UTC (permalink / raw) To: Anthony Liguori Cc: Lucas Meneghel Rodrigues, qemu-devel, Markus Armbruster, Gerd Hoffmann, Cleber Rosa, Avi Kivity Am 26.10.2011 22:49, schrieb Anthony Liguori: > On 10/25/2011 10:22 AM, Kevin Wolf wrote: >> Am 25.10.2011 17:03, schrieb Eduardo Habkost: >> I think qemu-iotests could be considered an instance of B) >> >>> C) Functional tests that just need to run a small binary with no OS >>> installed in the guest, but running a fully-feature qemu process. >>> - The tests in the 'tests' directory do this, right? kvm-unittests >>> does this, right? >> >> Not sure what test/ does, but for kvm-unittests yes. And this is also >> what I was talking about. > > Thinking more about this... > > We could add a new '-x-test-server CHR' option. When this option is added, it > would do the following: > > 1) Open CHR character device > 2) Use /dev/shm for guest memory > 3) Listen for connections on CHR > 4) When something connects to CHR > a) reset device model > b) send /dev/shm fd over CHR > c) register CPU physical memory client > 1. upon CPU physical memory changes, send the change info over CHR > d) instead of doing [kvm_]cpu_exec(), block reading on CHR > > So when you launch qemu with -x-test-server, it'll sit there doing nothing > terribly useful. But this lets you write a program that connects to CHR, and > then by mapping {out,in}[bwl] to RPCs over the connection, and accessing RAM via > mmap()'ing the passed fd using the client mapping table, you can essentially > write kvm-unittest style tests while still having full access to libc. IRQs need to go through the connection as well. Oh, and you would finally have a C user for libqmp. The test cases definitely need to be able to access the monitor. For example I would really love to have test cases for the I/O error paths that stop the VM (or actually it's the resume that must be tested). > And since each test program can reset QEMU after running, you could very nicely > tie into something like gtest as a unit test framework. I think it's pretty > appealing from a debugability perspective too. > > It also means that it's possible to have 100% C test cases such that you could > still build something like ppc64-softmmu and run it against the written test > cases without having to really understand ppc64 assembly or have a ppc64 build > environment (to generate native binaries to run under ppc64 TCG). > > I think this could work out fairly well as a unit test framework. Sounds great, where are the patches? ;-) Kevin ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-27 8:20 ` Kevin Wolf @ 2011-10-27 17:58 ` Anthony Liguori 2011-10-27 21:22 ` Michael Roth 0 siblings, 1 reply; 24+ messages in thread From: Anthony Liguori @ 2011-10-27 17:58 UTC (permalink / raw) To: Kevin Wolf Cc: Lucas Meneghel Rodrigues, qemu-devel, Markus Armbruster, Avi Kivity, Cleber Rosa, Gerd Hoffmann On 10/27/2011 03:20 AM, Kevin Wolf wrote: > Am 26.10.2011 22:49, schrieb Anthony Liguori: >> On 10/25/2011 10:22 AM, Kevin Wolf wrote: >>> Am 25.10.2011 17:03, schrieb Eduardo Habkost: >>> I think qemu-iotests could be considered an instance of B) >>> >>>> C) Functional tests that just need to run a small binary with no OS >>>> installed in the guest, but running a fully-feature qemu process. >>>> - The tests in the 'tests' directory do this, right? kvm-unittests >>>> does this, right? >>> >>> Not sure what test/ does, but for kvm-unittests yes. And this is also >>> what I was talking about. >> >> Thinking more about this... >> >> We could add a new '-x-test-server CHR' option. When this option is added, it >> would do the following: >> >> 1) Open CHR character device >> 2) Use /dev/shm for guest memory >> 3) Listen for connections on CHR >> 4) When something connects to CHR >> a) reset device model >> b) send /dev/shm fd over CHR >> c) register CPU physical memory client >> 1. upon CPU physical memory changes, send the change info over CHR >> d) instead of doing [kvm_]cpu_exec(), block reading on CHR >> >> So when you launch qemu with -x-test-server, it'll sit there doing nothing >> terribly useful. But this lets you write a program that connects to CHR, and >> then by mapping {out,in}[bwl] to RPCs over the connection, and accessing RAM via >> mmap()'ing the passed fd using the client mapping table, you can essentially >> write kvm-unittest style tests while still having full access to libc. > > IRQs need to go through the connection as well. Yes, forgot to mention that. > > Oh, and you would finally have a C user for libqmp. The test cases > definitely need to be able to access the monitor. For example I would > really love to have test cases for the I/O error paths that stop the VM > (or actually it's the resume that must be tested). Yeah, tunnelling a monitor session sounds like a really good idea. >> And since each test program can reset QEMU after running, you could very nicely >> tie into something like gtest as a unit test framework. I think it's pretty >> appealing from a debugability perspective too. >> >> It also means that it's possible to have 100% C test cases such that you could >> still build something like ppc64-softmmu and run it against the written test >> cases without having to really understand ppc64 assembly or have a ppc64 build >> environment (to generate native binaries to run under ppc64 TCG). >> >> I think this could work out fairly well as a unit test framework. > > Sounds great, where are the patches? ;-) Heh, need to find a volunteer although I spent a few minutes this afternoon trying to figure out how hard it would be. Turns out, it's much simpler than I expected if you do the same trick that Xen does. Instead of mucking with hooking cpu_exec, Xen simply starts the CPUs in the halted state such that TCG simply never runs. The following patch is all we really need. test_init() just needs to register the appropriate file descriptor callbacks and then in the data path dispatch PIO/MMIO. It would also need to override cpu_interrupt_handler to intercept interrupt operations. diff --git a/hw/pc.c b/hw/pc.c index eb4c2d8..f3fd32d 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -923,12 +923,18 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level) } } +extern int test_allowed; + static void pc_cpu_reset(void *opaque) { CPUState *env = opaque; cpu_reset(env); - env->halted = !cpu_is_bsp(env); + if (test_allowed) { + env->halted = 1; + } else { + env->halted = !cpu_is_bsp(env); + } } static CPUState *pc_new_cpu(const char *cpu_model) diff --git a/vl.c b/vl.c index 1ddb17b..adc626a 100644 --- a/vl.c +++ b/vl.c @@ -1988,6 +1988,19 @@ static int tcg_init(void) return 0; } +static int test_init(void) +{ + printf("Hello World\n"); + return 0; +} + +static int test_available(void) +{ + return 1; +} + +int test_allowed = 1; + static struct { const char *opt_name; const char *name; @@ -1998,6 +2011,7 @@ static struct { { "tcg", "tcg", tcg_available, tcg_init, &tcg_allowed }, { "xen", "Xen", xen_available, xen_init, &xen_allowed }, { "kvm", "KVM", kvm_available, kvm_init, &kvm_allowed }, + { "test", "Test", test_available, test_init, &test_allowed }, }; static int configure_accelerator(void) > Kevin > ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-27 17:58 ` Anthony Liguori @ 2011-10-27 21:22 ` Michael Roth 0 siblings, 0 replies; 24+ messages in thread From: Michael Roth @ 2011-10-27 21:22 UTC (permalink / raw) To: Anthony Liguori, Kevin Wolf Cc: Lucas Meneghel Rodrigues, qemu-devel, Markus Armbruster, Avi Kivity, Cleber Rosa, Gerd Hoffmann On Thu, 27 Oct 2011 12:58:32 -0500, Anthony Liguori <anthony@codemonkey.ws> wrote: > On 10/27/2011 03:20 AM, Kevin Wolf wrote: > > Am 26.10.2011 22:49, schrieb Anthony Liguori: > >> On 10/25/2011 10:22 AM, Kevin Wolf wrote: > >>> Am 25.10.2011 17:03, schrieb Eduardo Habkost: > >>> I think qemu-iotests could be considered an instance of B) > >>> > >>>> C) Functional tests that just need to run a small binary with no OS > >>>> installed in the guest, but running a fully-feature qemu process. > >>>> - The tests in the 'tests' directory do this, right? kvm-unittests > >>>> does this, right? > >>> > >>> Not sure what test/ does, but for kvm-unittests yes. And this is also > >>> what I was talking about. > >> > >> Thinking more about this... > >> > >> We could add a new '-x-test-server CHR' option. When this option is added, it > >> would do the following: > >> > >> 1) Open CHR character device > >> 2) Use /dev/shm for guest memory > >> 3) Listen for connections on CHR > >> 4) When something connects to CHR > >> a) reset device model > >> b) send /dev/shm fd over CHR > >> c) register CPU physical memory client > >> 1. upon CPU physical memory changes, send the change info over CHR > >> d) instead of doing [kvm_]cpu_exec(), block reading on CHR > >> > >> So when you launch qemu with -x-test-server, it'll sit there doing nothing > >> terribly useful. But this lets you write a program that connects to CHR, and > >> then by mapping {out,in}[bwl] to RPCs over the connection, and accessing RAM via > >> mmap()'ing the passed fd using the client mapping table, you can essentially > >> write kvm-unittest style tests while still having full access to libc. > > > > IRQs need to go through the connection as well. > > Yes, forgot to mention that. > > > > > Oh, and you would finally have a C user for libqmp. The test cases > > definitely need to be able to access the monitor. For example I would > > really love to have test cases for the I/O error paths that stop the VM > > (or actually it's the resume that must be tested). > > Yeah, tunnelling a monitor session sounds like a really good idea. > > >> And since each test program can reset QEMU after running, you could very nicely > >> tie into something like gtest as a unit test framework. I think it's pretty > >> appealing from a debugability perspective too. > >> > >> It also means that it's possible to have 100% C test cases such that you could > >> still build something like ppc64-softmmu and run it against the written test > >> cases without having to really understand ppc64 assembly or have a ppc64 build > >> environment (to generate native binaries to run under ppc64 TCG). > >> > >> I think this could work out fairly well as a unit test framework. > > > > Sounds great, where are the patches? ;-) > > Heh, need to find a volunteer although I spent a few minutes this afternoon > trying to figure out how hard it would be. Turns out, it's much simpler than I > expected if you do the same trick that Xen does. Instead of mucking with > hooking cpu_exec, Xen simply starts the CPUs in the halted state such that TCG > simply never runs. The following patch is all we really need. > > test_init() just needs to register the appropriate file descriptor callbacks and > then in the data path dispatch PIO/MMIO. It would also need to override > cpu_interrupt_handler to intercept interrupt operations. > > diff --git a/hw/pc.c b/hw/pc.c > index eb4c2d8..f3fd32d 100644 > --- a/hw/pc.c > +++ b/hw/pc.c > @@ -923,12 +923,18 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level) > } > } > > +extern int test_allowed; > + > static void pc_cpu_reset(void *opaque) > { > CPUState *env = opaque; > > cpu_reset(env); > - env->halted = !cpu_is_bsp(env); > + if (test_allowed) { > + env->halted = 1; > + } else { > + env->halted = !cpu_is_bsp(env); > + } > } > > static CPUState *pc_new_cpu(const char *cpu_model) > diff --git a/vl.c b/vl.c > index 1ddb17b..adc626a 100644 > --- a/vl.c > +++ b/vl.c > @@ -1988,6 +1988,19 @@ static int tcg_init(void) > return 0; > } > > +static int test_init(void) > +{ > + printf("Hello World\n"); > + return 0; > +} > + > +static int test_available(void) > +{ > + return 1; > +} > + > +int test_allowed = 1; > + > static struct { > const char *opt_name; > const char *name; > @@ -1998,6 +2011,7 @@ static struct { > { "tcg", "tcg", tcg_available, tcg_init, &tcg_allowed }, > { "xen", "Xen", xen_available, xen_init, &xen_allowed }, > { "kvm", "KVM", kvm_available, kvm_init, &kvm_allowed }, > + { "test", "Test", test_available, test_init, &test_allowed }, > }; > > static int configure_accelerator(void) > vl.c seems to have changed a bit since then, but I think this was the same basic approach we took with QTest: http://repo.or.cz/w/qemu/mdroth.git/commitdiff/2e55d8a892297512f6c7883aff2c4b082bd6a3d5?hp=07d8a50cb0e096c5cdc5a81b8030e40833664dd3 http://wiki.qemu.org/Features/QTest There was even a brief period where we were using gtest, but GLib hadn't been merged yet so we pulled it "temporarially" before going off and working on other stuff. > > > Kevin > > > > -- Sincerely, Mike Roth IBM Linux Technology Center ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 15:03 ` Eduardo Habkost 2011-10-25 15:22 ` Kevin Wolf @ 2011-10-25 15:54 ` Gerd Hoffmann 2011-10-25 16:30 ` Lucas Meneghel Rodrigues 2 siblings, 0 replies; 24+ messages in thread From: Gerd Hoffmann @ 2011-10-25 15:54 UTC (permalink / raw) To: Kevin Wolf, Avi Kivity, Markus Armbruster, qemu-devel, Lucas Meneghel Rodrigues, Cleber Rosa Hi, > Just trying to enumerate the kind of tests somebody may need: > > A) Simple unit tests for internal qemu C functions > - 'make check' can run them, using either libcheck or gtest. Yes, those tests should work and succeed anywhere where qemu compiles. > B) Functional tests that tests actual virtualization/emulation, but only > of some specific subsystems, not using a fully-featured qemu process. > - We don't have anything like that, today, right? I am not sure we > need it. qemu-kvm used to have such tests. There is no point in not using qemu here though, thats why they where transformed into (C). > C) Functional tests that just need to run a small binary with no OS > installed in the guest, but running a fully-feature qemu process. > - The tests in the 'tests' directory do this, right? kvm-unittests > does this, right? kvm-unittests yes. tests/ seems to be a somewhat bitrotted random collection of tests. > D) Functional tests that need a minimal OS installed, with, e.g., at > least a Linux kernel and a shell. > - This is what Gerd's patch below does, right? Also, KVM-Autotest can > be used for this. What the patch tries is to use the bits which are on the host anyway to run guest tests (obviously linux-on-linux only), so we can do parts of (E) with less configuration, less internet bandwith (no iso downloads), less disk space requirements (no iso + disk storage) and faster (no guest installs needed). Didn't try yet to mount the host filesystem using 9p yet, with that we could possibly take the approach quite far as we suddenly don't have just a "minimal os" any more but alot more. Using the host bits is at the same time a possible trouble spot. There are lots of different distros with more or less subtile differences out there. Each machine has a different set of packages installed. So having this "just work" everythere probably isn't that easy ... > E) Functional tests that need a full OS installed and configured. > - Today we use KVM-Autotest for this. cheers, Gerd ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 15:03 ` Eduardo Habkost 2011-10-25 15:22 ` Kevin Wolf 2011-10-25 15:54 ` Gerd Hoffmann @ 2011-10-25 16:30 ` Lucas Meneghel Rodrigues 2 siblings, 0 replies; 24+ messages in thread From: Lucas Meneghel Rodrigues @ 2011-10-25 16:30 UTC (permalink / raw) To: Gerd Hoffmann, Kevin Wolf, Avi Kivity, Markus Armbruster, qemu-devel, Cleber Rosa On 10/25/2011 01:03 PM, Eduardo Habkost wrote: > On Tue, Oct 25, 2011 at 03:27:35PM +0200, Gerd Hoffmann wrote: > [...] >>> A while ago I played with some simple IDE tests. It basically was a >>> small x86 kernel with an empty image that sends IDE commands and prints >>> some results, and a script that invokes the guest and checks whether the >>> test has passed or failed. >> >> That reminds me that I've started toying with running tests inside a >> guest too. Stopped working on it a while back due to other priorities. >> Attached what I have so far. >> >>> So at first I started with my own multiboot kernel and copied over some >>> parts of kvm-unittest's libc. Clearly not the best idea once it's more >>> than a couple of lines, so at some point I took the code and integrated >>> with my real kvm-unittests repository. >>> >>> Now I don't have to duplicate code any more, but at the same time >>> there's no chance that a 'make check' in an upstream qemu tree could run >>> this. Tests for other devices will have exactly the same problem. >>> >>> Any suggestions on how to go forward with this kind of tests? Should >>> this go into qemu or into kvm-unittests? Or should kvm-unittests be >>> merged into the qemu tree? Or is the approach completely wrong? >> >> I think we should have some framework to run tests inside the guest in >> the qemu source tree. I'm not sure kvm-unittests is the right tool for >> the job though. It is quite low-level and mainly targets the kvm bits >> inside the linux kernel. Testing -- for example -- usb device emulation >> would pretty much require writing a usb stack for kvm-unitests ... > > We have a framework to run tests inside a fully-installed guest, that's > KVM-Autotest. But maybe it's too much for the kind of tests you need, I > don't know. There are different "levels" of testing, with different > reequirements, and we need to have good tools for all levels. > > Just trying to enumerate the kind of tests somebody may need: > > A) Simple unit tests for internal qemu C functions > - 'make check' can run them, using either libcheck or gtest. > B) Functional tests that tests actual virtualization/emulation, but only > of some specific subsystems, not using a fully-featured qemu process. > - We don't have anything like that, today, right? I am not sure we > need it. > C) Functional tests that just need to run a small binary with no OS > installed in the guest, but running a fully-feature qemu process. > - The tests in the 'tests' directory do this, right? kvm-unittests > does this, right? > D) Functional tests that need a minimal OS installed, with, e.g., at > least a Linux kernel and a shell. > - This is what Gerd's patch below does, right? Also, KVM-Autotest can > be used for this. Yes, it can. There's instrumentation to get ssh sessions, serial sessions, execute monitor commands... Only need to specify the path to a pre-installed minimal guest, skip install tests and have the bare minimal subset of tests needed. If only a small subset of tests are run with a minimal image, a kvm autotest job is quick to run and suitable for development use. I've looked at Gerd's initial patch to add an initramfs for such lightweight testing, and the init ramdisk generation could be implemented in autotest. However... Problem today is that KVM autotest is considered to be hard to approach and integrate with qemu developer's workflow. I won't argue it's not, so I'm admitting failure here. We've been juggling to keep it stable for the regular, full blown QE case, with expanding test and targets coverage (see libvirt test), and yet trying to make it better documented and sufficiently easy to approach to do simpler ad hoc testing. I wish it was easier to accomplish the later, though. So, an in-tree solution for D) like the one Gerd started to work on seems better, as it won't generate frustration from folks having to learn kvm autotest, which will ultimately lead to them ignoring the effort. > E) Functional tests that need a full OS installed and configured. > - Today we use KVM-Autotest for this. > > > Does the above model look correct/complete, or is there some case I > missed? > > >> >> cheers, >> Gerd > >> From 096f68ea08c3c4baf1bbdc549b257a67ecc87e25 Mon Sep 17 00:00:00 2001 >> From: Gerd Hoffmann<kraxel@redhat.com> >> Date: Tue, 13 Sep 2011 17:38:37 +0200 >> Subject: [PATCH] initramfs test framework >> >> Signed-off-by: Gerd Hoffmann<kraxel@redhat.com> >> --- >> initramfs/.gitignore | 3 + >> initramfs/10-qemu-udev.rules | 5 + >> initramfs/Makefile | 36 +++++++ >> initramfs/README | 44 ++++++++ >> initramfs/init.c | 225 ++++++++++++++++++++++++++++++++++++++++++ >> initramfs/initramfs-boot | 32 ++++++ >> initramfs/initramfs-create | 111 +++++++++++++++++++++ >> initramfs/test-ehci | 3 + >> initramfs/test-ehci.good | 8 ++ >> initramfs/test-hello.c | 7 ++ >> initramfs/test-hello.good | 1 + >> initramfs/test-uhci | 3 + >> initramfs/test-uhci.good | 3 + >> 13 files changed, 481 insertions(+), 0 deletions(-) >> create mode 100644 initramfs/.gitignore >> create mode 100644 initramfs/10-qemu-udev.rules >> create mode 100644 initramfs/Makefile >> create mode 100644 initramfs/README >> create mode 100644 initramfs/init.c >> create mode 100755 initramfs/initramfs-boot >> create mode 100755 initramfs/initramfs-create >> create mode 100755 initramfs/test-ehci >> create mode 100644 initramfs/test-ehci.good >> create mode 100644 initramfs/test-hello.c >> create mode 100644 initramfs/test-hello.good >> create mode 100755 initramfs/test-uhci >> create mode 100644 initramfs/test-uhci.good >> > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-10-25 12:21 ` Kevin Wolf 2011-10-25 13:27 ` Gerd Hoffmann @ 2011-10-25 15:03 ` Anthony Liguori 1 sibling, 0 replies; 24+ messages in thread From: Anthony Liguori @ 2011-10-25 15:03 UTC (permalink / raw) To: Kevin Wolf; +Cc: Markus Armbruster, Gerd Hoffmann, qemu-devel, Avi Kivity On 10/25/2011 07:21 AM, Kevin Wolf wrote: > Am 24.10.2011 20:57, schrieb Anthony Liguori: >> On 10/24/2011 01:43 PM, Eduardo Habkost wrote: >>> On Mon, Sep 05, 2011 at 09:55:20AM +0200, Markus Armbruster wrote: >>>> Gerd Hoffmann<kraxel@redhat.com> writes: >>>> >>>>> Hi, >>>>> >>>>> This patch series intends to make unit testing easier. It adds a new >>>>> "make check" target which can be used to run all unit tests which are >>>>> currently in the tree. It also enables the unit tests by default, so >>>>> you don't have to re-run configure with a special switch. >>>> >>>> Reviewed-by: Markus Armbruster<armbru@redhat.com> >>>> >>>> One test fails, but Luiz has a fix in his tree. >>> >>> Reviewed-by: Eduardo Habkost<ehabkost@redhat.com> >>> >>> Now all tests are passing. Why this was not applied yet? >> >> I was hoping for more, but maybe we just need to start here and grow >> organically, I'll queue it again. > > A while ago I played with some simple IDE tests. It basically was a > small x86 kernel with an empty image that sends IDE commands and prints > some results, and a script that invokes the guest and checks whether the > test has passed or failed. > > So at first I started with my own multiboot kernel and copied over some > parts of kvm-unittest's libc. Clearly not the best idea once it's more > than a couple of lines, so at some point I took the code and integrated > with my real kvm-unittests repository. I did something very similar. I just added PCI support to kvm-unittests such that I could write a small virtio driver. I actually do think tests like this could live in a make check framework in QEMU. Regards, Anthony Liguori > > Now I don't have to duplicate code any more, but at the same time > there's no chance that a 'make check' in an upstream qemu tree could run > this. Tests for other devices will have exactly the same problem. > > Any suggestions on how to go forward with this kind of tests? Should > this go into qemu or into kvm-unittests? Or should kvm-unittests be > merged into the qemu tree? Or is the approach completely wrong? > > Kevin > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [PATCH 0/4] add "make check" 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann ` (4 preceding siblings ...) 2011-09-05 7:55 ` [Qemu-devel] [PATCH 0/4] add "make check" Markus Armbruster @ 2011-11-01 18:03 ` Anthony Liguori 5 siblings, 0 replies; 24+ messages in thread From: Anthony Liguori @ 2011-11-01 18:03 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: qemu-devel On 09/01/2011 10:42 AM, Gerd Hoffmann wrote: > Hi, > > This patch series intends to make unit testing easier. It adds a new > "make check" target which can be used to run all unit tests which are > currently in the tree. It also enables the unit tests by default, so > you don't have to re-run configure with a special switch. Applied all. Thanks. Regards, Anthony Liguori > > cheers, > Gerd > > Gerd Hoffmann (4): > Probe for libcheck by default. > move checks to separate variable > add "make check" target > add test-coroutine to checks > > Makefile | 10 ++++++++-- > configure | 7 ++++--- > 2 files changed, 12 insertions(+), 5 deletions(-) > > > ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2011-11-01 18:03 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-09-01 15:42 [Qemu-devel] [PATCH 0/4] add "make check" Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 1/4] Probe for libcheck by default Gerd Hoffmann 2011-09-01 19:37 ` Anthony Liguori 2011-09-02 7:42 ` Gerd Hoffmann 2011-09-05 7:39 ` Markus Armbruster 2011-09-01 15:42 ` [Qemu-devel] [PATCH 2/4] move checks to separate variable Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 3/4] add "make check" target Gerd Hoffmann 2011-09-01 15:42 ` [Qemu-devel] [PATCH 4/4] add test-coroutine to checks Gerd Hoffmann 2011-09-05 7:55 ` [Qemu-devel] [PATCH 0/4] add "make check" Markus Armbruster 2011-10-24 18:43 ` Eduardo Habkost 2011-10-24 18:57 ` Anthony Liguori 2011-10-25 12:21 ` Kevin Wolf 2011-10-25 13:27 ` Gerd Hoffmann 2011-10-25 14:16 ` Kevin Wolf 2011-10-25 15:03 ` Eduardo Habkost 2011-10-25 15:22 ` Kevin Wolf 2011-10-26 20:49 ` Anthony Liguori 2011-10-27 8:20 ` Kevin Wolf 2011-10-27 17:58 ` Anthony Liguori 2011-10-27 21:22 ` Michael Roth 2011-10-25 15:54 ` Gerd Hoffmann 2011-10-25 16:30 ` Lucas Meneghel Rodrigues 2011-10-25 15:03 ` Anthony Liguori 2011-11-01 18:03 ` Anthony Liguori
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.