From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51133) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VAOn9-0004bt-Pq for qemu-devel@nongnu.org; Fri, 16 Aug 2013 14:30:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VAOn6-0008Hh-Gy for qemu-devel@nongnu.org; Fri, 16 Aug 2013 14:30:31 -0400 Received: from mail.avalus.com ([2001:41c8:10:1dd::10]:59151) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VAOn6-0008HQ-4I for qemu-devel@nongnu.org; Fri, 16 Aug 2013 14:30:28 -0400 From: Alex Bligh Date: Fri, 16 Aug 2013 19:29:41 +0100 Message-Id: <1376677812-9448-1-git-send-email-alex@alex.org.uk> Subject: [Qemu-devel] [PATCHv12 00/31] aio / timers: Add AioContext timers and use ppoll List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Kevin Wolf , Anthony Liguori , Alex Bligh , Jan Kiszka , liu ping fan , Stefan Hajnoczi , Paolo Bonzini , MORITA Kazutaka , rth@twiddle.net [ This patch set is available from git at: https://github.com/abligh/qemu/tree/aio-timers12 As autogenerated patch 30 of the series is too large for the mailing list. ] This patch series adds support for timers attached to an AioContext clock which get called within aio_poll. In doing so it removes alarm timers and moves to use ppoll where possible. This patch set 'sort of' passes make check (see below for caveat) including a new test harness for the aio timers, but has not been tested much beyond that. In particular, the win32 changes have not even been compile tested. Equally, alterations to use_icount are untested. Caveat: I have had to alter tests/test-aio.c so the following error no longer occurs. ERROR:tests/test-aio.c:346:test_wait_event_notifier_noflush: assertion failed: (aio_poll(ctx, false)) As gar as I can tell, this check was incorrect, in that it checking aio_poll makes progress when in fact it should not make progress. I fixed an issue where aio_poll was (as far as I can tell) wrongly returning true on a timeout, and that generated this error. Note also the comment on patch 19 in relation to a possible bug in cpus.c. The penultimate patch is patch which is created in an automated manner using scripts/switch-timer-api, added in this patch set. It violates some coding standards (e.g. line length >= 80 characters), but this is preferable in terms of giving a provably correct conversion. This patch is too large for the mailing list, so EITHER: get it from git at the URL at the top of this message. OR: Do the following: 1. Apply patches 0000-0029 inclusive 2. Run scripts/switch-timer-api 3. git commit -a (+ suitable commit message) 4. Apply patch 0031 If there is demand I can split it one commit per file. This patch set has been compile tested & make check tested on a 'christmas-tree' configuration, meaning a configuration with every --enable-* value tested that can be easily configured on Ubuntu Precise, after application of each patch. Changes since v11: * Rebase to stefanha's block-next ab97be1 * Fix up warning about excessive iterations in main-loop.c Changes since v10: * Rebase to maser 1ee2dae (1.6.0 release) * Attempt to fix double 'typedef struct' issue by only having typedef struct in typedefs.h (cannot test, my compiler doesn't complain) Changes since v9: * Rebase to master 2e985fe * Wrap QEMUTimerListGroup in a struct as we're keeping it for the time being Changes since v8: * PR_SET_TIMERSLACK commit should have relevant configure patch within * Delete timerlist_set_notify_cb, put into timerlist_new * Add missing QLIST_INIT of clock->timerlists * Fix documentation for timerlist_get_clock * Rename qemu_timer_xxx to timer_xxx * Remove unintentional change to pc-bios/slof.bin * Introduce timer_init and aio_timer_init Changes since v7: * Rebase to master 6fdf98f281f85ae6e2883bed2f691bcfe33b1f9f * Add qemu_clock_get_ms and qemu_clock_get_ms * Rename qemu_get_clock to qemu_clock_ptr * Reorder qemu-timer.h to utilise the legacy API * Hide qemu_clock_new & qemu_clock_free * Rename default_timerlist to main_loop_timerlist * Remove main_loop_timerlist once main_loop_tlg is in * Add script to convert to new API * Make rtc_clock use new API * Convert tests/test-aio to use new API * Run script on entire source code * Remove legacy API functions Changes since v6: * Fix build failure in vnc-auth-sasl.c * Split first patch into 3 * Add assert on timerlist_free * Fix ==/= error on qemu_clock_use_for_deadline * Remove unnecessary cast in aio_timerlist_notify * Fix bad deadline comparison in aio_ctx_check * Add assert to timerlist_new_from_clock to check init_clocks * Use timer_list not tl * Change default_timerlistgroup to main_loop_timerlistgroup * Add comment on commit for qemu_clock_use_for_deadline * Fixed various include file issues * Convert *_has_timers and *_has_expired to return bool * Make loop variable consistent when looping through clock types * Add documentation to existing qemu_timer calls * Remove qemu_clock_deadline and move to qemu_clock_deadline_ns Changes since v5: * Rebase onto master (b9ac5d9) * Fix spacing in typedef QEMUTimerList * Rename 'QEMUClocks' extern to 'qemu_clocks' Changes since v4: * Rename qemu_timerlist_ functions to timer_list (per Paolo Bonzini) * Rename qemu_timer_.*timerlist.* to timer_ (per Paolo Bonzini) * Use enum for QEMUClockType * Put clocks into an array; remove global variables * Introduce QEMUTimerListGroup - a timeliest of each type * Add a QEMUTimerListGroup to AioContext * Use a callback on timer modification, rather than binding in AioContext into the timeliest * Make cpus.c iterate over all timerlists when it does a notify * Make cpus.c icount timeout use soonest timeout across all timerlists Changes since v3: * Split up QEMUClock and QEMUClock list * Improve commenting * Fix comment in vl.c * Change test/test-aio.c to reflect correct behaviour in aio_poll. Changes since v2: * Reordered to remove alarm timers last * Added prctl(PR_SET_TIMERSLACK, 1, ...) * Renamed qemu_g_poll_ns to qemu_poll_ns * Moved declaration of above & drop glib types * Do not use a global list of qemu clocks * Add AioContext * to QEMUClock * Split up conversion to use ppoll and timers * Indentation fix * Fix aio_win32.c aio_poll to return progress * aio_notify / qemu_notify when timers are modified * change comment in deprecation of clock options Alex Bligh (31): aio / timers: Rename qemu_timer_* functions aio / timers: Rename qemu_new_clock and expose clock types aio / timers: add qemu-timer.c utility functions aio / timers: Consistent treatment of disabled clocks for deadlines aio / timers: add ppoll support with qemu_poll_ns aio / timers: Add prctl(PR_SET_TIMERSLACK, 1, ...) to reduce timer slack aio / timers: Make qemu_run_timers and qemu_run_all_timers return progress aio / timers: Split QEMUClock into QEMUClock and QEMUTimerList aio / timers: Untangle include files aio / timers: Add QEMUTimerListGroup and helper functions aio / timers: Add QEMUTimerListGroup to AioContext aio / timers: Add a notify callback to QEMUTimerList aio / timers: aio_ctx_prepare sets timeout from AioContext timers aio / timers: Add aio_timer_init & aio_timer_new wrappers aio / timers: Convert aio_poll to use AioContext timers' deadline aio / timers: Convert mainloop to use timeout aio / timers: On timer modification, qemu_notify or aio_notify aio / timers: Introduce new API timer_new and friends aio / timers: Use all timerlists in icount warp calculations aio / timers: Add documentation and new format calls aio / timers: Remove alarm timers aio / timers: Remove legacy qemu_clock_deadline & qemu_timerlist_deadline aio / timers: Add qemu_clock_get_ms and qemu_clock_get_ms aio / timers: Rearrange timer.h & make legacy functions call non-legacy aio / timers: Remove main_loop_timerlist aio / timers: Convert rtc_clock to be a QEMUClockType aio / timers: convert block_job_sleep_ns and co_sleep_ns to new API aio / timers: Add test harness for AioContext timers aio / timers: Add scripts/switch-timer-api aio / timers: Switch entire codebase to the new timer API aio / timers: Remove legacy interface aio-posix.c | 20 +- aio-win32.c | 22 +- arch_init.c | 12 +- async.c | 20 +- audio/audio.c | 6 +- audio/noaudio.c | 4 +- audio/spiceaudio.c | 4 +- audio/wavaudio.c | 2 +- backends/baum.c | 16 +- block.c | 14 +- block/backup.c | 4 +- block/commit.c | 2 +- block/iscsi.c | 14 +- block/mirror.c | 10 +- block/qed.c | 10 +- block/stream.c | 2 +- blockdev.c | 2 +- blockjob.c | 4 +- configure | 37 ++ cpus.c | 138 ++++--- dma-helpers.c | 1 + hmp.c | 8 +- hw/acpi/core.c | 8 +- hw/acpi/piix4.c | 2 +- hw/alpha/typhoon.c | 2 +- hw/arm/omap1.c | 52 +-- hw/arm/pxa2xx.c | 61 ++-- hw/arm/spitz.c | 6 +- hw/arm/stellaris.c | 10 +- hw/arm/strongarm.c | 34 +- hw/audio/adlib.c | 2 +- hw/audio/intel-hda.c | 4 +- hw/audio/sb16.c | 6 +- hw/block/fdc.c | 6 +- hw/block/nvme.c | 20 +- hw/block/pflash_cfi01.c | 2 +- hw/block/pflash_cfi02.c | 10 +- hw/bt/hci-csr.c | 4 +- hw/bt/hci.c | 38 +- hw/bt/l2cap.c | 8 +- hw/char/cadence_uart.c | 12 +- hw/char/serial.c | 22 +- hw/char/virtio-serial-bus.c | 10 +- hw/core/ptimer.c | 18 +- hw/display/qxl-logger.c | 2 +- hw/display/qxl.c | 2 +- hw/display/vga.c | 6 +- hw/dma/pl330.c | 6 +- hw/dma/rc4030.c | 4 +- hw/dma/soc_dma.c | 8 +- hw/dma/xilinx_axidma.c | 1 + hw/i386/kvm/apic.c | 2 +- hw/i386/kvm/i8254.c | 6 +- hw/i386/xen_domainbuild.c | 6 +- hw/ide/core.c | 6 +- hw/input/hid.c | 10 +- hw/input/lm832x.c | 8 +- hw/input/tsc2005.c | 16 +- hw/input/tsc210x.c | 32 +- hw/intc/apic.c | 16 +- hw/intc/apic_common.c | 2 +- hw/intc/armv7m_nvic.c | 16 +- hw/intc/i8259.c | 4 +- hw/mips/cputimer.c | 16 +- hw/misc/arm_sysctl.c | 2 +- hw/misc/macio/cuda.c | 34 +- hw/misc/macio/macio.c | 4 +- hw/misc/vfio.c | 14 +- hw/net/dp8393x.c | 20 +- hw/net/e1000.c | 12 +- hw/net/lan9118.c | 4 +- hw/net/pcnet-pci.c | 4 +- hw/net/pcnet.c | 10 +- hw/net/rtl8139.c | 28 +- hw/net/virtio-net.c | 20 +- hw/openrisc/cputimer.c | 10 +- hw/ppc/ppc.c | 64 ++-- hw/ppc/ppc405_uc.c | 8 +- hw/ppc/ppc_booke.c | 10 +- hw/ppc/spapr.c | 8 +- hw/sd/sdhci.c | 28 +- hw/sparc64/sun4u.c | 24 +- hw/timer/arm_mptimer.c | 12 +- hw/timer/arm_timer.c | 1 + hw/timer/cadence_ttc.c | 6 +- hw/timer/etraxfs_timer.c | 2 +- hw/timer/exynos4210_mct.c | 3 +- hw/timer/exynos4210_pwm.c | 1 + hw/timer/grlib_gptimer.c | 2 + hw/timer/hpet.c | 20 +- hw/timer/i8254.c | 26 +- hw/timer/i8254_common.c | 4 +- hw/timer/imx_epit.c | 1 + hw/timer/imx_gpt.c | 1 + hw/timer/lm32_timer.c | 1 + hw/timer/m48t59.c | 18 +- hw/timer/mc146818rtc.c | 50 +-- hw/timer/omap_gptimer.c | 24 +- hw/timer/omap_synctimer.c | 2 +- hw/timer/pl031.c | 19 +- hw/timer/puv3_ost.c | 1 + hw/timer/pxa2xx_timer.c | 34 +- hw/timer/sh_timer.c | 1 + hw/timer/slavio_timer.c | 1 + hw/timer/tusb6010.c | 12 +- hw/timer/twl92230.c | 14 +- hw/timer/xilinx_timer.c | 1 + hw/tpm/tpm_tis.c | 1 + hw/usb/hcd-ehci.c | 10 +- hw/usb/hcd-musb.c | 6 +- hw/usb/hcd-ohci.c | 12 +- hw/usb/hcd-uhci.c | 15 +- hw/usb/hcd-xhci.c | 26 +- hw/usb/host-libusb.c | 6 +- hw/usb/host-linux.c | 6 +- hw/usb/redirect.c | 16 +- hw/virtio/virtio-balloon.c | 8 +- hw/virtio/virtio-rng.c | 14 +- hw/watchdog/wdt_i6300esb.c | 6 +- hw/watchdog/wdt_ib700.c | 10 +- hw/xtensa/pic_cpu.c | 10 +- include/block/aio.h | 52 ++- include/block/block_int.h | 1 + include/block/blockjob.h | 2 +- include/block/coroutine.h | 3 +- include/hw/acpi/acpi.h | 2 +- include/qemu/ratelimit.h | 2 +- include/qemu/timer.h | 676 ++++++++++++++++++++++++++++++++--- include/qemu/typedefs.h | 3 + include/sysemu/sysemu.h | 2 +- main-loop.c | 51 ++- migration-exec.c | 1 + migration-fd.c | 1 + migration-tcp.c | 1 + migration-unix.c | 1 + migration.c | 17 +- monitor.c | 8 +- nbd.c | 1 + net/dump.c | 2 +- net/net.c | 1 + net/socket.c | 1 + qemu-char.c | 2 +- qemu-coroutine-io.c | 1 + qemu-coroutine-sleep.c | 10 +- qemu-io-cmds.c | 1 + qemu-nbd.c | 1 + qemu-timer.c | 834 +++++++++++++++---------------------------- qtest.c | 10 +- savevm.c | 24 +- scripts/switch-timer-api | 178 +++++++++ slirp/if.c | 2 +- slirp/misc.c | 1 + slirp/slirp.c | 4 +- stubs/clock-warp.c | 2 +- target-alpha/sys_helper.c | 12 +- target-ppc/kvm.c | 8 +- target-ppc/kvm_ppc.c | 6 +- target-s390x/cpu.c | 4 +- target-s390x/misc_helper.c | 6 +- target-xtensa/op_helper.c | 2 +- tests/libqtest.h | 24 +- tests/test-aio.c | 143 +++++++- tests/test-thread-pool.c | 3 + thread-pool.c | 1 + ui/console.c | 30 +- ui/input.c | 6 +- ui/spice-core.c | 10 +- ui/vnc-auth-sasl.h | 1 + ui/vnc-auth-vencrypt.c | 2 +- ui/vnc-ws.c | 1 + vl.c | 14 +- xen-all.c | 12 +- 172 files changed, 2289 insertions(+), 1459 deletions(-) create mode 100755 scripts/switch-timer-api -- 1.7.9.5 *** BLURB STOP *** Alex Bligh (31): aio / timers: Rename qemu_timer_* functions aio / timers: Rename qemu_new_clock and expose clock types aio / timers: add qemu-timer.c utility functions aio / timers: Consistent treatment of disabled clocks for deadlines aio / timers: add ppoll support with qemu_poll_ns aio / timers: Add prctl(PR_SET_TIMERSLACK, 1, ...) to reduce timer slack aio / timers: Make qemu_run_timers and qemu_run_all_timers return progress aio / timers: Split QEMUClock into QEMUClock and QEMUTimerList aio / timers: Untangle include files aio / timers: Add QEMUTimerListGroup and helper functions aio / timers: Add QEMUTimerListGroup to AioContext aio / timers: Add a notify callback to QEMUTimerList aio / timers: aio_ctx_prepare sets timeout from AioContext timers aio / timers: Add aio_timer_init & aio_timer_new wrappers aio / timers: Convert aio_poll to use AioContext timers' deadline aio / timers: Convert mainloop to use timeout aio / timers: On timer modification, qemu_notify or aio_notify aio / timers: Introduce new API timer_new and friends aio / timers: Use all timerlists in icount warp calculations aio / timers: Add documentation and new format calls aio / timers: Remove alarm timers aio / timers: Remove legacy qemu_clock_deadline & qemu_timerlist_deadline aio / timers: Add qemu_clock_get_ms and qemu_clock_get_ms aio / timers: Rearrange timer.h & make legacy functions call non-legacy aio / timers: Remove main_loop_timerlist aio / timers: Convert rtc_clock to be a QEMUClockType aio / timers: convert block_job_sleep_ns and co_sleep_ns to new API aio / timers: Add test harness for AioContext timers aio / timers: Add scripts/switch-timer-api aio / timers: Switch entire codebase to the new timer API aio / timers: Remove legacy interface aio-posix.c | 18 +- aio-win32.c | 20 +- arch_init.c | 12 +- async.c | 20 +- audio/audio.c | 6 +- audio/noaudio.c | 4 +- audio/spiceaudio.c | 4 +- audio/wavaudio.c | 2 +- backends/baum.c | 16 +- block.c | 14 +- block/backup.c | 4 +- block/commit.c | 2 +- block/iscsi.c | 14 +- block/mirror.c | 10 +- block/qed.c | 10 +- block/stream.c | 2 +- blockdev.c | 2 +- blockjob.c | 4 +- configure | 37 ++ cpus.c | 138 ++++--- dma-helpers.c | 1 + hmp.c | 8 +- hw/acpi/core.c | 8 +- hw/acpi/piix4.c | 2 +- hw/alpha/typhoon.c | 2 +- hw/arm/omap1.c | 52 +-- hw/arm/pxa2xx.c | 61 ++-- hw/arm/spitz.c | 6 +- hw/arm/stellaris.c | 10 +- hw/arm/strongarm.c | 34 +- hw/audio/adlib.c | 2 +- hw/audio/intel-hda.c | 4 +- hw/audio/sb16.c | 6 +- hw/block/fdc.c | 6 +- hw/block/nvme.c | 20 +- hw/block/pflash_cfi01.c | 2 +- hw/block/pflash_cfi02.c | 10 +- hw/bt/hci-csr.c | 4 +- hw/bt/hci.c | 38 +- hw/bt/l2cap.c | 8 +- hw/char/cadence_uart.c | 12 +- hw/char/serial.c | 22 +- hw/char/virtio-serial-bus.c | 10 +- hw/core/ptimer.c | 18 +- hw/display/qxl-logger.c | 2 +- hw/display/qxl.c | 2 +- hw/display/vga.c | 6 +- hw/dma/pl330.c | 6 +- hw/dma/rc4030.c | 4 +- hw/dma/soc_dma.c | 8 +- hw/dma/xilinx_axidma.c | 1 + hw/i386/kvm/apic.c | 2 +- hw/i386/kvm/i8254.c | 6 +- hw/i386/xen_domainbuild.c | 6 +- hw/ide/core.c | 6 +- hw/input/hid.c | 10 +- hw/input/lm832x.c | 8 +- hw/input/tsc2005.c | 16 +- hw/input/tsc210x.c | 32 +- hw/intc/apic.c | 16 +- hw/intc/apic_common.c | 2 +- hw/intc/armv7m_nvic.c | 16 +- hw/intc/i8259.c | 4 +- hw/mips/cputimer.c | 16 +- hw/misc/arm_sysctl.c | 2 +- hw/misc/macio/cuda.c | 34 +- hw/misc/macio/macio.c | 4 +- hw/misc/vfio.c | 14 +- hw/net/dp8393x.c | 20 +- hw/net/e1000.c | 12 +- hw/net/lan9118.c | 4 +- hw/net/pcnet-pci.c | 4 +- hw/net/pcnet.c | 10 +- hw/net/rtl8139.c | 28 +- hw/net/virtio-net.c | 20 +- hw/openrisc/cputimer.c | 10 +- hw/ppc/ppc.c | 64 ++-- hw/ppc/ppc405_uc.c | 8 +- hw/ppc/ppc_booke.c | 10 +- hw/ppc/spapr.c | 8 +- hw/sd/sdhci.c | 28 +- hw/sparc64/sun4u.c | 24 +- hw/timer/arm_mptimer.c | 12 +- hw/timer/arm_timer.c | 1 + hw/timer/cadence_ttc.c | 6 +- hw/timer/etraxfs_timer.c | 2 +- hw/timer/exynos4210_mct.c | 3 +- hw/timer/exynos4210_pwm.c | 1 + hw/timer/grlib_gptimer.c | 2 + hw/timer/hpet.c | 20 +- hw/timer/i8254.c | 26 +- hw/timer/i8254_common.c | 4 +- hw/timer/imx_epit.c | 1 + hw/timer/imx_gpt.c | 1 + hw/timer/lm32_timer.c | 1 + hw/timer/m48t59.c | 18 +- hw/timer/mc146818rtc.c | 50 +-- hw/timer/omap_gptimer.c | 24 +- hw/timer/omap_synctimer.c | 2 +- hw/timer/pl031.c | 19 +- hw/timer/puv3_ost.c | 1 + hw/timer/pxa2xx_timer.c | 34 +- hw/timer/sh_timer.c | 1 + hw/timer/slavio_timer.c | 1 + hw/timer/tusb6010.c | 12 +- hw/timer/twl92230.c | 14 +- hw/timer/xilinx_timer.c | 1 + hw/tpm/tpm_tis.c | 1 + hw/usb/hcd-ehci.c | 10 +- hw/usb/hcd-musb.c | 6 +- hw/usb/hcd-ohci.c | 12 +- hw/usb/hcd-uhci.c | 15 +- hw/usb/hcd-xhci.c | 26 +- hw/usb/host-libusb.c | 6 +- hw/usb/host-linux.c | 6 +- hw/usb/redirect.c | 16 +- hw/virtio/virtio-balloon.c | 8 +- hw/virtio/virtio-rng.c | 14 +- hw/watchdog/wdt_i6300esb.c | 6 +- hw/watchdog/wdt_ib700.c | 10 +- hw/xtensa/pic_cpu.c | 10 +- include/block/aio.h | 52 ++- include/block/block_int.h | 1 + include/block/blockjob.h | 2 +- include/block/coroutine.h | 3 +- include/hw/acpi/acpi.h | 2 +- include/qemu/ratelimit.h | 2 +- include/qemu/timer.h | 676 ++++++++++++++++++++++++++++++++--- include/qemu/typedefs.h | 3 + include/sysemu/sysemu.h | 2 +- main-loop.c | 57 ++- migration-exec.c | 1 + migration-fd.c | 1 + migration-tcp.c | 1 + migration-unix.c | 1 + migration.c | 17 +- monitor.c | 8 +- nbd.c | 1 + net/dump.c | 2 +- net/net.c | 1 + net/socket.c | 1 + qemu-char.c | 2 +- qemu-coroutine-io.c | 1 + qemu-coroutine-sleep.c | 10 +- qemu-io-cmds.c | 1 + qemu-nbd.c | 1 + qemu-timer.c | 834 +++++++++++++++---------------------------- qtest.c | 10 +- savevm.c | 24 +- scripts/switch-timer-api | 178 +++++++++ slirp/if.c | 2 +- slirp/misc.c | 1 + slirp/slirp.c | 4 +- stubs/clock-warp.c | 2 +- target-alpha/sys_helper.c | 12 +- target-ppc/kvm.c | 8 +- target-ppc/kvm_ppc.c | 6 +- target-s390x/cpu.c | 4 +- target-s390x/misc_helper.c | 6 +- target-xtensa/op_helper.c | 2 +- tests/libqtest.h | 24 +- tests/test-aio.c | 137 +++++++ tests/test-thread-pool.c | 3 + thread-pool.c | 1 + ui/console.c | 30 +- ui/input.c | 6 +- ui/spice-core.c | 10 +- ui/vnc-auth-sasl.h | 1 + ui/vnc-auth-vencrypt.c | 2 +- ui/vnc-ws.c | 1 + vl.c | 14 +- xen-all.c | 12 +- 172 files changed, 2286 insertions(+), 1458 deletions(-) create mode 100755 scripts/switch-timer-api -- 1.7.9.5