Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

[Jinhao Fan <fanjinhao21s@ict.ac.cn>]

After digging through the source code, I found event_notifier_cleanup() only
closes the eventfd, but does not de-register the event from QEMU’s event
loop. event_notifier_set_handler() actually interacts with the event loop.
It is a wrapper around aio_set_event_notifier(), which is again a wrapper of
aio_set_fd_handler(). Passing in a handler of NULL means removing the fd
handler from the event loop.

Therefore, originally in nvme_free_sq/cq(), we closed the eventfd but did
not delete its handler. I guess maybe sometime later the fd is reused and
that corresponding file is somehow written (A POLLIN event), this will
trigger the event loop to call the stale handler, which causes this bug.

So the correct order is:

Init:
1. event_notifier_init: Open the eventfd
2. event_notifier_set_handler: Register on the event loop
3. memory_region_add_eventfd: Associate with IO region

Cleanup:
1. memory_region_del_eventfd: De-associate with IO region
2. event_notifier_set_handler(NULL): Remove from the event loop
3. event_notifier_cleanup: Close the eventfd