From: Stefan Berger <stefanb@linux.vnet.ibm.com>
To: kvm@vger.kernel.org
Subject: Errors on MMIO read access on VM suspend / resume operations
Date: Tue, 11 Jan 2011 11:19:17 -0500 [thread overview]
Message-ID: <4D2C8305.2090609@linux.vnet.ibm.com> (raw)
Hi!
I am currently doing some long-term testing of a device model using
memory mapped IO (TPM TIS) and am seeing some strange errors when the
suspend occurs in the middle of a read operation in the Linux TPM TIS
device driver where the driver reads the result packet from the mmio
location.
Short background: The TPM response packet is read in 2 chunks. First
the first 10 bytes are read containing the response's header.
Subsequently the rest of the packet is read using knowledge of the total
size of the response packet from the header (bytes 2-5 in big endian
format). The corresponding code reading the data from the hardware
interface is here:
http://lxr.linux.no/#linux+v2.6.37/drivers/char/tpm/tpm_tis.c#L228
The test I am running is setup as follows:
- inside the VM keys are permanently generated by sending commands to
the TPM; packets read from the interface are dumped to the screen
- on the host a script suspends the VM every 6 seconds and resumes it
immediately afterwards (using libvirt)
As it happens, sometimes the VM is suspended in the middle of a read
operation on the TPM TIS interface -- see above code reference. I see
that because I do dump the state of the TPM TIS when suspending and see
that the read offset is pointing to a location somewhere in the middle
of the packet - so the TPM TIS Linux driver is in the above loop
currently reading the data. I am observing two types of results if this
happens:
- either the result read by the Linux TPM TIS driver is ok, so no
problem here
- or the problematic case where the TPM TIS driver reads a packet with a
byte missing and then at the end gets a zero byte from the TPM TIS
interface indicating that it read beyond the available data. If the
suspend happened while reading the first chunk of data (header), the TPM
TIS driver will also complain that the available data for the 2nd chunk
(burst size) is less than what's expected -- it's an off-by-one error
So, I then modified the TPM TIS device model to decrement the read
offset pointer by '1' in case it was detected that the suspend happened
in the middle of the read operation -- in Qemu I do this in the
post-load 'method'. This then leads to the following types of results:
- the problematic(!) case where the read packet was ok
- the expected case where the TPM TIS driver reads the packet and ends
up having two same bytes in the result in consecutive array locations;
besides that the TPM TIS driver will in this case complain that it has
left-over data
So my conclusion from the above tests are:
- for some reason the memory read to the MMIO location happens as the
last instruction executed on suspend and again as the very first on
executed on resume. This explains to me that the TPM TIS model internal
pointer into the packet was advanced by '1' (the packet is read by
subsequently reading from the same memory location) and the above
problematic cases make sense
- the other instruction in the Linux TPM TIS drivers that for example
advance the buffer location do not execute twice, i.e., size++ in the
buf[size++] = ... in the Linux driver.
What puzzles me is that the read operation may be run twice but others
don't.
If you have insights as why the above may be occurring, please let me
know. A simple solution to work around this may be to introduce a
register holding the index into the result packet where to read the next
byte from (rather than advancing an internal pointer to the next byte),
though this would deviate the driver from the standard interface the
model currently implements.
Regards,
Stefan
next reply other threads:[~2011-01-11 16:19 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-01-11 16:19 Stefan Berger [this message]
2011-01-13 10:22 ` Errors on MMIO read access on VM suspend / resume operations Avi Kivity
2011-01-14 19:27 ` Stefan Berger
2011-01-16 14:43 ` Avi Kivity
2011-01-18 3:03 ` Stefan Berger
2011-01-18 3:03 ` [Qemu-devel] " Stefan Berger
2011-01-18 8:53 ` Jan Kiszka
2011-01-18 8:53 ` [Qemu-devel] " Jan Kiszka
2011-01-24 18:27 ` Stefan Berger
2011-01-24 18:27 ` [Qemu-devel] " Stefan Berger
2011-01-24 22:34 ` Jan Kiszka
2011-01-24 22:34 ` [Qemu-devel] " Jan Kiszka
2011-01-25 3:13 ` Stefan Berger
2011-01-25 7:26 ` Jan Kiszka
2011-01-25 16:49 ` Stefan Berger
2011-01-26 8:14 ` Jan Kiszka
2011-01-26 12:05 ` Stefan Berger
2011-01-26 12:09 ` Jan Kiszka
2011-01-26 13:08 ` Stefan Berger
2011-01-26 13:15 ` Jan Kiszka
2011-01-26 13:31 ` Jan Kiszka
2011-01-26 13:52 ` Stefan Berger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D2C8305.2090609@linux.vnet.ibm.com \
--to=stefanb@linux.vnet.ibm.com \
--cc=kvm@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.