From: Richard Henderson <richard.henderson@linaro.org>
To: "Jørgen Hansen" <Jorgen.Hansen@wdc.com>
Cc: Ajay Joshi <Ajay.Joshi@wdc.com>,
qemu-devel <qemu-devel@nongnu.org>,
Sid Manning <sidneym@quicinc.com>
Subject: Re: An issue with x86 tcg and MMIO
Date: Wed, 1 Feb 2023 11:50:50 -1000 [thread overview]
Message-ID: <ff3f25ee-1c98-242b-905e-0b01d9f0948d@linaro.org> (raw)
In-Reply-To: <78bc53e3-bad3-a5c3-9e53-7a89054aa37a@wdc.com>
On 2/1/23 05:24, Jørgen Hansen wrote:
> Hello Richard,
>
> We are using x86 qemu to test some CXL stuff, and in that process we are
> running into an issue with tcg. In qemu, CXL memory is mapped as an MMIO
> region, so when using CXL memory as part of the system memory,
> application code and data can be spread out across a combination of DRAM
> and CXL memory (we are using the Linux tiered memory numa balancing,
> that will migrate individual pages between DRAM and CXL memory based on
> access patterns). When we are running memory intensive applications, we
> hit the following assert in translator_access:
>
> /* We cannot handle MMIO as second page. */
> assert(tb->page_addr[1] != -1);
>
> introduced in your commit 50627f1b. This is using existing applications
> and standard Linux. We discussed this with Alistair Francis and he
> mentioned that it looks like a load across a page boundary is happening,
> and it so happens that the first page is DRAM and second page MMIO. We
> tried - as a workaround - to return NULL instead of the assert to
> trigger the slow path processing, and that allows the system to make
> forward progress, but we aren't familiar with tcg, and as such aren't
> sure if that is a correct fix.
>
> So we'd like to get your input on this - and understand whether the
> above usage isn't supported at all or if there are other possible
> workarounds.
Well, this may answer my question in
https://lore.kernel.org/qemu-devel/1d6b1894-9c45-2d70-abde-9c10c1b3b93f@linaro.org/
as to how this could occur.
Until relatively recently, TCG would refuse to execute out of MMIO *at all*. This was
relaxed to support Arm m-profile, which needs to execute a few instructions out of MMIO
during the boot process, before jumping into flash.
This works by reading one instruction, translating it, executing it, and immediately
discarding the translation. It could be possible to adjust the translator to allow the
second half of an instruction to be in MMIO, such that we execute and discard, however...
What is it about CXL that requires modeling with MMIO? If it is intended to be used
interchangeably with RAM by the guest, then you really won't like the performance you will
see with TCG executing out of these regions.
Could memory across the CXL link be modeled as a ROM device, similar to flash? This does
not have the same restrictions as MMIO.
r~
next parent reply other threads:[~2023-02-01 21:51 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <78bc53e3-bad3-a5c3-9e53-7a89054aa37a@wdc.com>
2023-02-01 21:50 ` Richard Henderson [this message]
2023-02-02 9:39 ` An issue with x86 tcg and MMIO Jonathan Cameron via
2023-02-02 10:56 ` Richard Henderson
2023-02-02 11:39 ` Peter Maydell
2023-02-02 12:31 ` Jonathan Cameron via
2023-02-02 13:31 ` Peter Maydell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ff3f25ee-1c98-242b-905e-0b01d9f0948d@linaro.org \
--to=richard.henderson@linaro.org \
--cc=Ajay.Joshi@wdc.com \
--cc=Jorgen.Hansen@wdc.com \
--cc=qemu-devel@nongnu.org \
--cc=sidneym@quicinc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).