Re: [tip:x86/urgent] x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

From: Karol Herbst <kherbst@redhat.com>
To: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	linux-kernel@vger.kernel.org,
	Pekka Paalanen <ppaalanen@gmail.com>,
	Lyude Paul <lyude@redhat.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	torvalds@linux-foundation.org, hpa@zytor.com,
	linux-tip-commits@vger.kernel.org,
	Karol Herbst <karolherbst@gmail.com>
Subject: Re: [tip:x86/urgent] x86/mm/kmmio: Fix mmiotrace for page unaligned addresses
Date: Tue, 12 Dec 2017 15:32:42 +0100	[thread overview]
Message-ID: <CACO55tuuOb6GRg0VPjWLs1nRanC5wZ1yixePSFsEhaaaFA=njw@mail.gmail.com> (raw)
In-Reply-To: <20171212140418.56xld2lxja5rywp7@gmail.com>

Hi Peter,

the basic idea is to detect if a driver accesses a memory region
mapped through ioremap. This is super usefull for reverse engineering
closed source drivers like the Nvidia GPU driver.

So here is what it does:
1. on ioremap the entire memory region mapped is registered in the
mmiotracer and marked as not presen, which basically leads to page
faults on acces
2. mmiotrace is the registered page fault handler for those pages and
while handling the page (which basically means marking them as presen,
because they were never missing in the first place) it parses the
current instruction to detect if it was a read or write and writes
relevant information into a file. This includes address accessed,
value read/written, type of instruction
3. after single stepping, the page is marked as not present again
4. on unmap time, mmiotrace unregisteres those regions and marks them as present

this is more or less the basic idea.

And to answer your question how it is not completely broken: I don't
know. It works for us (more or less, we can't parse repeat
instructions as one example what does not work) and if we come across
issues we try to fix them on the way.

Anyway, this is a super useful tool to record and debug what a driver
is doing with hardware and helps tracking down a lot of this,
especially for Nouveau.

I hope that helps.

On Tue, Dec 12, 2017 at 3:04 PM, Ingo Molnar <mingo@kernel.org> wrote:
>
> * Peter Zijlstra <peterz@infradead.org> wrote:
>
>> On Tue, Dec 12, 2017 at 02:55:30AM -0800, tip-bot for Karol Herbst wrote:
>> > Commit-ID:  6d60ce384d1d5ca32b595244db4077a419acc687
>> > Gitweb:     https://git.kernel.org/tip/6d60ce384d1d5ca32b595244db4077a419acc687
>> > Author:     Karol Herbst <kherbst@redhat.com>
>> > AuthorDate: Mon, 27 Nov 2017 08:51:39 +0100
>> > Committer:  Ingo Molnar <mingo@kernel.org>
>> > CommitDate: Mon, 11 Dec 2017 15:35:18 +0100
>> >
>> > x86/mm/kmmio: Fix mmiotrace for page unaligned addresses
>>
>> OK, let me hijack this thread since apparently people use and care about
>> mmiotrace.
>>
>> I was recently auditing the x86 tlb flushing and ran across this
>> 'thing'. Can someone please explain to me how this is supposed to work
>> and how its not completely broken?
>
> (I have Cc:-ed other gents as well.)
>
> Thanks,
>
>         Ingo