Re: [PATCH v1 0/7] Implement support for external IPT monitoring

["Michał Leszczyński" <michal.leszczynski@cert.pl>]

----- 16 cze 2020 o 20:17, Andrew Cooper andrew.cooper3@citrix.com napisał(a):

> On 16/06/2020 16:16, Michał Leszczyński wrote:
>> Intel Processor Trace is an architectural extension available in modern Intel
>> family CPUs. It allows recording the detailed trace of activity while the
>> processor executes the code. One might use the recorded trace to reconstruct
>> the code flow. It means, to find out the executed code paths, determine
>> branches taken, and so forth.
>>
>> The abovementioned feature is described in Intel(R) 64 and IA-32 Architectures
>> Software Developer's Manual Volume 3C: System Programming Guide, Part 3,
>> Chapter 36: "Intel Processor Trace."
>>
>> This patch series implements an interface that Dom0 could use in order to enable
>> IPT for particular vCPUs in DomU, allowing for external monitoring. Such a
>> feature has numerous applications like malware monitoring, fuzzing, or
>> performance testing.
> 
> Hello,
> 
> I'm very excited to see support like this appearing.  However, be aware
> that we're currently in code freeze for the 4.14 release, so in-depth
> reviews will probably be delayed somewhat due to our bug queue and
> release activities.

Sure, take your time :)


> 
> That said, I've had a very quick look through the series, and have a few
> general questions first.
> 
> AFAICT, this is strictly for external monitoring of the VM, not for the
> VM to use itself?  If so, it shouldn't have the H tag here:
> 
> XEN_CPUFEATURE(IPT,           5*32+25) /*H  Intel Processor Trace */
> 
> because that exposes the feature to the guest, with the implication that
> all other parts of the feature work as advertised.

Ok, I will remove the H tag.


> 
> 
> Are there any restrictions on EPT being enabled in the first place?  I'm
> not aware of any, and in principle we could use this functionality for
> PV guests as well (using the CPL filter).  Therefore, I think it would
> be helpful to not tie the functionality to HVM guests, even if that is
> the only option enabled to start with.

I think at the moment it's not required to have EPT. This patch series doesn't use any translation feature flags, so the output address is always a machine physical address, regardless of context. I will check if it could be easily used with PV.


> 
> The buffer mapping and creation logic is fairly problematic.  Instead of
> fighting with another opencoded example, take a look at the IOREQ
> server's use of "acquire resource" which is a mapping interface which
> supports allocating memory on behalf of the guest, outside of the guest
> memory, for use by control tools.
> 
> I think what this wants is a bit somewhere in domain_create to indicate
> that external tracing is used for this domain (and allocate whatever
> structures/buffers are necessary), acquire resource to map the buffers
> themselves, and a domctl for any necessary runtime controls.
> 

I will check this out, this sounds like a good option as it would remove lots of complexity from the existing ipt_enable domctl.

> 
> What semantics do you want for the buffer becoming full?  Given that
> debugging/tracing is the goal, I presume "pause vcpu on full" is the
> preferred behaviour, rather than drop packets on full?
> 

Right now this is a ring-style buffer and when it would become full it would simply wrap and override the old data.

> 
> When this subject was broached on xen-devel before, one issue was the
> fact that all actions which are intercepted don't end up writing any
> appropriate packets.  This is perhaps less of an issue for this example,
> where the external agent can see VMExits in the trace, but it still
> results in missing information.  (It is a major problem for PT within
> the guest, and needs Xen's intercept/emulation framework being updated
> to be PT-aware so it can fill in the same packets which hardware would
> have done for equivalent actions.)

Ok, this sounds like a hard issue. Could you point out what could be the particular problematic cases? For instance, if something would alter EIP/RIP or CR3 then I belive it would still be recorded in PT trace (i.e. these values will be logged on VM entry).

> 
> 
> Thanks,
> 
> ~Andrew


Best regards,
Michał Leszczyński
CERT Polska