Re: [PATCH v7 06/10] xen/domctl: Add XEN_DOMCTL_vmtrace_op

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 26/01/2021 14:18, Jan Beulich wrote:
>> +static int vmtrace_set_option(struct vcpu *v, uint64_t key, uint64_t value)
>> +{
>> +    struct vcpu_msrs *msrs = v->arch.msrs;
>> +    bool new_en, old_en = msrs->rtit.ctl & RTIT_CTL_TRACE_EN;
>> +
>> +    switch ( key )
>> +    {
>> +    case MSR_RTIT_OUTPUT_MASK:
>> +        /*
>> +         * MSR_RTIT_OUTPUT_MASK, when using Single Output mode, has a limit
>> +         * field in the lower 32 bits, and an offset in the upper 32 bits.
>> +         *
>> +         * Limit is fixed by the vmtrace buffer size and must not be
>> +         * controlled by userspace, while offset must be within the limit.
>> +         *
>> +         * Drop writes to the limit field to simply userspace wanting to reset
>> +         * the offset by writing 0.
>> +         */
>> +        if ( (value >> 32) > msrs->rtit.output_limit )
>> +            return -EINVAL;
>> +        msrs->rtit.output_offset = value >> 32;
>> +        break;
>> +
>> +    case MSR_RTIT_CTL:
>> +        if ( value & ~RTIT_CTL_MASK )
>> +            return -EINVAL;
>> +
>> +        msrs->rtit.ctl &= ~RTIT_CTL_MASK;
>> +        msrs->rtit.ctl |= (value & RTIT_CTL_MASK);
>> +        break;
>> +
>> +    case MSR_RTIT_STATUS:
>> +        if ( value & ~RTIT_STATUS_MASK )
>> +            return -EINVAL;
>> +
>> +        msrs->rtit.status &= ~RTIT_STATUS_MASK;
>> +        msrs->rtit.status |= (value & RTIT_STATUS_MASK);
>> +        break;
>> +
>> +    default:
>> +        return -EINVAL;
>> +    }
>> +
>> +    new_en = msrs->rtit.ctl & RTIT_CTL_TRACE_EN;
>> +
>> +    /* ctl.trace_en changed => update MSR load/save lists appropriately. */
>> +    if ( !old_en && new_en )
>> +    {
>> +        if ( vmx_add_guest_msr(v, MSR_RTIT_CTL, msrs->rtit.ctl) ||
>> +             vmx_add_host_load_msr(v, MSR_RTIT_CTL, 0) )
>> +        {
>> +            /*
>> +             * The only failure cases here are failing the
>> +             * singleton-per-domain memory allocation, or exceeding the space
>> +             * in the allocation.  We could unwind in principle, but there is
>> +             * nothing userspace can usefully do to continue using this VM.
>> +             */
>> +            domain_crash(v->domain);
>> +            return -ENXIO;
> I don't think I fully agree with the 2nd half of the last
> sentence, but well, so be it then for the time being at least.
> Why could userspace not decide to continue running this VM
> with ipt disabled?

Technically speaking, it could.  That wouldn't malfunction.

However, it would be exceedingly poor behaviour.

One major limitation IPT has is that it cant pause on a full ring (or at
least, not in any shipping hardware yet, and this series works back to
Broadwell).  You can't just leave IPT enabled and let the VM run,
because the buffer will wrap and corrupt itself.

The driving usecase for adding IPT is introspection based.  Frequent
breaks, combined with massive trace buffers, is the best effort attempt
not to lose data.

IPT is a niche usecase - it does come with a substantial frequency hit,
and lots of userspace complexity to do anything interesting with. 
Anyone who turns it on to begin with has a usecase which totally depends
on it working.

>> +static int vmtrace_control(struct vcpu *v, bool enable, bool reset)
>> +{
>> +    struct vcpu_msrs *msrs = v->arch.msrs;
>> +    uint64_t new_ctl;
>> +    int rc;
>> +
>> +    if ( v->arch.hvm.vmx.ipt_active == enable )
>> +        return -EINVAL;
> Why is XEN_DOMCTL_vmtrace_reset_and_enable not permitted
> when ipt_active is true?

Because absolutely nothing good can come of userspace and Xen getting
out of sync with their combined idea of whether IPT is active or not.

And I really don't feel like doing an ipt_pause reference count, because
there cannot plausibly be more than one entity handling the data.

>  And, considering ...
>
>> +    if ( reset )
>> +    {
>> +        msrs->rtit.status = 0;
>> +        msrs->rtit.output_offset = 0;
>> +    }
>> +
>> +    new_ctl = msrs->rtit.ctl & ~RTIT_CTL_TRACE_EN;
>> +    if ( enable )
>> +        new_ctl |= RTIT_CTL_TRACE_EN;
>> +
>> +    rc = vmtrace_set_option(v, MSR_RTIT_CTL, new_ctl);
> ... this is just a wrapper around a function directly
> reachable via XEN_DOMCTL_vmtrace_set_option, why any
> restriction at all?

This partial alias is a consequence of the split between the platform
neutral, and platform specific parts of the interface.

It is by no means certain that such an alias would exist on other
platforms, and passing TRACE_EN to set_option() falls firmly in the
"don't do that" category IMO.

~Andrew