Re: [RFC PATCH 00/19] Guest introspection

From: Jan Kiszka <jan.kiszka@siemens.com>
To: alazar@bitdefender.com, Stefan Hajnoczi <stefanha@gmail.com>
Cc: "Mihai Dontu" <mdontu@bitdefender.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Radim Krčmář" <rkrcmar@redhat.com>,
	kvm@vger.kernel.org
Subject: Re: [RFC PATCH 00/19] Guest introspection
Date: Tue, 20 Jun 2017 17:03:37 +0200	[thread overview]
Message-ID: <da4620cf-3949-0ee9-24ff-8dbadcd6e421@siemens.com> (raw)
In-Reply-To: <1497970721.139b8aD.12149@host>

On 2017-06-20 16:58, alazar@bitdefender.com wrote:
> On Mon, 19 Jun 2017 10:39:28 +0100, Stefan Hajnoczi <stefanha@gmail.com> wrote:
>> On Fri, Jun 16, 2017 at 05:34:48PM +0200, Jan Kiszka wrote:
>>> On 2017-06-16 17:18, Mihai Donțu wrote:
>>>> On Fri, 2017-06-16 at 16:45 +0200, Jan Kiszka wrote:
>>>>> On 2017-06-16 15:43, Adalbert Lazar wrote:
>>>>>> This patch series proposes an interface that will allow a guest
>>>>>> introspection tool to monitor and control other guests, in order to
>>>>>> protect them against different forms of exploits. This type of interface
>>>>>> is already present in the XEN hypervisor.
>>>>>>
>>>>>> With the current implementation, the introspection tool connects to
>>>>>> the KVMi (the introspection subsystem from KVM) using a vsock socket,
>>>>>> establishes a main communication channel, used for a few messages
>>>>>> (KVMI_EVENT_GUEST_ON, KVMI_EVENT_GUEST_OFF, KVMI_GET_GUESTS and
>>>>>> KVMI_GET_VERSION).
>>>>>>
>>>>>> Every KVMI_EVENT_GUEST_ON notification, makes the introspection tool
>>>>>> establish a new connection, used to monitor and control that guest.
>>>>
>>>>> What prevented building this on top of the already existing guest debug
>>>>> interfaces of KVM, maybe extending it where needed? Could be win-win.
>>>>
>>>> I might be mistaking, but this would require the application using the
>>>> introspection capabilities to run on the host. If so, what we are
>>>> trying to do is to isolate the application into its own VM. This is why
>>>> we use vSock to communicate with the host.
>>>
>>> The hypervisor process could terminate your link, providing that other
>>> VM the introspection access. Or you even have a gdb-speaking process
>>> running on the host, just reusing the existing gdbstub of QEMU. Just
>>> wild ideas, I didn't look into details, and you may further elaborate on
>>> your requirements.
>>
>> QEMU userspace can provide the interface on an AF_VSOCK listen socket.
>> Here is a rough idea:
>>
>>   qemu --chardev socket,id=chardev0,type=vsock,port=1234,server,nowait \
>>        --guest-introspection chardev=chardev0,allowed-cids=10
>>
>> Since it uses a chardev this means the guest introspection API is also
>> available via UNIX domain sockets to local applications, etc.
> 
> Exposing the introspections commands and events as IOCTLs can be done easily.
> 
> However, dropping the vsock access from the kernel, or having
> the IOCTL <-> vsock adapter only in userspace will change more things.
> With the proposed API, the guest introspection looks like:
> 
>  ------------------                  -----------------------------
> |                  |<-- /dev/kvm -->| qemu        VM1             |
> |                  |                |-------                      |
> |                  |                | Linux |                     |
> | KVM              |                 -----------------------------
> |                  |<-- /dev/kvm -->| qemu        VM2             |
> |                  |                |---------                    |
> |                  |                | Windows |                   |
> |                  |                 -----------------------------
> |                  |<-- /dev/kvm -->| qemu        VM3             |
> |    --------------|                |---------------------------  |
> |   | kvmi <- vhost-vsock -----------> guest introspection tool | |
>  ------------------                  -----------------------------
> 
> The introspection tool connects directly to the introspection subsystem (kvmi).
> 
> Moving the vsock to userland will change this:
> 
>                                      -----------------------------
>                  /----- /dev/kvm -->| new_tool (guest on/off/list)|<-- vsock -->\
>                  |                   -----------------------------              |
>                  |                                                              |
>  ----------------v-                  -----------------------------              |
> |                  |<-- /dev/kvm -->| qemu        VM1             |<-- vsock -->|
> |                  |                |-------                      |             |
> |                  |                | Linux |                     |             |
> | KVM              |                 -----------------------------              |
> |                  |<-- /dev/kvm -->| qemu        VM2             |<-- vsock -->|
> |                  |                |---------                    |             |
> |                  |                | Windows |                   |             |
> |                  |                 -----------------------------              |
> |                  |<-- /dev/kvm -->| qemu        VM3      /----->|<-- vsock -->/
> |           -------|                |---------------------v----   |
> |          | kvmi  |                | guest introspection tool |  |
>  ------------------                  -----------------------------
> 
> There will be a need for a new tool (and/or libvirt modified) to get
> the guest events (on/off/list) and change the VM1, VM2 invocations (to
> make them connect with the introspection tool). This might also be a
> problem with products having the host locked down (eg. RHEV).
> 
> Moving more code to QEMU will make the introspection harder with other
> hw emulators (from kvmtool to Google's implementation) because they will

I would bet that (not only) Google folks will look rather skeptical at
any proposal to widen the existing kvm user interface anyway, because of
security implications. Reducing that should be a primary design goal IMHO.

Jan

> need more than just:
>    - migration notification: the introspection tool needs to unhook
>      from the introspected guests [minor change]
>    - KVM_SET_VM_UUID support [minor change]
>    - vsock [medium change], but only if this particular emulator is used
>      to start the guest introspection tool
> 
> Also, the path between kvmi and the introspection tool (when running
> isolated in a guest) will be bigger, adding some overhead, which is
> a big problem with live introspectors. EXTERIOR VMI paper shows[1]
> between 5-20 times slower execution for small programs (ps, uptime).
> So, a smaller path will help in keeping the overhead lower.
> 
> My colleague will follow up with some stats collected during an
> introspection session. Hopefully they will shed more light on the
> performance required from the tool-KVM communication channel.
> 
> [1]: https://www.utdallas.edu/~zhiqiang.lin/file/VEE13-Slides.pdf
> 

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux