[RFC PATCH v4 00/18] VM introspection

["Adalber Lazăr" <alazar@bitdefender.com>]

From: Adalbert Lazar <alazar@bitdefender.com>

This patch series proposes a VM introspection subsystem for KVM (KVMI).

The previous RFC can be read here: https://marc.info/?l=kvm&m=150514457912721

These patches were tested on kvm/master,
commit 43aabca38aa9668eee3c3c1206207034614c0901 (Merge tag 'kvm-arm-fixes-for-v4.15-2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD).

In this iteration we refactored the code based on the feedback received
from Paolo and others.

The handshake
-------------
We no longer listen on a vsock in kernel, accepting introspectors
to control all the other VM-s. Instead, QEMU (ie. every introspected
guest) initiates the connection with an introspection tool (running on
the same host, in another VM, etc.) and passes the control to KVM where
the in-kernel mechanism will take over.

The administrator has to choose which guests should be introspected, by
which introspectors, what commands and events are allowed and for which
guests. Currently, there is a bitmask for allowed commands/events, but
it seems to be too complicated. For example, being allowed to set page
accesses (eg. r--) and not being allowed to receive page fault events
(eg. -wx) doesn't make sense.

The memory maping
-----------------
Besides the read/write commands to access guest memory, for performance
reasons, we've implemented memory mapping for introspection tools running
in another guest (on the same host, like page sharing between guests,
but without copy-on-write): the KVMI_GET_TOKEN command is used to obtain
a token, which is passed with a hypercall from the introspecting guest
to the KVMI.

While this didn't had a high priority, somehow the stars aligned and we
have it.

Page tracking
-------------
The current page tracking mechanism from KVM has support to track write
accesses (after the write operation took place). We've extended it with
preread, prewrite and preexec tracking.

We also added a notification for when a new memory slot is being
created (see track_create_slot()).

Pause VM
--------
We've removed the commands to pause/resume VM. Having a "pause vCPU"
command and a "paused vCPU" event seems to be enough for now.

Not implemented yet
-------------------

There are a few things documented, but not implemented yet: virtualized
exceptions, single-stepping and EPT views.

We are also working on accomodating SPP (Sub Page Protection).

We hope to make public our repositories (kernel, QEMU,
userland/simple-introspector) in a couple of days and we're looking
forward to add unit tests.

Changes since v3:
  - move the accept/handshake worker to QEMU
  - extend and use the 'page_track' infrastructure to intercept page
    accesses during emulation
  - remove the 0x40000000-0x40001fff range from monitored MSR-s
  - make small changes to the wire protocol (error codes, padding, names)
  - simplify KVMI_PAUSE_VCPU
  - add new commands: KVMI_GET_MAP_TOKEN, KVMI_GET_XSAVE
  - add pat to KVMI_EVENT
  - document KVM_HC_MEM_MAP and KVM_HC_MEM_UNMAP hypercalls

Changes since v2:
  - make small changes to the wire protocol (eg. use kvmi_error_code
    with every command reply, a few renames, etc.)
  - removed '_x86' from x86 specific structure names. Architecture
    specific structures will have the same name.
  - drop KVMI_GET_MTRR_TYPE and KVMI_GET_MTRRS (use KVMI_SET_REGISTERS)
  - drop KVMI_EVENT_ACTION_SET_REGS (use KVMI_SET_REGISTERS)
  - remove KVMI_MAP_PHYSICAL_PAGE_TO_GUEST and KVMI_UNMAP_PHYSICAL_PAGE_FROM_GUEST
    (to be replaced by a token+hypercall pair)
  - extend KVMI_GET_VERSION with allowed commnd/event masks
  - replace KVMI_PAUSE_GUEST/KVMI_UNPAUSE_GUEST with KVMI_PAUSE_VCPU
  - replace KVMI_SHUTDOWN_GUEST with KVMI_EVENT_ACTION_CRASH
  - replace KVMI_GET_XSAVE_INFO with KVMI_GET_CPUID
  - merge KVMI_INJECT_PAGE_FAULT and KVMI_INJECT_BREAKPOINT
    in KVMI_INJECT_EXCEPTION
  - replace event reply flags with ALLOW/SKIP/RETRY/CRASH actions
  - make KVMI_SET_REGISTERS work with vCPU events only
  - add EPT view support in KVMI_GET_PAGE_ACCESS/KVMI_SET_PAGE_ACCESS
  - add support for multiple pages in KVMI_GET_PAGE_ACCESS/KVMI_SET_PAGE_ACCESS
  - add (back) KVMI_READ_PHYSICAL/KVMI_WRITE_PHYSICAL
  - add KVMI_CONTROL_VE
  - add cstar to KVMI_EVENT
  - add new events: KVMI_EVENT_VCPU_PAUSED, KVMI_EVENT_CREATE_VCPU, 
    KVMI_EVENT_DESCRIPTOR_ACCESS, KVMI_EVENT_SINGLESTEP
  - add new sections: "Introspection capabilities", "Live migrations",
    "Guest snapshots with memory", "Memory access safety"
  - document the hypercall used by the KVMI_EVENT_HYPERCALL command
    (was KVMI_EVENT_USER_CALL)

Changes since v1:
  - add documentation and ABI [Paolo, Jan]
  - drop all the other patches for now [Paolo]
  - remove KVMI_GET_GUESTS, KVMI_EVENT_GUEST_ON, KVMI_EVENT_GUEST_OFF,
    and let libvirt/qemu handle this [Stefan, Paolo]
  - change the license from LGPL to GPL [Jan]
  - remove KVMI_READ_PHYSICAL and KVMI_WRITE_PHYSICAL (not used anymore)
  - make the interface a little more consistent


Adalbert Lazar (18):
  kvm: add documentation and ABI/API headers for the VM introspection
    subsystem
  add memory map/unmap support for VM introspection on the guest side
  kvm: x86: add kvm_arch_msr_intercept()
  kvm: x86: add kvm_mmu_nested_guest_page_fault() and
    kvmi_mmu_fault_gla()
  kvm: x86: add kvm_arch_vcpu_set_regs()
  kvm: vmx: export the availability of EPT views
  kvm: page track: add support for preread, prewrite and preexec
  kvm: add the VM introspection subsystem
  kvm: hook in the VM introspection subsystem
  kvm: x86: handle the new vCPU request (KVM_REQ_INTROSPECTION)
  kvm: x86: hook in the page tracking
  kvm: x86: hook in kvmi_breakpoint_event()
  kvm: x86: hook in kvmi_descriptor_event()
  kvm: x86: hook in kvmi_cr_event()
  kvm: x86: hook in kvmi_xsetbv_event()
  kvm: x86: hook in kvmi_msr_event()
  kvm: x86: handle the introspection hypercalls
  kvm: x86: hook in kvmi_trap_event()

 Documentation/virtual/kvm/00-INDEX       |    2 +
 Documentation/virtual/kvm/hypercalls.txt |   66 ++
 Documentation/virtual/kvm/kvmi.rst       | 1323 ++++++++++++++++++++++++++++
 arch/x86/Kconfig                         |    9 +
 arch/x86/include/asm/kvm_emulate.h       |    1 +
 arch/x86/include/asm/kvm_host.h          |   13 +
 arch/x86/include/asm/kvm_page_track.h    |   24 +-
 arch/x86/include/asm/kvmi_guest.h        |   10 +
 arch/x86/include/asm/vmx.h               |    2 +
 arch/x86/include/uapi/asm/kvmi.h         |  213 +++++
 arch/x86/kernel/Makefile                 |    1 +
 arch/x86/kernel/kvmi_mem_guest.c         |   26 +
 arch/x86/kvm/Makefile                    |    1 +
 arch/x86/kvm/emulate.c                   |    9 +-
 arch/x86/kvm/mmu.c                       |  156 +++-
 arch/x86/kvm/mmu.h                       |    4 +
 arch/x86/kvm/page_track.c                |  129 ++-
 arch/x86/kvm/svm.c                       |   66 ++
 arch/x86/kvm/vmx.c                       |  109 ++-
 arch/x86/kvm/x86.c                       |  141 ++-
 include/linux/kvm_host.h                 |    5 +
 include/linux/kvmi.h                     |   32 +
 include/linux/mm.h                       |    3 +
 include/trace/events/kvmi.h              |  174 ++++
 include/uapi/linux/kvm.h                 |    8 +
 include/uapi/linux/kvm_para.h            |   10 +-
 include/uapi/linux/kvmi.h                |  150 ++++
 mm/internal.h                            |    5 -
 virt/kvm/kvm_main.c                      |   19 +
 virt/kvm/kvmi.c                          | 1410 ++++++++++++++++++++++++++++++
 virt/kvm/kvmi_int.h                      |  121 +++
 virt/kvm/kvmi_mem.c                      |  730 ++++++++++++++++
 virt/kvm/kvmi_mem_guest.c                |  379 ++++++++
 virt/kvm/kvmi_msg.c                      | 1134 ++++++++++++++++++++++++
 34 files changed, 6438 insertions(+), 47 deletions(-)
 create mode 100644 Documentation/virtual/kvm/kvmi.rst
 create mode 100644 arch/x86/include/asm/kvmi_guest.h
 create mode 100644 arch/x86/include/uapi/asm/kvmi.h
 create mode 100644 arch/x86/kernel/kvmi_mem_guest.c
 create mode 100644 include/linux/kvmi.h
 create mode 100644 include/trace/events/kvmi.h
 create mode 100644 include/uapi/linux/kvmi.h
 create mode 100644 virt/kvm/kvmi.c
 create mode 100644 virt/kvm/kvmi_int.h
 create mode 100644 virt/kvm/kvmi_mem.c
 create mode 100644 virt/kvm/kvmi_mem_guest.c
 create mode 100644 virt/kvm/kvmi_msg.c

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>