* [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking
@ 2022-02-22 15:26 Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
` (15 more replies)
0 siblings, 16 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel
Cc: Andrew Cooper, Jan Beulich, Stefano Stabellini, Wei Liu,
Julien Grall, Roger Pau Monné,
Juergen Gross, Marek Marczykowski-Górecki
To avoid spamming everyone, I have only re-sent patches with changes in v3.
CET Indirect Branch Tracking is a hardware feature designed to protect against
forward-edge control flow hijacking (Call/Jump oriented programming), and is a
companion feature to CET Shadow Stacks added in Xen 4.14.
Patches 1 thru 5 are prerequisites. Patches 6 thru 59 are fairly mechanical
annotations of function pointer targets. Patches 60 thru 70 are the final
enablement of CET-IBT.
This series functions correctly with GCC 9 and later, although an experimental
GCC patch is required to get more helpful typechecking at build time. A
container with this fix has been added to CI.
Tested on a TigerLake NUC by me, and by Marek also.
CI pipelines:
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/476819536
https://cirrus-ci.com/build/4634902334275584
Andrew Cooper (67):
xen/sort: Switch to an extern inline implementation
xen/xsm: Move {do,compat}_flask_op() declarations into a header
x86/kexec: Annotate embedded data with ELF metadata
x86: Introduce support for CET-IBT
xen: CFI hardening for x86 hypercalls
xen: CFI hardening for custom_param()
xen: CFI hardening for __initcall()
xen: CFI hardening for notifier callbacks
xen: CFI hardening for acpi_table_parse()
xen: CFI hardening for continue_hypercall_on_cpu()
xen: CFI hardening for init_timer()
xen: CFI hardening for call_rcu()
xen: CFI hardening for IPIs
xen: CFI hardening for open_softirq()
xsm/flask/ss: CFI hardening
xsm: CFI hardening
xen/sched: CFI hardening
xen/evtchn: CFI hardening
xen/hypfs: CFI hardening
xen/tasklet: CFI hardening
xen/keyhandler: CFI hardening
xen/vpci: CFI hardening
xen/decompress: CFI hardening
xen/iommu: CFI hardening
xen/video: CFI hardening
xen/console: CFI hardening
xen/misc: CFI hardening
x86: CFI hardening for request_irq()
x86/hvm: CFI hardening for hvm_funcs
x86/hvm: CFI hardening for device emulation
x86/emul: CFI hardening
x86/ucode: CFI hardening
x86/power: CFI hardening
x86/apic: CFI hardening
x86/nmi: CFI hardening
x86/mtrr: CFI hardening
x86/idle: CFI hardening
x86/quirks: CFI hardening
x86/hvmsave: CFI hardening
x86/mce: CFI hardening
x86/pmu: CFI hardening
x86/cpu: CFI hardening
x86/guest: CFI hardening
x86/logdirty: CFI hardening
x86/shadow: CFI hardening
x86/hap: CFI hardening
x86/p2m: CFI hardening
x86/irq: CFI hardening
x86/apei: CFI hardening
x86/psr: CFI hardening
x86/dpci: CFI hardening
x86/pt: CFI hardening
x86/time: CFI hardening
x86/misc: CFI hardening
x86/stack: CFI hardening
x86/bugframe: CFI hardening
x86: Use control flow typechecking where possible
x86/setup: Read CR4 earlier in __start_xen()
x86/alternatives: Clear CR4.CET when clearing CR0.WP
x86/traps: Rework write_stub_trampoline() to not hardcode the jmp
x86: Introduce helpers/checks for endbr64 instructions
x86/emul: Update emulation stubs to be CET-IBT compatible
x86/entry: Make syscall/sysenter entrypoints CET-IBT compatible
x86/entry: Make IDT entrypoints CET-IBT compatible
x86/setup: Rework MSR_S_CET handling for CET-IBT
x86/efi: Disable CET-IBT around Runtime Services calls
x86: Enable CET Indirect Branch Tracking
Juergen Gross (2):
x86/pv-shim: Don't modify the hypercall table
x86: Don't use the hypercall table for calling compat hypercalls
Marek Marczykowski-Górecki (1):
x86: Build check for embedded endbr64 instructions
Config.mk | 1 -
README | 1 +
automation/build/debian/buster-gcc-ibt.dockerfile | 66 ++++
automation/gitlab-ci/build.yaml | 6 +
automation/scripts/containerize | 1 +
docs/misc/xen-command-line.pandoc | 16 +-
tools/firmware/Makefile | 2 +
tools/libs/guest/xg_dom_decompress_unsafe.h | 2 +
tools/tests/x86_emulator/x86-emulate.h | 2 +
xen/arch/arm/bootfdt.c | 9 +-
xen/arch/arm/io.c | 9 +-
xen/arch/x86/Kconfig | 17 +
xen/arch/x86/Makefile | 6 +
xen/arch/x86/acpi/boot.c | 24 +-
xen/arch/x86/acpi/cpu_idle.c | 43 ++-
xen/arch/x86/acpi/cpufreq/cpufreq.c | 24 +-
xen/arch/x86/acpi/cpufreq/powernow.c | 21 +-
xen/arch/x86/acpi/cpuidle_menu.c | 6 +-
xen/arch/x86/acpi/lib.c | 2 +-
xen/arch/x86/acpi/power.c | 4 +-
xen/arch/x86/acpi/wakeup_prot.S | 38 +-
xen/arch/x86/alternative.c | 13 +-
xen/arch/x86/apic.c | 12 +-
xen/arch/x86/arch.mk | 7 +
xen/arch/x86/boot/x86_64.S | 30 +-
xen/arch/x86/compat.c | 21 +-
xen/arch/x86/configs/pvshim_defconfig | 1 +
xen/arch/x86/cpu/amd.c | 8 +-
xen/arch/x86/cpu/centaur.c | 2 +-
xen/arch/x86/cpu/common.c | 3 +-
xen/arch/x86/cpu/cpu.h | 2 +-
xen/arch/x86/cpu/hygon.c | 2 +-
xen/arch/x86/cpu/intel.c | 6 +-
xen/arch/x86/cpu/mcheck/amd_nonfatal.c | 4 +-
xen/arch/x86/cpu/mcheck/mce.c | 22 +-
xen/arch/x86/cpu/mcheck/mce.h | 2 +-
xen/arch/x86/cpu/mcheck/mce_amd.c | 9 +-
xen/arch/x86/cpu/mcheck/mce_amd.h | 4 +-
xen/arch/x86/cpu/mcheck/mce_intel.c | 49 ++-
xen/arch/x86/cpu/mcheck/non-fatal.c | 6 +-
xen/arch/x86/cpu/mcheck/vmce.c | 4 +-
xen/arch/x86/cpu/microcode/amd.c | 9 +-
xen/arch/x86/cpu/microcode/core.c | 15 +-
xen/arch/x86/cpu/microcode/intel.c | 10 +-
xen/arch/x86/cpu/mtrr/generic.c | 20 +-
xen/arch/x86/cpu/mtrr/main.c | 4 +-
xen/arch/x86/cpu/mtrr/mtrr.h | 8 +-
xen/arch/x86/cpu/mwait-idle.c | 12 +-
xen/arch/x86/cpu/shanghai.c | 2 +-
xen/arch/x86/cpu/vpmu.c | 13 +-
xen/arch/x86/cpu/vpmu_amd.c | 16 +-
xen/arch/x86/cpu/vpmu_intel.c | 16 +-
xen/arch/x86/cpuid.c | 8 +-
xen/arch/x86/crash.c | 7 +-
xen/arch/x86/dmi_scan.c | 10 +-
xen/arch/x86/dom0_build.c | 8 +-
xen/arch/x86/domain.c | 16 +-
xen/arch/x86/emul-i8254.c | 14 +-
xen/arch/x86/extable.c | 18 +-
xen/arch/x86/genapic/bigsmp.c | 4 +-
xen/arch/x86/genapic/delivery.c | 12 +-
xen/arch/x86/genapic/probe.c | 2 +-
xen/arch/x86/genapic/x2apic.c | 18 +-
xen/arch/x86/guest/hyperv/hyperv.c | 10 +-
xen/arch/x86/guest/xen/xen.c | 15 +-
xen/arch/x86/hpet.c | 29 +-
xen/arch/x86/hvm/dm.c | 5 +-
xen/arch/x86/hvm/dom0_build.c | 16 +-
xen/arch/x86/hvm/emulate.c | 93 +++--
xen/arch/x86/hvm/hpet.c | 12 +-
xen/arch/x86/hvm/hvm.c | 47 +--
xen/arch/x86/hvm/hypercall.c | 5 +-
xen/arch/x86/hvm/intercept.c | 28 +-
xen/arch/x86/hvm/io.c | 38 +-
xen/arch/x86/hvm/ioreq.c | 2 +-
xen/arch/x86/hvm/irq.c | 16 +-
xen/arch/x86/hvm/mtrr.c | 8 +-
xen/arch/x86/hvm/nestedhvm.c | 6 +-
xen/arch/x86/hvm/pmtimer.c | 10 +-
xen/arch/x86/hvm/quirks.c | 4 +-
xen/arch/x86/hvm/rtc.c | 18 +-
xen/arch/x86/hvm/stdvga.c | 19 +-
xen/arch/x86/hvm/svm/nestedsvm.c | 22 +-
xen/arch/x86/hvm/svm/svm.c | 404 +++++++++++-----------
xen/arch/x86/hvm/svm/vmcb.c | 2 +-
xen/arch/x86/hvm/vioapic.c | 12 +-
xen/arch/x86/hvm/viridian/time.c | 2 +-
xen/arch/x86/hvm/viridian/viridian.c | 17 +-
xen/arch/x86/hvm/vlapic.c | 25 +-
xen/arch/x86/hvm/vmsi.c | 16 +-
xen/arch/x86/hvm/vmx/intr.c | 2 +-
xen/arch/x86/hvm/vmx/vmcs.c | 22 +-
xen/arch/x86/hvm/vmx/vmx.c | 155 +++++----
xen/arch/x86/hvm/vmx/vvmx.c | 16 +-
xen/arch/x86/hvm/vpic.c | 8 +-
xen/arch/x86/hvm/vpt.c | 2 +-
xen/arch/x86/i8259.c | 10 +-
xen/arch/x86/include/asm/asm-defns.h | 6 +
xen/arch/x86/include/asm/bug.h | 10 +-
xen/arch/x86/include/asm/cpufeature.h | 1 +
xen/arch/x86/include/asm/cpufeatures.h | 1 +
xen/arch/x86/include/asm/cpuidle.h | 4 +-
xen/arch/x86/include/asm/current.h | 6 +-
xen/arch/x86/include/asm/endbr.h | 55 +++
xen/arch/x86/include/asm/flushtlb.h | 2 +-
xen/arch/x86/include/asm/genapic.h | 18 +-
xen/arch/x86/include/asm/hpet.h | 8 +-
xen/arch/x86/include/asm/hvm/emulate.h | 8 +-
xen/arch/x86/include/asm/hvm/save.h | 2 +-
xen/arch/x86/include/asm/hvm/svm/nestedsvm.h | 18 +-
xen/arch/x86/include/asm/hvm/svm/svm.h | 1 -
xen/arch/x86/include/asm/hvm/vioapic.h | 2 +-
xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 8 +-
xen/arch/x86/include/asm/hvm/vmx/vmx.h | 4 +-
xen/arch/x86/include/asm/hvm/vmx/vvmx.h | 18 +-
xen/arch/x86/include/asm/hypercall.h | 81 +++--
xen/arch/x86/include/asm/irq.h | 24 +-
xen/arch/x86/include/asm/machine_kexec.h | 2 +-
xen/arch/x86/include/asm/mm.h | 16 +-
xen/arch/x86/include/asm/msi.h | 8 +-
xen/arch/x86/include/asm/msr-index.h | 1 +
xen/arch/x86/include/asm/mtrr.h | 2 +-
xen/arch/x86/include/asm/p2m.h | 4 +-
xen/arch/x86/include/asm/paging.h | 2 +-
xen/arch/x86/include/asm/processor.h | 4 +-
xen/arch/x86/include/asm/pv/domain.h | 4 +-
xen/arch/x86/include/asm/pv/shim.h | 11 +-
xen/arch/x86/include/asm/shadow.h | 2 +-
xen/arch/x86/include/asm/smp.h | 6 +-
xen/arch/x86/include/asm/tboot.h | 2 +-
xen/arch/x86/include/asm/time.h | 6 +-
xen/arch/x86/io_apic.c | 28 +-
xen/arch/x86/ioport_emulate.c | 4 +-
xen/arch/x86/irq.c | 28 +-
xen/arch/x86/livepatch.c | 2 +-
xen/arch/x86/machine_kexec.c | 2 +-
xen/arch/x86/mm.c | 35 +-
xen/arch/x86/mm/hap/guest_walk.c | 4 +-
xen/arch/x86/mm/hap/hap.c | 29 +-
xen/arch/x86/mm/hap/nested_hap.c | 2 +-
xen/arch/x86/mm/hap/private.h | 30 +-
xen/arch/x86/mm/mem_sharing.c | 2 +-
xen/arch/x86/mm/p2m-ept.c | 34 +-
xen/arch/x86/mm/p2m-pt.c | 19 +-
xen/arch/x86/mm/paging.c | 3 +-
xen/arch/x86/mm/shadow/common.c | 33 +-
xen/arch/x86/mm/shadow/hvm.c | 16 +-
xen/arch/x86/mm/shadow/multi.c | 80 +++--
xen/arch/x86/mm/shadow/multi.h | 20 +-
xen/arch/x86/mm/shadow/none.c | 20 +-
xen/arch/x86/mm/shadow/private.h | 12 +-
xen/arch/x86/mm/shadow/pv.c | 4 +-
xen/arch/x86/msi.c | 18 +-
xen/arch/x86/nmi.c | 16 +-
xen/arch/x86/numa.c | 10 +-
xen/arch/x86/oprofile/nmi_int.c | 16 +-
xen/arch/x86/oprofile/op_model_athlon.c | 18 +-
xen/arch/x86/oprofile/op_model_p4.c | 14 +-
xen/arch/x86/oprofile/op_model_ppro.c | 26 +-
xen/arch/x86/percpu.c | 6 +-
xen/arch/x86/physdev.c | 2 +-
xen/arch/x86/platform_hypercall.c | 11 +-
xen/arch/x86/psr.c | 41 +--
xen/arch/x86/pv/callback.c | 25 +-
xen/arch/x86/pv/descriptor-tables.c | 14 +-
xen/arch/x86/pv/domain.c | 12 +-
xen/arch/x86/pv/emul-gate-op.c | 9 +-
xen/arch/x86/pv/emul-priv-op.c | 71 ++--
xen/arch/x86/pv/emulate.h | 7 -
xen/arch/x86/pv/hypercall.c | 11 +-
xen/arch/x86/pv/iret.c | 4 +-
xen/arch/x86/pv/misc-hypercalls.c | 10 +-
xen/arch/x86/pv/ro-page-fault.c | 31 +-
xen/arch/x86/pv/shim.c | 60 ++--
xen/arch/x86/pv/traps.c | 2 +-
xen/arch/x86/setup.c | 80 ++++-
xen/arch/x86/shutdown.c | 10 +-
xen/arch/x86/smp.c | 20 +-
xen/arch/x86/smpboot.c | 2 +-
xen/arch/x86/spec_ctrl.c | 6 +-
xen/arch/x86/srat.c | 4 +-
xen/arch/x86/sysctl.c | 4 +-
xen/arch/x86/tboot.c | 2 +-
xen/arch/x86/time.c | 68 ++--
xen/arch/x86/traps.c | 8 +-
xen/arch/x86/tsx.c | 2 +-
xen/arch/x86/x86_64/acpi_mmcfg.c | 2 +-
xen/arch/x86/x86_64/compat.c | 1 -
xen/arch/x86/x86_64/compat/entry.S | 1 +
xen/arch/x86/x86_64/compat/mm.c | 7 +-
xen/arch/x86/x86_64/entry.S | 49 ++-
xen/arch/x86/x86_64/kexec_reloc.S | 23 +-
xen/arch/x86/x86_64/mmconfig-shared.c | 10 +-
xen/arch/x86/x86_64/mmconfig.h | 2 +-
xen/arch/x86/x86_64/platform_hypercall.c | 2 +-
xen/arch/x86/x86_64/traps.c | 42 ++-
xen/arch/x86/x86_emulate.c | 34 +-
xen/arch/x86/x86_emulate/x86_emulate.c | 10 +-
xen/arch/x86/x86_emulate/x86_emulate.h | 33 +-
xen/arch/x86/xen.lds.S | 3 +-
xen/common/argo.c | 6 +-
xen/common/bunzip2.c | 2 +-
xen/common/compat/domain.c | 3 +-
xen/common/compat/grant_table.c | 5 +-
xen/common/compat/kernel.c | 2 +-
xen/common/compat/memory.c | 7 +-
xen/common/compat/multicall.c | 3 +-
xen/common/core_parking.c | 10 +-
xen/common/coverage/gcov.c | 8 +-
xen/common/cpu.c | 4 +-
xen/common/debugtrace.c | 10 +-
xen/common/decompress.c | 2 +-
xen/common/dm.c | 6 +-
xen/common/domain.c | 15 +-
xen/common/domctl.c | 2 +-
xen/common/efi/boot.c | 6 +-
xen/common/efi/runtime.c | 18 +
xen/common/event_2l.c | 21 +-
xen/common/event_channel.c | 18 +-
xen/common/event_fifo.c | 30 +-
xen/common/gdbstub.c | 9 +-
xen/common/grant_table.c | 29 +-
xen/common/hypfs.c | 63 ++--
xen/common/irq.c | 6 +-
xen/common/kernel.c | 6 +-
xen/common/kexec.c | 18 +-
xen/common/keyhandler.c | 47 +--
xen/common/livepatch.c | 15 +-
xen/common/memory.c | 8 +-
xen/common/multicall.c | 2 +-
xen/common/page_alloc.c | 14 +-
xen/common/perfc.c | 4 +-
xen/common/radix-tree.c | 8 +-
xen/common/random.c | 2 +-
xen/common/rangeset.c | 2 +-
xen/common/rcupdate.c | 8 +-
xen/common/sched/arinc653.c | 20 +-
xen/common/sched/compat.c | 2 +-
xen/common/sched/core.c | 40 +--
xen/common/sched/cpupool.c | 35 +-
xen/common/sched/credit.c | 59 ++--
xen/common/sched/credit2.c | 55 ++-
xen/common/sched/null.c | 60 ++--
xen/common/sched/rt.c | 47 +--
xen/common/spinlock.c | 12 +-
xen/common/stop_machine.c | 6 +-
xen/common/sysctl.c | 2 +-
xen/common/tasklet.c | 4 +-
xen/common/timer.c | 6 +-
xen/common/trace.c | 4 +-
xen/common/unlzma.c | 2 +-
xen/common/vm_event.c | 6 +-
xen/common/xenoprof.c | 2 +-
xen/common/xmalloc_tlsf.c | 4 +-
xen/common/zstd/zstd_common.c | 4 +-
xen/common/zstd/zstd_internal.h | 4 +-
xen/drivers/acpi/apei/apei-base.c | 32 +-
xen/drivers/acpi/apei/apei-internal.h | 20 +-
xen/drivers/acpi/apei/erst.c | 57 ++-
xen/drivers/acpi/apei/hest.c | 4 +-
xen/drivers/acpi/numa.c | 10 +-
xen/drivers/acpi/tables.c | 2 +-
xen/drivers/char/console.c | 36 +-
xen/drivers/char/ehci-dbgp.c | 28 +-
xen/drivers/char/ns16550.c | 34 +-
xen/drivers/cpufreq/cpufreq.c | 6 +-
xen/drivers/cpufreq/cpufreq_misc_governors.c | 22 +-
xen/drivers/cpufreq/cpufreq_ondemand.c | 10 +-
xen/drivers/passthrough/amd/iommu.h | 45 +--
xen/drivers/passthrough/amd/iommu_acpi.c | 15 +-
xen/drivers/passthrough/amd/iommu_guest.c | 12 +-
xen/drivers/passthrough/amd/iommu_init.c | 49 +--
xen/drivers/passthrough/amd/iommu_intr.c | 20 +-
xen/drivers/passthrough/amd/iommu_map.c | 22 +-
xen/drivers/passthrough/amd/pci_amd_iommu.c | 32 +-
xen/drivers/passthrough/iommu.c | 56 ++-
xen/drivers/passthrough/pci.c | 18 +-
xen/drivers/passthrough/vtd/dmar.c | 7 +-
xen/drivers/passthrough/vtd/extern.h | 38 +-
xen/drivers/passthrough/vtd/intremap.c | 14 +-
xen/drivers/passthrough/vtd/iommu.c | 94 ++---
xen/drivers/passthrough/vtd/qinval.c | 28 +-
xen/drivers/passthrough/vtd/quirks.c | 2 +-
xen/drivers/passthrough/vtd/utils.c | 2 +-
xen/drivers/passthrough/vtd/x86/hvm.c | 4 +-
xen/drivers/passthrough/x86/hvm.c | 14 +-
xen/drivers/video/lfb.c | 4 +-
xen/drivers/video/lfb.h | 4 +-
xen/drivers/video/vesa.c | 6 +-
xen/drivers/video/vga.c | 6 +-
xen/drivers/vpci/header.c | 18 +-
xen/drivers/vpci/msi.c | 42 +--
xen/drivers/vpci/msix.c | 20 +-
xen/drivers/vpci/vpci.c | 16 +-
xen/include/acpi/cpufreq/cpufreq.h | 1 -
xen/include/xen/acpi.h | 2 +-
xen/include/xen/compiler.h | 6 +
xen/include/xen/domain.h | 2 +-
xen/include/xen/hypercall.h | 69 ++--
xen/include/xen/hypfs.h | 49 ++-
xen/include/xen/irq.h | 6 +-
xen/include/xen/lib.h | 2 +-
xen/include/xen/perfc.h | 4 +-
xen/include/xen/sched.h | 2 +-
xen/include/xen/sort.h | 55 ++-
xen/include/xen/spinlock.h | 4 +-
xen/include/xen/vpci.h | 8 +-
xen/include/xsm/dummy.h | 211 +++++------
xen/lib/sort.c | 80 +----
xen/tools/check-endbr.sh | 85 +++++
xen/xsm/flask/avc.c | 2 +-
xen/xsm/flask/flask_op.c | 8 +-
xen/xsm/flask/hooks.c | 236 +++++++------
xen/xsm/flask/private.h | 9 +
xen/xsm/flask/ss/avtab.c | 4 +-
xen/xsm/flask/ss/conditional.c | 10 +-
xen/xsm/flask/ss/conditional.h | 6 +-
xen/xsm/flask/ss/policydb.c | 53 +--
xen/xsm/flask/ss/services.c | 6 +-
xen/xsm/flask/ss/symtab.c | 5 +-
xen/xsm/silo.c | 24 +-
xen/xsm/xsm_core.c | 6 +-
322 files changed, 3316 insertions(+), 2739 deletions(-)
create mode 100644 automation/build/debian/buster-gcc-ibt.dockerfile
create mode 100644 xen/arch/x86/include/asm/endbr.h
create mode 100755 xen/tools/check-endbr.sh
create mode 100644 xen/xsm/flask/private.h
--
2.11.0
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:52 ` Julien Grall
2022-02-22 15:26 ` [PATCH v3 03/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
` (14 subsequent siblings)
15 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel
Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Volodymyr Babchuk,
Bertrand Marquis
There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
loop like this are problematic for performance, especially with Spectre v2
protections, which is why extern inline is used commonly by libraries.
Both ARM callers pass in NULL for the swap function, and while this might seem
like an attractive option at first, it causes generic_swap() to be used, which
forced a byte-wise copy. Provide real swap functions so the compiler can
optimise properly, which is very important for ARM downstreams where
milliseconds until the system is up matters.
This is also important for Control Flow Integrity schemes (e.g. x86 CET-IBT,
ARM BTI), because tagged function(s) performing an arbitrary length swap of
two arbitrary pointers is a very valuable gadget for an attacker.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
v2:
* Adjust commit message
v3:
* Adjust comment
* Adjust commit message again
---
xen/arch/arm/bootfdt.c | 9 +++++-
xen/arch/arm/io.c | 9 +++++-
xen/include/xen/sort.h | 55 +++++++++++++++++++++++++++++++++-
xen/lib/sort.c | 80 ++------------------------------------------------
4 files changed, 72 insertions(+), 81 deletions(-)
diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index afaa0e249b71..e318ef960386 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -448,6 +448,13 @@ static int __init cmp_memory_node(const void *key, const void *elem)
return 0;
}
+static void __init swap_memory_node(void *_a, void *_b, size_t size)
+{
+ struct membank *a = _a, *b = _b;
+
+ SWAP(*a, *b);
+}
+
/**
* boot_fdt_info - initialize bootinfo from a DTB
* @fdt: flattened device tree binary
@@ -472,7 +479,7 @@ size_t __init boot_fdt_info(const void *fdt, paddr_t paddr)
* the banks sorted in ascending order. So sort them through.
*/
sort(bootinfo.mem.bank, bootinfo.mem.nr_banks, sizeof(struct membank),
- cmp_memory_node, NULL);
+ cmp_memory_node, swap_memory_node);
early_print_info();
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index 729287e37c59..1a066f9ae502 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -80,6 +80,13 @@ static int cmp_mmio_handler(const void *key, const void *elem)
return 0;
}
+static void swap_mmio_handler(void *_a, void *_b, size_t size)
+{
+ struct mmio_handler *a = _a, *b = _b;
+
+ SWAP(*a, *b);
+}
+
static const struct mmio_handler *find_mmio_handler(struct domain *d,
paddr_t gpa)
{
@@ -170,7 +177,7 @@ void register_mmio_handler(struct domain *d,
/* Sort mmio handlers in ascending order based on base address */
sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
- cmp_mmio_handler, NULL);
+ cmp_mmio_handler, swap_mmio_handler);
write_unlock(&vmmio->lock);
}
diff --git a/xen/include/xen/sort.h b/xen/include/xen/sort.h
index a403652948e7..2f52ff85b9e4 100644
--- a/xen/include/xen/sort.h
+++ b/xen/include/xen/sort.h
@@ -3,8 +3,61 @@
#include <xen/types.h>
+/*
+ * sort - sort an array of elements
+ * @base: pointer to data to sort
+ * @num: number of elements
+ * @size: size of each element
+ * @cmp: pointer to comparison function
+ * @swap: pointer to swap function
+ *
+ * This function does a heapsort on the given array. You may provide a
+ * swap function optimized to your element type.
+ *
+ * Sorting time is O(n log n) both on average and worst-case. While
+ * qsort is about 20% faster on average, it suffers from exploitable
+ * O(n*n) worst-case behavior and extra memory requirements that make
+ * it less suitable for kernel use.
+ */
+#ifndef SORT_IMPLEMENTATION
+extern gnu_inline
+#endif
void sort(void *base, size_t num, size_t size,
int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, size_t));
+ void (*swap)(void *, void *, size_t))
+{
+ /* pre-scale counters for performance */
+ size_t i = (num / 2) * size, n = num * size, c, r;
+
+ /* heapify */
+ while ( i > 0 )
+ {
+ for ( r = i -= size; r * 2 + size < n; r = c )
+ {
+ c = r * 2 + size;
+ if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
+ c += size;
+ if ( cmp(base + r, base + c) >= 0 )
+ break;
+ swap(base + r, base + c, size);
+ }
+ }
+
+ /* sort */
+ for ( i = n; i > 0; )
+ {
+ i -= size;
+ swap(base, base + i, size);
+ for ( r = 0; r * 2 + size < i; r = c )
+ {
+ c = r * 2 + size;
+ if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
+ c += size;
+ if ( cmp(base + r, base + c) >= 0 )
+ break;
+ swap(base + r, base + c, size);
+ }
+ }
+}
#endif /* __XEN_SORT_H__ */
diff --git a/xen/lib/sort.c b/xen/lib/sort.c
index 35ce0d7abdec..b7e78cc0e8d2 100644
--- a/xen/lib/sort.c
+++ b/xen/lib/sort.c
@@ -4,81 +4,5 @@
* Jan 23 2005 Matt Mackall <mpm@selenic.com>
*/
-#include <xen/types.h>
-
-static void u32_swap(void *a, void *b, size_t size)
-{
- uint32_t t = *(uint32_t *)a;
-
- *(uint32_t *)a = *(uint32_t *)b;
- *(uint32_t *)b = t;
-}
-
-static void generic_swap(void *a, void *b, size_t size)
-{
- char t;
-
- do {
- t = *(char *)a;
- *(char *)a++ = *(char *)b;
- *(char *)b++ = t;
- } while ( --size > 0 );
-}
-
-/*
- * sort - sort an array of elements
- * @base: pointer to data to sort
- * @num: number of elements
- * @size: size of each element
- * @cmp: pointer to comparison function
- * @swap: pointer to swap function or NULL
- *
- * This function does a heapsort on the given array. You may provide a
- * swap function optimized to your element type.
- *
- * Sorting time is O(n log n) both on average and worst-case. While
- * qsort is about 20% faster on average, it suffers from exploitable
- * O(n*n) worst-case behavior and extra memory requirements that make
- * it less suitable for kernel use.
- */
-
-void sort(void *base, size_t num, size_t size,
- int (*cmp)(const void *, const void *),
- void (*swap)(void *, void *, size_t size))
-{
- /* pre-scale counters for performance */
- size_t i = (num / 2) * size, n = num * size, c, r;
-
- if ( !swap )
- swap = (size == 4 ? u32_swap : generic_swap);
-
- /* heapify */
- while ( i > 0 )
- {
- for ( r = i -= size; r * 2 + size < n; r = c )
- {
- c = r * 2 + size;
- if ( (c < n - size) && (cmp(base + c, base + c + size) < 0) )
- c += size;
- if ( cmp(base + r, base + c) >= 0 )
- break;
- swap(base + r, base + c, size);
- }
- }
-
- /* sort */
- for ( i = n; i > 0; )
- {
- i -= size;
- swap(base, base + i, size);
- for ( r = 0; r * 2 + size < i; r = c )
- {
- c = r * 2 + size;
- if ( (c < i - size) && (cmp(base + c, base + c + size) < 0) )
- c += size;
- if ( cmp(base + r, base + c) >= 0 )
- break;
- swap(base + r, base + c, size);
- }
- }
-}
+#define SORT_IMPLEMENTATION
+#include <xen/sort.h>
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 03/70] x86/pv-shim: Don't modify the hypercall table
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 16:24 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 05/70] x86/kexec: Annotate embedded data with ELF metadata Andrew Cooper
` (13 subsequent siblings)
15 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Juergen Gross, Andrew Cooper
From: Juergen Gross <jgross@suse.com>
When running as pv-shim the hypercall is modified today in order to
replace the functions for __HYPERVISOR_event_channel_op and
__HYPERVISOR_grant_table_op hypercalls.
Change this to call the related functions from the normal handlers
instead when running as shim. The performance implications are not
really relevant, as a normal production hypervisor will not be
configured to support shim mode, so the related calls will be dropped
due to optimisation of the compiler.
Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
wrapper do_grant_table_op() needed, as in this case grant_table.c
isn't being built.
Signed-off-by: Juergen Gross <jgross@suse.com>
[Split out of series. Make compile in isolation]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
v3:
* Identify that it is modified as a side effect of pulling it out of the
middle of Juergen's series.
---
xen/arch/x86/include/asm/hypercall.h | 4 ++-
xen/arch/x86/include/asm/pv/shim.h | 3 ++
xen/arch/x86/pv/hypercall.c | 2 +-
xen/arch/x86/pv/shim.c | 54 ++++++++++++++++----------------
xen/arch/x86/x86_64/platform_hypercall.c | 2 +-
xen/common/compat/multicall.c | 3 +-
xen/common/event_channel.c | 9 ++++++
xen/common/grant_table.c | 9 ++++++
8 files changed, 54 insertions(+), 32 deletions(-)
diff --git a/xen/arch/x86/include/asm/hypercall.h b/xen/arch/x86/include/asm/hypercall.h
index 5d394d492318..f004824f16b6 100644
--- a/xen/arch/x86/include/asm/hypercall.h
+++ b/xen/arch/x86/include/asm/hypercall.h
@@ -145,6 +145,7 @@ do_set_segment_base(
#include <compat/arch-x86/xen.h>
#include <compat/physdev.h>
+#include <compat/platform.h>
extern int
compat_physdev_op(
@@ -161,8 +162,9 @@ extern int compat_mmuext_op(
XEN_GUEST_HANDLE_PARAM(uint) pdone,
unsigned int foreigndom);
+DEFINE_XEN_GUEST_HANDLE(compat_platform_op_t);
extern int compat_platform_op(
- XEN_GUEST_HANDLE_PARAM(void) u_xenpf_op);
+ XEN_GUEST_HANDLE_PARAM(compat_platform_op_t) u_xenpf_op);
extern long compat_callback_op(
int cmd, XEN_GUEST_HANDLE(void) arg);
diff --git a/xen/arch/x86/include/asm/pv/shim.h b/xen/arch/x86/include/asm/pv/shim.h
index 8a91f4f9dfbf..6415f8068e5c 100644
--- a/xen/arch/x86/include/asm/pv/shim.h
+++ b/xen/arch/x86/include/asm/pv/shim.h
@@ -19,6 +19,7 @@
#ifndef __X86_PV_SHIM_H__
#define __X86_PV_SHIM_H__
+#include <xen/hypercall.h>
#include <xen/types.h>
#if defined(CONFIG_PV_SHIM_EXCLUSIVE)
@@ -45,6 +46,8 @@ domid_t get_initial_domain_id(void);
uint64_t pv_shim_mem(uint64_t avail);
void pv_shim_fixup_e820(struct e820map *e820);
const struct platform_bad_page *pv_shim_reserved_pages(unsigned int *size);
+typeof(do_event_channel_op) pv_shim_event_channel_op;
+typeof(do_grant_table_op) pv_shim_grant_table_op;
#else
diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
index ecdd58deea69..50cd219c18fc 100644
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -64,7 +64,7 @@ const pv_hypercall_table_t pv_hypercall_table[] = {
COMPAT_CALL(xen_version),
HYPERCALL(console_io),
COMPAT_CALL(physdev_op_compat),
-#ifdef CONFIG_GRANT_TABLE
+#if defined(CONFIG_GRANT_TABLE) || defined(CONFIG_PV_SHIM)
COMPAT_CALL(grant_table_op),
#endif
HYPERCALL(vm_assist),
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index d9704121a739..7e891fe2f7a4 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -56,11 +56,6 @@ static DEFINE_SPINLOCK(balloon_lock);
static struct platform_bad_page __initdata reserved_pages[2];
-static long pv_shim_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg);
-static long pv_shim_grant_table_op(unsigned int cmd,
- XEN_GUEST_HANDLE_PARAM(void) uop,
- unsigned int count);
-
/*
* By default give the shim 1MB of free memory slack. Some users may wish to
* tune this constants for better memory utilization. This can be achieved
@@ -203,7 +198,6 @@ void __init pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start,
start_info_t *si)
{
bool compat = is_pv_32bit_domain(d);
- pv_hypercall_table_t *rw_pv_hypercall_table;
uint64_t param = 0;
long rc;
@@ -249,23 +243,6 @@ void __init pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start,
consoled_set_ring_addr(page);
}
- /*
- * Locate pv_hypercall_table[] (usually .rodata) in the directmap (which
- * is writeable) and insert some shim-specific hypercall handlers.
- */
- rw_pv_hypercall_table = __va(__pa(pv_hypercall_table));
- rw_pv_hypercall_table[__HYPERVISOR_event_channel_op].native =
- (hypercall_fn_t *)pv_shim_event_channel_op;
- rw_pv_hypercall_table[__HYPERVISOR_grant_table_op].native =
- (hypercall_fn_t *)pv_shim_grant_table_op;
-
-#ifdef CONFIG_PV32
- rw_pv_hypercall_table[__HYPERVISOR_event_channel_op].compat =
- (hypercall_fn_t *)pv_shim_event_channel_op;
- rw_pv_hypercall_table[__HYPERVISOR_grant_table_op].compat =
- (hypercall_fn_t *)pv_shim_grant_table_op;
-#endif
-
guest = d;
/*
@@ -435,7 +412,7 @@ int pv_shim_shutdown(uint8_t reason)
return 0;
}
-static long pv_shim_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
+long pv_shim_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
struct domain *d = current->domain;
struct evtchn_close close;
@@ -683,9 +660,9 @@ void pv_shim_inject_evtchn(unsigned int port)
# define compat_handle_okay guest_handle_okay
#endif
-static long pv_shim_grant_table_op(unsigned int cmd,
- XEN_GUEST_HANDLE_PARAM(void) uop,
- unsigned int count)
+long pv_shim_grant_table_op(unsigned int cmd,
+ XEN_GUEST_HANDLE_PARAM(void) uop,
+ unsigned int count)
{
struct domain *d = current->domain;
long rc = 0;
@@ -845,6 +822,29 @@ static long pv_shim_grant_table_op(unsigned int cmd,
return rc;
}
+#ifndef CONFIG_GRANT_TABLE
+/* Thin wrapper(s) needed. */
+long do_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
+ unsigned int count)
+{
+ if ( !pv_shim )
+ return -ENOSYS;
+
+ return pv_shim_grant_table_op(cmd, uop, count);
+}
+
+#ifdef CONFIG_PV32
+int compat_grant_table_op(unsigned int cmd, XEN_GUEST_HANDLE_PARAM(void) uop,
+ unsigned int count)
+{
+ if ( !pv_shim )
+ return -ENOSYS;
+
+ return pv_shim_grant_table_op(cmd, uop, count);
+}
+#endif
+#endif
+
long pv_shim_cpu_up(void *data)
{
struct vcpu *v = data;
diff --git a/xen/arch/x86/x86_64/platform_hypercall.c b/xen/arch/x86/x86_64/platform_hypercall.c
index fbba893a47cb..966fd27b5f22 100644
--- a/xen/arch/x86/x86_64/platform_hypercall.c
+++ b/xen/arch/x86/x86_64/platform_hypercall.c
@@ -6,8 +6,8 @@ EMIT_FILE;
#include <xen/lib.h>
#include <compat/platform.h>
+#include <xen/hypercall.h>
-DEFINE_XEN_GUEST_HANDLE(compat_platform_op_t);
#define xen_platform_op compat_platform_op
#define xen_platform_op_t compat_platform_op_t
#define do_platform_op(x) compat_platform_op(_##x)
diff --git a/xen/common/compat/multicall.c b/xen/common/compat/multicall.c
index a0e9918f4805..b17739d21829 100644
--- a/xen/common/compat/multicall.c
+++ b/xen/common/compat/multicall.c
@@ -5,7 +5,7 @@
EMIT_FILE;
#include <xen/types.h>
-#include <xen/multicall.h>
+#include <xen/hypercall.h>
#include <xen/trace.h>
#define COMPAT
@@ -19,7 +19,6 @@ static inline void xlat_multicall_entry(struct mc_state *mcs)
mcs->compat_call.args[i] = mcs->call.args[i];
}
-DEFINE_XEN_GUEST_HANDLE(multicall_entry_compat_t);
#define multicall_entry compat_multicall_entry
#define multicall_entry_t multicall_entry_compat_t
#define do_multicall_call compat_multicall_call
diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index da88ad141a69..c9912122d1e5 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -31,6 +31,10 @@
#include <public/event_channel.h>
#include <xsm/xsm.h>
+#ifdef CONFIG_PV_SHIM
+#include <asm/guest.h>
+#endif
+
#define ERROR_EXIT(_errno) \
do { \
gdprintk(XENLOG_WARNING, \
@@ -1189,6 +1193,11 @@ long do_event_channel_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int rc;
+#ifdef CONFIG_PV_SHIM
+ if ( unlikely(pv_shim) )
+ return pv_shim_event_channel_op(cmd, arg);
+#endif
+
switch ( cmd )
{
case EVTCHNOP_alloc_unbound: {
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 3d92fee59285..925ed7d6bee2 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -44,6 +44,10 @@
#include <asm/flushtlb.h>
#include <asm/guest_atomics.h>
+#ifdef CONFIG_PV_SHIM
+#include <asm/guest.h>
+#endif
+
/* Per-domain grant information. */
struct grant_table {
/*
@@ -3561,6 +3565,11 @@ do_grant_table_op(
long rc;
unsigned int opaque_in = cmd & GNTTABOP_ARG_MASK, opaque_out = 0;
+#ifdef CONFIG_PV_SHIM
+ if ( unlikely(pv_shim) )
+ return pv_shim_grant_table_op(cmd, uop, count);
+#endif
+
if ( (int)count < 0 )
return -EINVAL;
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 05/70] x86/kexec: Annotate embedded data with ELF metadata
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 03/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 06/70] x86: Introduce support for CET-IBT Andrew Cooper
` (12 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
Scanning for embedded endbranch instructions involves parsing the .text
disassembly. Data in the kexec trampoline has no ELF metadata, so objdump
treats it as instructions and tries to disassemble. Convert:
ffff82d040396108 <compatibility_mode_far>:
ffff82d040396108: 00 00 add %al,(%rax)
ffff82d04039610a: 00 00 add %al,(%rax)
ffff82d04039610c: 10 00 adc %al,(%rax)
ffff82d04039610e <compat_mode_gdt_desc>:
ffff82d04039610e: 17 (bad)
...
ffff82d040396118 <compat_mode_gdt>:
...
ffff82d040396120: ff (bad)
ffff82d040396121: ff 00 incl (%rax)
ffff82d040396123: 00 00 add %al,(%rax)
ffff82d040396125: 93 xchg %eax,%ebx
ffff82d040396126: cf iret
ffff82d040396127: 00 ff add %bh,%bh
ffff82d040396129: ff 00 incl (%rax)
ffff82d04039612b: 00 00 add %al,(%rax)
ffff82d04039612d: 9b fwait
ffff82d04039612e: cf iret
...
ffff82d040396130 <compat_mode_idt>:
...
ffff82d0403961b6 <kexec_reloc_size>:
ffff82d0403961b6: b6 01 mov $0x1,%dh
...
to:
ffff82d040396108 <compatibility_mode_far>:
ffff82d040396108: 00 00 00 00 10 00 ......
ffff82d04039610e <compat_mode_gdt_desc>:
ffff82d04039610e: 17 00 00 00 00 00 00 00 00 00 ..........
ffff82d040396118 <compat_mode_gdt>:
...
ffff82d040396120: ff ff 00 00 00 93 cf 00 ff ff 00 00 00 9b cf 00 ................
ffff82d040396130 <compat_mode_idt>:
ffff82d040396130: 00 00 00 00 00 00 ......
ffff82d040396136 <reloc_stack>:
...
Most data just gains type and size metadata.
The reloc_stack label is the wrong end of the data block to have a size, so
move it to the lowest address and introduce .Lreloc_stack_base as a
replacement. Also, fix the fact that it is misaligned by 2 bytes.
While kexec_reloc_size could gain metadata, it's use in the linker
assertion (while correct) is deeply confusing to follow. Drop it entirely,
using a linker symbol instead to denote the end of the trampoline.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
The remainder of the 32bit code has mode-invariant lengths, so disassembles
safely as 64bit. The only differences come from 32/64bit implicit register
sizes.
v2.1:
* New
v2.2:
* Fix stack alignment
---
xen/arch/x86/include/asm/machine_kexec.h | 2 +-
xen/arch/x86/machine_kexec.c | 2 +-
xen/arch/x86/x86_64/kexec_reloc.S | 23 ++++++++++++++++++-----
xen/arch/x86/xen.lds.S | 3 ++-
4 files changed, 22 insertions(+), 8 deletions(-)
diff --git a/xen/arch/x86/include/asm/machine_kexec.h b/xen/arch/x86/include/asm/machine_kexec.h
index ba0d469d077b..d4880818c1d9 100644
--- a/xen/arch/x86/include/asm/machine_kexec.h
+++ b/xen/arch/x86/include/asm/machine_kexec.h
@@ -9,7 +9,7 @@ extern void kexec_reloc(unsigned long reloc_code, unsigned long reloc_pt,
unsigned long ind_maddr, unsigned long entry_maddr,
unsigned long flags);
-extern unsigned int kexec_reloc_size;
+extern const char kexec_reloc_end[];
#endif
diff --git a/xen/arch/x86/machine_kexec.c b/xen/arch/x86/machine_kexec.c
index 08ec9fd43b1d..751a9efcaf6a 100644
--- a/xen/arch/x86/machine_kexec.c
+++ b/xen/arch/x86/machine_kexec.c
@@ -117,7 +117,7 @@ int machine_kexec_load(struct kexec_image *image)
}
code_page = __map_domain_page(image->control_code_page);
- memcpy(code_page, kexec_reloc, kexec_reloc_size);
+ memcpy(code_page, kexec_reloc, kexec_reloc_end - (char *)kexec_reloc);
unmap_domain_page(code_page);
/*
diff --git a/xen/arch/x86/x86_64/kexec_reloc.S b/xen/arch/x86/x86_64/kexec_reloc.S
index d488d127cfb9..89316bc3a7ac 100644
--- a/xen/arch/x86/x86_64/kexec_reloc.S
+++ b/xen/arch/x86/x86_64/kexec_reloc.S
@@ -34,7 +34,7 @@ ENTRY(kexec_reloc)
movq %rcx, %rbp
/* Setup stack. */
- leaq (reloc_stack - kexec_reloc)(%rdi), %rsp
+ leaq (.Lreloc_stack_base - kexec_reloc)(%rdi), %rsp
/* Load reloc page table. */
movq %rsi, %cr3
@@ -175,10 +175,16 @@ compatibility_mode_far:
.long 0x00000000 /* set in call_32_bit above */
.word 0x0010
+ .type compatibility_mode_far, @object
+ .size compatibility_mode_far, . - compatibility_mode_far
+
compat_mode_gdt_desc:
.word .Lcompat_mode_gdt_end - compat_mode_gdt -1
.quad 0x0000000000000000 /* set in call_32_bit above */
+ .type compat_mode_gdt_desc, @object
+ .size compat_mode_gdt_desc, . - compat_mode_gdt_desc
+
.align 8
compat_mode_gdt:
.quad 0x0000000000000000 /* null */
@@ -186,16 +192,23 @@ compat_mode_gdt:
.quad 0x00cf9b000000ffff /* 0x0010 ring 0 code, compatibility */
.Lcompat_mode_gdt_end:
+ .type compat_mode_gdt, @object
+ .size compat_mode_gdt, . - compat_mode_gdt
+
compat_mode_idt:
.word 0 /* limit */
.long 0 /* base */
+ .type compat_mode_idt, @object
+ .size compat_mode_idt, . - compat_mode_idt
+
/*
* 16 words of stack are more than enough.
*/
- .fill 16,8,0
+ .align 8
reloc_stack:
+ .fill 16,8,0
+.Lreloc_stack_base:
- .globl kexec_reloc_size
-kexec_reloc_size:
- .long . - kexec_reloc
+ .type reloc_stack, @object
+ .size reloc_stack, . - reloc_stack
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index 82ad8feb6e99..7ffecd463070 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -84,6 +84,7 @@ SECTIONS
_etextentry = .;
*(.text.kexec) /* Page aligned in the object file. */
+ kexec_reloc_end = .;
*(.text.cold)
*(.text.unlikely)
@@ -428,7 +429,7 @@ ASSERT(__2M_rwdata_end <= XEN_VIRT_END - XEN_VIRT_START + __XEN_VIRT_START -
"Xen image overlaps stubs area")
#ifdef CONFIG_KEXEC
-ASSERT(kexec_reloc_size - kexec_reloc <= PAGE_SIZE, "kexec_reloc is too large")
+ASSERT(kexec_reloc_end - kexec_reloc <= PAGE_SIZE, "kexec_reloc is too large")
#endif
/* The Multiboot setup paths relies on this to simplify superpage PTE creation. */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 06/70] x86: Introduce support for CET-IBT
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (2 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 05/70] x86/kexec: Annotate embedded data with ELF metadata Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-28 12:58 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 08/70] xen: CFI hardening for custom_param() Andrew Cooper
` (11 subsequent siblings)
15 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
CET Indirect Branch Tracking is a hardware feature designed to provide
forward-edge control flow integrity, protecting against jump/call oriented
programming.
IBT requires the placement of ENDBR{32,64} instructions at the target of every
indirect call/jmp, and every entrypoint.
However, the default -fcf-protection=branch places an ENDBR on every function
which far more than necessary, and reduces the quantity of protection
afforded. Therefore, we use manual placement using the cf_check attribute.
It is necessary to check for both compiler and assembler support, as the
notrack prefix can be emitted in certain cases.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
Clang/LLVM support for -mmanual-endbr is in progress:
https://reviews.llvm.org/D118355
v2:
* Correct CONFIG_HAS_CC_CET_IBT to CONFIG_XEN_IBT in some places
* Move cf_check compatibility into tools/tests/x86_emulator/x86-emulate.h
v3:
* Extend commit message
* Disable IBT for pvshim defconfig
---
Config.mk | 1 -
tools/firmware/Makefile | 2 ++
tools/libs/guest/xg_dom_decompress_unsafe.h | 2 ++
tools/tests/x86_emulator/x86-emulate.h | 2 ++
xen/arch/x86/Kconfig | 17 +++++++++++++++++
xen/arch/x86/arch.mk | 6 ++++++
xen/arch/x86/configs/pvshim_defconfig | 1 +
xen/arch/x86/include/asm/asm-defns.h | 6 ++++++
xen/arch/x86/include/asm/cpufeature.h | 1 +
xen/arch/x86/include/asm/cpufeatures.h | 1 +
xen/include/xen/compiler.h | 6 ++++++
11 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/Config.mk b/Config.mk
index 95c053212ec3..f56f7dc33468 100644
--- a/Config.mk
+++ b/Config.mk
@@ -190,7 +190,6 @@ APPEND_CFLAGS += $(foreach i, $(APPEND_INCLUDES), -I$(i))
EMBEDDED_EXTRA_CFLAGS := -nopie -fno-stack-protector -fno-stack-protector-all
EMBEDDED_EXTRA_CFLAGS += -fno-exceptions -fno-asynchronous-unwind-tables
-EMBEDDED_EXTRA_CFLAGS += -fcf-protection=none
XEN_EXTFILES_URL ?= http://xenbits.xen.org/xen-extfiles
# All the files at that location were downloaded from elsewhere on
diff --git a/tools/firmware/Makefile b/tools/firmware/Makefile
index 345037b93b7f..53ed4f161edb 100644
--- a/tools/firmware/Makefile
+++ b/tools/firmware/Makefile
@@ -6,6 +6,8 @@ TARGET := hvmloader/hvmloader
INST_DIR := $(DESTDIR)$(XENFIRMWAREDIR)
DEBG_DIR := $(DESTDIR)$(DEBUG_DIR)$(XENFIRMWAREDIR)
+EMBEDDED_EXTRA_CFLAGS += -fcf-protection=none
+
SUBDIRS-y :=
SUBDIRS-$(CONFIG_OVMF) += ovmf-dir
SUBDIRS-$(CONFIG_SEABIOS) += seabios-dir
diff --git a/tools/libs/guest/xg_dom_decompress_unsafe.h b/tools/libs/guest/xg_dom_decompress_unsafe.h
index 4e0bf23aa587..ac6b94288d5e 100644
--- a/tools/libs/guest/xg_dom_decompress_unsafe.h
+++ b/tools/libs/guest/xg_dom_decompress_unsafe.h
@@ -8,6 +8,8 @@ typedef int decompress_fn(unsigned char *inbuf, unsigned int len,
void (*error)(const char *x));
#endif
+#define cf_check /* No Control Flow Integriy checking */
+
int xc_dom_decompress_unsafe(
decompress_fn fn, struct xc_dom_image *dom, void **blob, size_t *size)
__attribute__((visibility("internal")));
diff --git a/tools/tests/x86_emulator/x86-emulate.h b/tools/tests/x86_emulator/x86-emulate.h
index 7f60ef9e89ba..18ae40d01712 100644
--- a/tools/tests/x86_emulator/x86-emulate.h
+++ b/tools/tests/x86_emulator/x86-emulate.h
@@ -54,6 +54,8 @@
#define likely(x) __builtin_expect(!!(x), true)
#define unlikely(x) __builtin_expect(!!(x), false)
+#define cf_check /* No Control Flow Integriy checking */
+
#define container_of(ptr, type, member) ({ \
typeof(((type *)0)->member) *mptr__ = (ptr); \
(type *)((char *)mptr__ - offsetof(type, member)); \
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 41198b0f96ed..8e70f9a44847 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -40,6 +40,11 @@ config HAS_AS_CET_SS
# binutils >= 2.29 or LLVM >= 6
def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy)
+config HAS_CC_CET_IBT
+ # GCC >= 9 and binutils >= 2.29
+ # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
+ def_bool $(cc-option,-fcf-protection=branch -mmanual-endbr -mindirect-branch=thunk-extern) && $(as-instr,endbr64)
+
menu "Architecture Features"
source "arch/Kconfig"
@@ -125,6 +130,18 @@ config XEN_SHSTK
When CET-SS is active, 32bit PV guests cannot be used. Backwards
compatiblity can be provided via the PV Shim mechanism.
+config XEN_IBT
+ bool "Supervisor Indirect Branch Tracking"
+ depends on HAS_CC_CET_IBT
+ default y
+ help
+ Control-flow Enforcement Technology (CET) is a set of features in
+ hardware designed to combat Return-oriented Programming (ROP, also
+ call/jump COP/JOP) attacks. Indirect Branch Tracking is one CET
+ feature designed to provide function pointer protection.
+
+ This option arranges for Xen to use CET-IBT for its own protection.
+
config SHADOW_PAGING
bool "Shadow Paging"
default !PV_SHIM_EXCLUSIVE
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index edfc043dbbaf..f780c912a9cf 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -52,6 +52,12 @@ CFLAGS-$(CONFIG_CC_IS_GCC) += -fno-jump-tables
CFLAGS-$(CONFIG_CC_IS_CLANG) += -mretpoline-external-thunk
endif
+ifdef CONFIG_XEN_IBT
+CFLAGS += -fcf-protection=branch -mmanual-endbr
+else
+$(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
+endif
+
# If supported by the compiler, reduce stack alignment to 8 bytes. But allow
# this to be overridden elsewhere.
$(call cc-option-add,CFLAGS_stack_boundary,CC,-mpreferred-stack-boundary=3)
diff --git a/xen/arch/x86/configs/pvshim_defconfig b/xen/arch/x86/configs/pvshim_defconfig
index 787376df5a27..d0e92c2ded1f 100644
--- a/xen/arch/x86/configs/pvshim_defconfig
+++ b/xen/arch/x86/configs/pvshim_defconfig
@@ -8,6 +8,7 @@ CONFIG_NR_CPUS=32
CONFIG_EXPERT=y
# Disable features not used by the PV shim
# CONFIG_XEN_SHSTK is not set
+# CONFIG_XEN_IBT is not set
# CONFIG_GRANT_TABLE is not set
# CONFIG_HYPFS is not set
# CONFIG_BIGMEM is not set
diff --git a/xen/arch/x86/include/asm/asm-defns.h b/xen/arch/x86/include/asm/asm-defns.h
index 505f39ad5f76..8bd9007731d5 100644
--- a/xen/arch/x86/include/asm/asm-defns.h
+++ b/xen/arch/x86/include/asm/asm-defns.h
@@ -57,6 +57,12 @@
INDIRECT_BRANCH jmp \arg
.endm
+#ifdef CONFIG_XEN_IBT
+# define ENDBR64 endbr64
+#else
+# define ENDBR64
+#endif
+
.macro guest_access_mask_ptr ptr:req, scratch1:req, scratch2:req
#if defined(CONFIG_SPECULATIVE_HARDEN_GUEST_ACCESS)
/*
diff --git a/xen/arch/x86/include/asm/cpufeature.h b/xen/arch/x86/include/asm/cpufeature.h
index a0ab6d7d78ea..f2c6f255ace9 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -152,6 +152,7 @@
#define cpu_has_nscb boot_cpu_has(X86_FEATURE_NSCB)
#define cpu_has_xen_lbr boot_cpu_has(X86_FEATURE_XEN_LBR)
#define cpu_has_xen_shstk boot_cpu_has(X86_FEATURE_XEN_SHSTK)
+#define cpu_has_xen_ibt boot_cpu_has(X86_FEATURE_XEN_IBT)
#define cpu_has_msr_tsc_aux (cpu_has_rdtscp || cpu_has_rdpid)
diff --git a/xen/arch/x86/include/asm/cpufeatures.h b/xen/arch/x86/include/asm/cpufeatures.h
index b10154fc44bb..7413febd7ad8 100644
--- a/xen/arch/x86/include/asm/cpufeatures.h
+++ b/xen/arch/x86/include/asm/cpufeatures.h
@@ -39,6 +39,7 @@ XEN_CPUFEATURE(SC_VERW_PV, X86_SYNTH(23)) /* VERW used by Xen for PV */
XEN_CPUFEATURE(SC_VERW_HVM, X86_SYNTH(24)) /* VERW used by Xen for HVM */
XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */
XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */
+XEN_CPUFEATURE(XEN_IBT, X86_SYNTH(27)) /* Xen uses CET Indirect Branch Tracking */
/* Bug words follow the synthetic words. */
#define X86_NR_BUG 1
diff --git a/xen/include/xen/compiler.h b/xen/include/xen/compiler.h
index 696c7eb89e4c..933aec09a92d 100644
--- a/xen/include/xen/compiler.h
+++ b/xen/include/xen/compiler.h
@@ -37,6 +37,12 @@
# define nocall
#endif
+#ifdef CONFIG_XEN_IBT
+# define cf_check __attribute__((__cf_check__))
+#else
+# define cf_check
+#endif
+
#if (!defined(__clang__) && (__GNUC__ == 4) && (__GNUC_MINOR__ < 5))
#define unreachable() do {} while (1)
#else
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 08/70] xen: CFI hardening for custom_param()
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (3 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 06/70] x86: Introduce support for CET-IBT Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 12/70] xen: CFI hardening for continue_hypercall_on_cpu() Andrew Cooper
` (10 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
The "watchdog_timeout" and "cpu_type" handlers were missing __init.
The "numa", "acpi", "irq_vector_map" and "flask" handlers can skip forward
declarations by altering the custom_param() position.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3:
* Rebase over recent "vesa" changes.
---
xen/arch/x86/acpi/cpu_idle.c | 2 +-
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/apic.c | 4 ++--
xen/arch/x86/cpu/mcheck/mce.c | 2 +-
xen/arch/x86/cpu/microcode/core.c | 2 +-
xen/arch/x86/cpu/vpmu.c | 2 +-
xen/arch/x86/cpuid.c | 8 ++++----
xen/arch/x86/dom0_build.c | 8 ++++----
xen/arch/x86/genapic/probe.c | 2 +-
xen/arch/x86/hpet.c | 2 +-
xen/arch/x86/hvm/viridian/viridian.c | 2 +-
xen/arch/x86/hvm/vmx/vmcs.c | 8 ++++----
xen/arch/x86/io_apic.c | 2 +-
xen/arch/x86/irq.c | 6 ++----
xen/arch/x86/mm.c | 2 +-
xen/arch/x86/nmi.c | 4 ++--
xen/arch/x86/numa.c | 6 ++----
xen/arch/x86/oprofile/nmi_int.c | 2 +-
xen/arch/x86/psr.c | 2 +-
xen/arch/x86/pv/domain.c | 8 ++++----
xen/arch/x86/pv/shim.c | 2 +-
xen/arch/x86/setup.c | 11 +++++------
xen/arch/x86/shutdown.c | 2 +-
xen/arch/x86/spec_ctrl.c | 6 +++---
xen/arch/x86/time.c | 2 +-
xen/arch/x86/tsx.c | 2 +-
xen/arch/x86/x86_64/mmconfig-shared.c | 2 +-
xen/common/argo.c | 2 +-
xen/common/core_parking.c | 2 +-
xen/common/debugtrace.c | 2 +-
xen/common/domain.c | 2 +-
xen/common/efi/boot.c | 2 +-
xen/common/grant_table.c | 14 +++++++-------
xen/common/kexec.c | 6 +++---
xen/common/memory.c | 2 +-
xen/common/page_alloc.c | 2 +-
xen/common/sched/cpupool.c | 2 +-
xen/common/sched/credit2.c | 2 +-
xen/drivers/acpi/tables.c | 2 +-
xen/drivers/char/console.c | 18 +++++++++---------
xen/drivers/cpufreq/cpufreq.c | 2 +-
xen/drivers/passthrough/amd/iommu_acpi.c | 6 +++---
xen/drivers/passthrough/iommu.c | 4 ++--
xen/drivers/passthrough/pci.c | 4 ++--
xen/drivers/passthrough/vtd/dmar.c | 2 +-
xen/drivers/passthrough/vtd/quirks.c | 2 +-
xen/drivers/video/vesa.c | 2 +-
xen/xsm/flask/flask_op.c | 5 ++---
xen/xsm/xsm_core.c | 2 +-
49 files changed, 92 insertions(+), 98 deletions(-)
diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index d788c8bffc84..5d73eb5917af 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -106,7 +106,7 @@ void (*__read_mostly pm_idle_save)(void);
unsigned int max_cstate __read_mostly = UINT_MAX;
unsigned int max_csubstate __read_mostly = UINT_MAX;
-static int __init parse_cstate(const char *s)
+static int __init cf_check parse_cstate(const char *s)
{
max_cstate = simple_strtoul(s, &s, 0);
if ( *s == ',' )
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 5eaa77f66a28..912d4c4d62f4 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -35,7 +35,7 @@
uint32_t system_reset_counter = 1;
-static int __init parse_acpi_sleep(const char *s)
+static int __init cf_check parse_acpi_sleep(const char *s)
{
const char *ss;
unsigned int flag = 0;
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index 583656158532..68e4d870c749 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -775,7 +775,7 @@ int lapic_resume(void)
* Original code written by Keir Fraser.
*/
-static int __init lapic_disable(const char *str)
+static int __init cf_check lapic_disable(const char *str)
{
enable_local_apic = -1;
setup_clear_cpu_cap(X86_FEATURE_APIC);
@@ -784,7 +784,7 @@ static int __init lapic_disable(const char *str)
custom_param("nolapic", lapic_disable);
boolean_param("lapic", enable_local_apic);
-static int __init apic_set_verbosity(const char *str)
+static int __init cf_check apic_set_verbosity(const char *str)
{
if (strcmp("debug", str) == 0)
apic_verbosity = APIC_DEBUG;
diff --git a/xen/arch/x86/cpu/mcheck/mce.c b/xen/arch/x86/cpu/mcheck/mce.c
index eae08caa07cd..ea86d84481b2 100644
--- a/xen/arch/x86/cpu/mcheck/mce.c
+++ b/xen/arch/x86/cpu/mcheck/mce.c
@@ -63,7 +63,7 @@ struct mca_banks *mca_allbanks;
#endif
int mce_verbosity;
-static int __init mce_set_verbosity(const char *str)
+static int __init cf_check mce_set_verbosity(const char *str)
{
if ( strcmp("verbose", str) == 0 )
mce_verbosity = MCE_VERBOSE;
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index ac3ceb567c41..95d35ca0f3f7 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -111,7 +111,7 @@ void __init microcode_set_module(unsigned int idx)
* optional. If the EFI has forced which of the multiboot payloads is to be
* used, only nmi=<bool> is parsed.
*/
-static int __init parse_ucode(const char *s)
+static int __init cf_check parse_ucode(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c
index 598291f4ece9..b10d6e2eb458 100644
--- a/xen/arch/x86/cpu/vpmu.c
+++ b/xen/arch/x86/cpu/vpmu.c
@@ -56,7 +56,7 @@ static unsigned vpmu_count;
static DEFINE_PER_CPU(struct vcpu *, last_vcpu);
-static int __init parse_vpmu_params(const char *s)
+static int __init cf_check parse_vpmu_params(const char *s)
{
const char *ss;
int rc = 0, val;
diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 7c638eff2bf4..bb554b06a73f 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -101,7 +101,7 @@ static int __init always_inline parse_cpuid(
return rc;
}
-static void __init _parse_xen_cpuid(unsigned int feat, bool val)
+static void __init cf_check _parse_xen_cpuid(unsigned int feat, bool val)
{
if ( !val )
setup_clear_cpu_cap(feat);
@@ -110,7 +110,7 @@ static void __init _parse_xen_cpuid(unsigned int feat, bool val)
setup_force_cpu_cap(X86_FEATURE_RDRAND);
}
-static int __init parse_xen_cpuid(const char *s)
+static int __init cf_check parse_xen_cpuid(const char *s)
{
return parse_cpuid(s, _parse_xen_cpuid);
}
@@ -120,13 +120,13 @@ static bool __initdata dom0_cpuid_cmdline;
static uint32_t __initdata dom0_enable_feat[FSCAPINTS];
static uint32_t __initdata dom0_disable_feat[FSCAPINTS];
-static void __init _parse_dom0_cpuid(unsigned int feat, bool val)
+static void __init cf_check _parse_dom0_cpuid(unsigned int feat, bool val)
{
__set_bit (feat, val ? dom0_enable_feat : dom0_disable_feat);
__clear_bit(feat, val ? dom0_disable_feat : dom0_enable_feat );
}
-static int __init parse_dom0_cpuid(const char *s)
+static int __init cf_check parse_dom0_cpuid(const char *s)
{
dom0_cpuid_cmdline = true;
diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
index a7fec05956c1..4d1c5c60e407 100644
--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -115,7 +115,7 @@ static int __init parse_amt(const char *s, const char **ps, struct memsize *sz)
return 0;
}
-static int __init parse_dom0_mem(const char *s)
+static int __init cf_check parse_dom0_mem(const char *s)
{
int ret;
@@ -144,7 +144,7 @@ custom_param("dom0_mem", parse_dom0_mem);
static unsigned int __initdata opt_dom0_max_vcpus_min = 1;
static unsigned int __initdata opt_dom0_max_vcpus_max = UINT_MAX;
-static int __init parse_dom0_max_vcpus(const char *s)
+static int __init cf_check parse_dom0_max_vcpus(const char *s)
{
if ( *s == '-' ) /* -M */
opt_dom0_max_vcpus_max = simple_strtoul(s + 1, &s, 0);
@@ -168,7 +168,7 @@ static __initdata unsigned int dom0_pxms[MAX_NUMNODES] =
{ [0 ... MAX_NUMNODES - 1] = ~0 };
bool __initdata dom0_affinity_relaxed;
-static int __init parse_dom0_nodes(const char *s)
+static int __init cf_check parse_dom0_nodes(const char *s)
{
const char *ss;
int rc = 0;
@@ -266,7 +266,7 @@ bool __initdata opt_dom0_pvh = !IS_ENABLED(CONFIG_PV);
bool __initdata opt_dom0_verbose = IS_ENABLED(CONFIG_VERBOSE_DEBUG);
bool __initdata opt_dom0_msr_relaxed;
-static int __init parse_dom0_param(const char *s)
+static int __init cf_check parse_dom0_param(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/arch/x86/genapic/probe.c b/xen/arch/x86/genapic/probe.c
index 66bc5ce072dc..ad57912f506b 100644
--- a/xen/arch/x86/genapic/probe.c
+++ b/xen/arch/x86/genapic/probe.c
@@ -43,7 +43,7 @@ void __init generic_bigsmp_probe(void)
}
}
-static int __init genapic_apic_force(const char *str)
+static int __init cf_check genapic_apic_force(const char *str)
{
int i, rc = -EINVAL;
diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index 2fe8b005a513..7a810e4e7171 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -65,7 +65,7 @@ u8 __initdata hpet_flags;
static bool __initdata force_hpet_broadcast;
boolean_param("hpetbroadcast", force_hpet_broadcast);
-static int __init parse_hpet_param(const char *s)
+static int __init cf_check parse_hpet_param(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
index 8986b8e03c2e..7ebcaa1c899f 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -1186,7 +1186,7 @@ static int viridian_load_vcpu_ctxt(struct domain *d,
HVM_REGISTER_SAVE_RESTORE(VIRIDIAN_VCPU, viridian_save_vcpu_ctxt,
viridian_load_vcpu_ctxt, 1, HVMSR_PER_VCPU);
-static int __init parse_viridian_version(const char *arg)
+static int __init cf_check parse_viridian_version(const char *arg)
{
const char *t;
unsigned int n[3];
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 7ab15e07a0b2..f72a7db0453d 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -71,7 +71,7 @@ static bool __read_mostly opt_ept_pml = true;
static s8 __read_mostly opt_ept_ad = -1;
int8_t __read_mostly opt_ept_exec_sp = -1;
-static int __init parse_ept_param(const char *s)
+static int __init cf_check parse_ept_param(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -107,16 +107,16 @@ static void update_ept_param(void)
opt_ept_exec_sp);
}
-static void __init init_ept_param(struct param_hypfs *par)
+static void __init cf_check init_ept_param(struct param_hypfs *par)
{
update_ept_param();
custom_runtime_set_var(par, opt_ept_setting);
}
-static int parse_ept_param_runtime(const char *s);
+static int cf_check parse_ept_param_runtime(const char *s);
custom_runtime_only_param("ept", parse_ept_param_runtime, init_ept_param);
-static int parse_ept_param_runtime(const char *s)
+static int cf_check parse_ept_param_runtime(const char *s)
{
struct domain *d;
int val;
diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index 1c49a0fe1478..4135a9c06052 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -1601,7 +1601,7 @@ static unsigned int startup_level_ioapic_irq(struct irq_desc *desc)
return 0; /* don't check for pending */
}
-static int __init setup_ioapic_ack(const char *s)
+static int __init cf_check setup_ioapic_ack(const char *s)
{
if ( !strcmp(s, "old") )
{
diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 67cbf6b979dc..84b174d0f51f 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -28,8 +28,6 @@
#include <irq_vectors.h>
#include <public/physdev.h>
-static int parse_irq_vector_map_param(const char *s);
-
/* opt_noirqbalance: If true, software IRQ balancing/affinity is disabled. */
bool __read_mostly opt_noirqbalance;
boolean_param("noirqbalance", opt_noirqbalance);
@@ -40,7 +38,6 @@ integer_param("nr_irqs", nr_irqs);
/* This default may be changed by the AMD IOMMU code */
int __read_mostly opt_irq_vector_map = OPT_IRQ_VECTOR_MAP_DEFAULT;
-custom_param("irq_vector_map", parse_irq_vector_map_param);
/* Max number of guests IRQ could be shared with */
static unsigned char __read_mostly irq_max_guests;
@@ -66,7 +63,7 @@ static struct timer irq_ratelimit_timer;
static unsigned int __read_mostly irq_ratelimit_threshold = 10000;
integer_param("irq_ratelimit", irq_ratelimit_threshold);
-static int __init parse_irq_vector_map_param(const char *s)
+static int __init cf_check parse_irq_vector_map_param(const char *s)
{
const char *ss;
int rc = 0;
@@ -90,6 +87,7 @@ static int __init parse_irq_vector_map_param(const char *s)
return rc;
}
+custom_param("irq_vector_map", parse_irq_vector_map_param);
/* Must be called when irq disabled */
void lock_vector_lock(void)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index cdbd04e19752..a1b8737096c4 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -189,7 +189,7 @@ static uint32_t base_disallow_mask;
static s8 __read_mostly opt_mmio_relax;
-static int __init parse_mmio_relax(const char *s)
+static int __init cf_check parse_mmio_relax(const char *s)
{
if ( !*s )
opt_mmio_relax = 1;
diff --git a/xen/arch/x86/nmi.c b/xen/arch/x86/nmi.c
index c515de63365a..515a3633e5aa 100644
--- a/xen/arch/x86/nmi.c
+++ b/xen/arch/x86/nmi.c
@@ -48,7 +48,7 @@ bool __initdata opt_watchdog;
/* watchdog_force: If true, process unknown NMIs when running the watchdog. */
bool watchdog_force;
-static int __init parse_watchdog(const char *s)
+static int __init cf_check parse_watchdog(const char *s)
{
if ( !*s )
{
@@ -78,7 +78,7 @@ custom_param("watchdog", parse_watchdog);
/* opt_watchdog_timeout: Number of seconds to wait before panic. */
static unsigned int opt_watchdog_timeout = 5;
-static int parse_watchdog_timeout(const char *s)
+static int __init cf_check parse_watchdog_timeout(const char *s)
{
const char *q;
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index ce79ee44cefe..6be5a0c93322 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -19,9 +19,6 @@
#include <xen/sched.h>
#include <xen/softirq.h>
-static int numa_setup(const char *s);
-custom_param("numa", numa_setup);
-
#ifndef Dprintk
#define Dprintk(x...)
#endif
@@ -294,7 +291,7 @@ void numa_set_node(int cpu, nodeid_t node)
}
/* [numa=off] */
-static __init int numa_setup(const char *opt)
+static int __init cf_check numa_setup(const char *opt)
{
if ( !strncmp(opt,"off",3) )
numa_off = true;
@@ -321,6 +318,7 @@ static __init int numa_setup(const char *opt)
return 0;
}
+custom_param("numa", numa_setup);
/*
* Setup early cpu_to_node.
diff --git a/xen/arch/x86/oprofile/nmi_int.c b/xen/arch/x86/oprofile/nmi_int.c
index a13bd82915ac..7842d95b95ea 100644
--- a/xen/arch/x86/oprofile/nmi_int.c
+++ b/xen/arch/x86/oprofile/nmi_int.c
@@ -340,7 +340,7 @@ static int __init p4_init(char ** cpu_type)
static int force_arch_perfmon;
-static int force_cpu_type(const char *str)
+static int __init cf_check force_cpu_type(const char *str)
{
if (!strcmp(str, "arch_perfmon")) {
force_arch_perfmon = 1;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index d805b85dc60b..56916344cb1d 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -573,7 +573,7 @@ static bool __init parse_psr_bool(const char *s, const char *delim,
return false;
}
-static int __init parse_psr_param(const char *s)
+static int __init cf_check parse_psr_param(const char *s)
{
const char *ss, *val_delim;
const char *q;
diff --git a/xen/arch/x86/pv/domain.c b/xen/arch/x86/pv/domain.c
index 6ad533183bcd..125c4561a7ea 100644
--- a/xen/arch/x86/pv/domain.c
+++ b/xen/arch/x86/pv/domain.c
@@ -20,7 +20,7 @@
int8_t __read_mostly opt_pv32 = -1;
#endif
-static __init int parse_pv(const char *s)
+static int __init cf_check parse_pv(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -63,16 +63,16 @@ static const char opt_pcid_2_string[][7] = {
[PCID_NOXPTI] = "noxpti",
};
-static void __init opt_pcid_init(struct param_hypfs *par)
+static void __init cf_check opt_pcid_init(struct param_hypfs *par)
{
custom_runtime_set_var(par, opt_pcid_2_string[opt_pcid]);
}
#endif
-static int parse_pcid(const char *s);
+static int cf_check parse_pcid(const char *s);
custom_runtime_param("pcid", parse_pcid, opt_pcid_init);
-static int parse_pcid(const char *s)
+static int cf_check parse_pcid(const char *s)
{
int rc = 0;
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index 4c710ad8913f..ae4d8913faa1 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -73,7 +73,7 @@ static uint64_t __initdata shim_nrpages;
static uint64_t __initdata shim_min_nrpages;
static uint64_t __initdata shim_max_nrpages;
-static int __init parse_shim_mem(const char *s)
+static int __init cf_check parse_shim_mem(const char *s)
{
do {
if ( !strncmp(s, "min:", 4) )
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 2f6e10d0cf77..3a4ec1fcfd04 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -81,8 +81,6 @@ unsigned long __read_mostly cr4_pv32_mask;
/* "acpi=ht": Limit ACPI just to boot-time to enable HT. */
/* "acpi=noirq": Disables ACPI interrupt routing. */
/* "acpi=verbose": Enables more verbose ACPI boot time logging. */
-static int parse_acpi_param(const char *s);
-custom_param("acpi", parse_acpi_param);
/* **** Linux config option: propagated to domain0. */
/* noapic: Disable IOAPIC setup. */
@@ -104,7 +102,7 @@ static bool __initdata opt_xen_shstk = true;
#define opt_xen_shstk false
#endif
-static int __init parse_cet(const char *s)
+static int __init cf_check parse_cet(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -159,7 +157,7 @@ static s8 __initdata opt_smep = -1;
*/
static struct domain *__initdata dom0;
-static int __init parse_smep_param(const char *s)
+static int __init cf_check parse_smep_param(const char *s)
{
if ( !*s )
{
@@ -190,7 +188,7 @@ custom_param("smep", parse_smep_param);
#define SMAP_HVM_ONLY (-2)
static s8 __initdata opt_smap = -1;
-static int __init parse_smap_param(const char *s)
+static int __init cf_check parse_smap_param(const char *s)
{
if ( !*s )
{
@@ -221,7 +219,7 @@ bool __read_mostly acpi_disabled;
bool __initdata acpi_force;
static char __initdata acpi_param[10] = "";
-static int __init parse_acpi_param(const char *s)
+static int __init cf_check parse_acpi_param(const char *s)
{
/* Interpret the parameter for use within Xen. */
if ( !parse_bool(s, NULL) )
@@ -257,6 +255,7 @@ static int __init parse_acpi_param(const char *s)
return 0;
}
+custom_param("acpi", parse_acpi_param);
static const module_t *__initdata initial_images;
static unsigned int __initdata nr_initial_images;
diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index acef03314372..a01354d93319 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -56,7 +56,7 @@ static int reboot_mode;
*/
static enum reboot_type reboot_type = BOOT_INVALID;
-static int __init set_reboot_type(const char *str)
+static int __init cf_check set_reboot_type(const char *str)
{
int rc = 0;
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index db27bc3f6451..2d4cf5c7ef80 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -68,7 +68,7 @@ static bool __initdata cpu_has_bug_mds; /* Any other M{LP,SB,FB}DS combination.
static int8_t __initdata opt_srb_lock = -1;
-static int __init parse_spec_ctrl(const char *s)
+static int __init cf_check parse_spec_ctrl(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -218,7 +218,7 @@ static __init void xpti_init_default(uint64_t caps)
}
}
-static __init int parse_xpti(const char *s)
+static int __init cf_check parse_xpti(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -264,7 +264,7 @@ custom_param("xpti", parse_xpti);
int8_t __read_mostly opt_pv_l1tf_hwdom = -1;
int8_t __read_mostly opt_pv_l1tf_domu = -1;
-static __init int parse_pv_l1tf(const char *s)
+static int __init cf_check parse_pv_l1tf(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index bc41a3aa37b2..818a42a406c5 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -2354,7 +2354,7 @@ int hwdom_pit_access(struct ioreq *ioreq)
* tsc=skewed: Assume TSCs are individually reliable, but skewed across CPUs.
* tsc=stable:socket: Assume TSCs are reliable across sockets.
*/
-static int __init tsc_parse(const char *s)
+static int __init cf_check tsc_parse(const char *s)
{
if ( !strcmp(s, "unstable") )
{
diff --git a/xen/arch/x86/tsx.c b/xen/arch/x86/tsx.c
index be89741a2f6d..b156844cdec1 100644
--- a/xen/arch/x86/tsx.c
+++ b/xen/arch/x86/tsx.c
@@ -22,7 +22,7 @@ int8_t __read_mostly opt_tsx = -1;
int8_t __read_mostly cpu_has_tsx_ctrl = -1;
bool __read_mostly rtm_disabled;
-static int __init parse_tsx(const char *s)
+static int __init cf_check parse_tsx(const char *s)
{
int rc = 0, val = parse_bool(s, NULL);
diff --git a/xen/arch/x86/x86_64/mmconfig-shared.c b/xen/arch/x86/x86_64/mmconfig-shared.c
index 7c3ed64b4c6c..2fa7f3f0bc4b 100644
--- a/xen/arch/x86/x86_64/mmconfig-shared.c
+++ b/xen/arch/x86/x86_64/mmconfig-shared.c
@@ -29,7 +29,7 @@
unsigned int pci_probe = PCI_PROBE_CONF1 | PCI_PROBE_MMCONF;
-static int __init parse_mmcfg(const char *s)
+static int __init cf_check parse_mmcfg(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/common/argo.c b/xen/common/argo.c
index 1448faf65731..297f6d11f04d 100644
--- a/xen/common/argo.c
+++ b/xen/common/argo.c
@@ -78,7 +78,7 @@ DEFINE_COMPAT_HANDLE(compat_argo_iov_t);
static bool __read_mostly opt_argo;
static bool __read_mostly opt_argo_mac_permissive;
-static int __init parse_argo(const char *s)
+static int __init cf_check parse_argo(const char *s)
{
const char *ss;
int val, rc = 0;
diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c
index 411106c675c9..aa432ed2f57b 100644
--- a/xen/common/core_parking.c
+++ b/xen/common/core_parking.c
@@ -40,7 +40,7 @@ static enum core_parking_controller {
PERFORMANCE_FIRST
} core_parking_controller __initdata = POWER_FIRST;
-static int __init setup_core_parking_option(const char *str)
+static int __init cf_check setup_core_parking_option(const char *str)
{
if ( !strcmp(str, "power") )
core_parking_controller = POWER_FIRST;
diff --git a/xen/common/debugtrace.c b/xen/common/debugtrace.c
index f3794b945376..29b11239f5a5 100644
--- a/xen/common/debugtrace.c
+++ b/xen/common/debugtrace.c
@@ -38,7 +38,7 @@ static bool debugtrace_buf_empty = true;
static bool debugtrace_used;
static DEFINE_SPINLOCK(debugtrace_lock);
-static int __init debugtrace_parse_param(const char *s)
+static int __init cf_check debugtrace_parse_param(const char *s)
{
unsigned long bytes;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index ac2746d0d69a..dacd03254cb2 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -354,7 +354,7 @@ static int late_hwdom_init(struct domain *d)
static unsigned int __read_mostly extra_hwdom_irqs;
static unsigned int __read_mostly extra_domU_irqs = 32;
-static int __init parse_extra_guest_irqs(const char *s)
+static int __init cf_check parse_extra_guest_irqs(const char *s)
{
if ( isdigit(*s) )
extra_domU_irqs = simple_strtoul(s, &s, 0);
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 12fd0844bd55..f31f68fd4cd1 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1417,7 +1417,7 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
static bool __initdata efi_map_uc;
-static int __init parse_efi_param(const char *s)
+static int __init cf_check parse_efi_param(const char *s)
{
const char *ss;
int rc = 0, val;
diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 153332b7bfbe..b663845d9c6c 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -117,12 +117,12 @@ static void update_gnttab_par(unsigned int val, struct param_hypfs *par,
custom_runtime_set_var_sz(par, parval, GRANT_CUSTOM_VAL_SZ);
}
-static void __init gnttab_max_frames_init(struct param_hypfs *par)
+static void __init cf_check gnttab_max_frames_init(struct param_hypfs *par)
{
update_gnttab_par(opt_max_grant_frames, par, opt_max_grant_frames_val);
}
-static void __init max_maptrack_frames_init(struct param_hypfs *par)
+static void __init cf_check max_maptrack_frames_init(struct param_hypfs *par)
{
update_gnttab_par(opt_max_maptrack_frames, par,
opt_max_maptrack_frames_val);
@@ -156,23 +156,23 @@ static int parse_gnttab_limit(const char *arg, unsigned int *valp,
return 0;
}
-static int parse_gnttab_max_frames(const char *arg);
+static int cf_check parse_gnttab_max_frames(const char *arg);
custom_runtime_param("gnttab_max_frames", parse_gnttab_max_frames,
gnttab_max_frames_init);
-static int parse_gnttab_max_frames(const char *arg)
+static int cf_check parse_gnttab_max_frames(const char *arg)
{
return parse_gnttab_limit(arg, &opt_max_grant_frames,
param_2_parfs(parse_gnttab_max_frames),
opt_max_grant_frames_val);
}
-static int parse_gnttab_max_maptrack_frames(const char *arg);
+static int cf_check parse_gnttab_max_maptrack_frames(const char *arg);
custom_runtime_param("gnttab_max_maptrack_frames",
parse_gnttab_max_maptrack_frames,
max_maptrack_frames_init);
-static int parse_gnttab_max_maptrack_frames(const char *arg)
+static int cf_check parse_gnttab_max_maptrack_frames(const char *arg)
{
return parse_gnttab_limit(arg, &opt_max_maptrack_frames,
param_2_parfs(parse_gnttab_max_maptrack_frames),
@@ -191,7 +191,7 @@ static bool __ro_after_init opt_grant_transfer = true;
#define opt_grant_transfer false
#endif
-static int __init parse_gnttab(const char *s)
+static int __init cf_check parse_gnttab(const char *s)
{
const char *ss, *e;
int val, rc = 0;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 8471590aeea2..6286c0bbf08b 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -104,7 +104,7 @@ static void *crash_heap_current = NULL, *crash_heap_end = NULL;
* < and below are synonyomous, the latter being useful for grub2 systems
* which would otherwise require escaping of the < option
*/
-static int __init parse_crashkernel(const char *str)
+static int __init cf_check parse_crashkernel(const char *str)
{
const char *cur;
int rc = 0;
@@ -201,7 +201,7 @@ custom_param("crashkernel", parse_crashkernel);
* - all will allocate additional structures such as domain and vcpu structs
* low so the crash kernel can perform an extended analysis of state.
*/
-static int __init parse_low_crashinfo(const char *str)
+static int __init cf_check parse_low_crashinfo(const char *str)
{
if ( !strlen(str) )
@@ -230,7 +230,7 @@ custom_param("low_crashinfo", parse_low_crashinfo);
*
* <addr> will be rounded down to the nearest power of two. Defaults to 64G
*/
-static int __init parse_crashinfo_maxaddr(const char *str)
+static int __init cf_check parse_crashinfo_maxaddr(const char *str)
{
u64 addr;
const char *q;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 38732dde6fd7..ede45c4af9db 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -62,7 +62,7 @@ static unsigned int __read_mostly hwdom_max_order = CONFIG_HWDOM_MAX_ORDER;
static unsigned int __read_mostly ptdom_max_order = CONFIG_PTDOM_MAX_ORDER;
#endif
-static int __init parse_max_order(const char *s)
+static int __init cf_check parse_max_order(const char *s)
{
if ( *s != ',' )
domu_max_order = simple_strtoul(s, &s, 0);
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index f8749b0787a6..ad06655158d2 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -179,7 +179,7 @@ enum bootscrub_mode {
* https://bugs.llvm.org/show_bug.cgi?id=39707
*/
static enum bootscrub_mode __read_mostly opt_bootscrub = BOOTSCRUB_IDLE;
-static int __init parse_bootscrub_param(const char *s)
+static int __init cf_check parse_bootscrub_param(const char *s)
{
/* Interpret 'bootscrub' alone in its positive boolean form */
if ( *s == '\0' )
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 8c6e6eb9ccd5..f0dd626054a6 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -93,7 +93,7 @@ static int sched_gran_get(const char *str, enum sched_gran *mode)
return -EINVAL;
}
-static int __init sched_select_granularity(const char *str)
+static int __init cf_check sched_select_granularity(const char *str)
{
return sched_gran_get(str, &opt_sched_granularity);
}
diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
index 6396b38e044c..a5f073cda51e 100644
--- a/xen/common/sched/credit2.c
+++ b/xen/common/sched/credit2.c
@@ -456,7 +456,7 @@ static const char *const opt_runqueue_str[] = {
};
static int __read_mostly opt_runqueue = OPT_RUNQUEUE_SOCKET;
-static int __init parse_credit2_runqueue(const char *s)
+static int __init cf_check parse_credit2_runqueue(const char *s)
{
unsigned int i;
diff --git a/xen/drivers/acpi/tables.c b/xen/drivers/acpi/tables.c
index f39cd5eaac89..96ff96b84c66 100644
--- a/xen/drivers/acpi/tables.c
+++ b/xen/drivers/acpi/tables.c
@@ -472,7 +472,7 @@ int __init acpi_table_init(void)
return 0;
}
-static int __init acpi_parse_apic_instance(const char *str)
+static int __init cf_check acpi_parse_apic_instance(const char *str)
{
const char *q;
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index a043e9521afd..4694be83db45 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -88,7 +88,7 @@ static const char con_timestamp_mode_2_string[][7] = {
[TSM_RAW] = "raw",
};
-static void con_timestamp_mode_upd(struct param_hypfs *par)
+static void cf_check con_timestamp_mode_upd(struct param_hypfs *par)
{
const char *val = con_timestamp_mode_2_string[opt_con_timestamp_mode];
@@ -98,7 +98,7 @@ static void con_timestamp_mode_upd(struct param_hypfs *par)
#define con_timestamp_mode_upd(par)
#endif
-static int parse_console_timestamps(const char *s);
+static int cf_check parse_console_timestamps(const char *s);
custom_runtime_param("console_timestamps", parse_console_timestamps,
con_timestamp_mode_upd);
@@ -160,8 +160,8 @@ static int __read_mostly xenlog_guest_upper_thresh =
static int __read_mostly xenlog_guest_lower_thresh =
XENLOG_GUEST_LOWER_THRESHOLD;
-static int parse_loglvl(const char *s);
-static int parse_guest_loglvl(const char *s);
+static int cf_check parse_loglvl(const char *s);
+static int cf_check parse_guest_loglvl(const char *s);
#ifdef CONFIG_HYPFS
#define LOGLVL_VAL_SZ 16
@@ -176,13 +176,13 @@ static void xenlog_update_val(int lower, int upper, char *val)
snprintf(val, LOGLVL_VAL_SZ, "%s/%s", lvl2opt[lower], lvl2opt[upper]);
}
-static void __init xenlog_init(struct param_hypfs *par)
+static void __init cf_check xenlog_init(struct param_hypfs *par)
{
xenlog_update_val(xenlog_lower_thresh, xenlog_upper_thresh, xenlog_val);
custom_runtime_set_var(par, xenlog_val);
}
-static void __init xenlog_guest_init(struct param_hypfs *par)
+static void __init cf_check xenlog_guest_init(struct param_hypfs *par)
{
xenlog_update_val(xenlog_guest_lower_thresh, xenlog_guest_upper_thresh,
xenlog_guest_val);
@@ -240,7 +240,7 @@ static int _parse_loglvl(const char *s, int *lower, int *upper, char *val)
return *s ? -EINVAL : 0;
}
-static int parse_loglvl(const char *s)
+static int cf_check parse_loglvl(const char *s)
{
int ret;
@@ -251,7 +251,7 @@ static int parse_loglvl(const char *s)
return ret;
}
-static int parse_guest_loglvl(const char *s)
+static int cf_check parse_guest_loglvl(const char *s)
{
int ret;
@@ -793,7 +793,7 @@ static int printk_prefix_check(char *p, char **pp)
((loglvl < upper_thresh) && printk_ratelimit()));
}
-static int parse_console_timestamps(const char *s)
+static int cf_check parse_console_timestamps(const char *s)
{
switch ( parse_bool(s, NULL) )
{
diff --git a/xen/drivers/cpufreq/cpufreq.c b/xen/drivers/cpufreq/cpufreq.c
index 419aae83eea6..36b079296235 100644
--- a/xen/drivers/cpufreq/cpufreq.c
+++ b/xen/drivers/cpufreq/cpufreq.c
@@ -65,7 +65,7 @@ enum cpufreq_controller cpufreq_controller = FREQCTL_xen;
static int __init cpufreq_cmdline_parse(const char *s);
-static int __init setup_cpufreq_option(const char *str)
+static int __init cf_check setup_cpufreq_option(const char *str)
{
const char *arg = strpbrk(str, ",:");
int choice;
diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index b07fa4c40124..5ea227732821 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -704,7 +704,7 @@ static u16 __init parse_ivhd_device_extended_range(
return dev_length;
}
-static int __init parse_ivrs_ioapic(const char *str)
+static int __init cf_check parse_ivrs_ioapic(const char *str)
{
const char *s = str;
unsigned long id;
@@ -742,7 +742,7 @@ static int __init parse_ivrs_ioapic(const char *str)
}
custom_param("ivrs_ioapic[", parse_ivrs_ioapic);
-static int __init parse_ivrs_hpet(const char *str)
+static int __init cf_check parse_ivrs_hpet(const char *str)
{
const char *s = str;
unsigned long id;
@@ -1369,7 +1369,7 @@ int __init amd_iommu_get_supported_ivhd_type(void)
* Format:
* ivmd=<start>[-<end>][=<bdf1>[-<bdf1>'][,<bdf2>[-<bdf2>'][,...]]][;<start>...]
*/
-static int __init parse_ivmd_param(const char *s)
+static int __init cf_check parse_ivmd_param(const char *s)
{
do {
unsigned long start, end;
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index fc18f63bd4ac..6ee267d2bfd4 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -64,7 +64,7 @@ bool_t __read_mostly amd_iommu_perdev_intremap = 1;
DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
-static int __init parse_iommu_param(const char *s)
+static int __init cf_check parse_iommu_param(const char *s)
{
const char *ss;
int val, rc = 0;
@@ -135,7 +135,7 @@ static int __init parse_iommu_param(const char *s)
}
custom_param("iommu", parse_iommu_param);
-static int __init parse_dom0_iommu_param(const char *s)
+static int __init cf_check parse_dom0_iommu_param(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 70b6684981c1..4b59b332f01f 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -146,7 +146,7 @@ static struct phantom_dev {
} phantom_devs[8];
static unsigned int nr_phantom_devs;
-static int __init parse_phantom_dev(const char *str)
+static int __init cf_check parse_phantom_dev(const char *str)
{
const char *s;
unsigned int seg, bus, slot;
@@ -182,7 +182,7 @@ custom_param("pci-phantom", parse_phantom_dev);
static u16 __read_mostly command_mask;
static u16 __read_mostly bridge_ctl_mask;
-static int __init parse_pci_param(const char *s)
+static int __init cf_check parse_pci_param(const char *s)
{
const char *ss;
int rc = 0;
diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index 33a12b2ae976..b152f3da916b 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -1084,7 +1084,7 @@ int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
* If a segment is specified for other than the first device, and it does not
* match the one specified for the first one, an error will be reported.
*/
-static int __init parse_rmrr_param(const char *str)
+static int __init cf_check parse_rmrr_param(const char *str)
{
const char *s = str, *cur, *stmp;
unsigned int seg, bus, dev, func, dev_count;
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 52b47dd89325..0590ddeea7c4 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -308,7 +308,7 @@ void vtd_ops_postamble_quirk(struct vtd_iommu *iommu)
}
}
-static int __init parse_snb_timeout(const char *s)
+static int __init cf_check parse_snb_timeout(const char *s)
{
int t;
const char *q = NULL;
diff --git a/xen/drivers/video/vesa.c b/xen/drivers/video/vesa.c
index 0342a3dedef0..c8f81a5cc5fc 100644
--- a/xen/drivers/video/vesa.c
+++ b/xen/drivers/video/vesa.c
@@ -29,7 +29,7 @@ integer_param("vesa-ram", vram_total);
static unsigned int __initdata vram_remap;
static unsigned int __initdata font_height;
-static int __init parse_font_height(const char *s)
+static int __init cf_check parse_font_height(const char *s)
{
if ( simple_strtoul(s, &s, 10) == 8 && (*s++ == 'x') )
font_height = simple_strtoul(s, &s, 10);
diff --git a/xen/xsm/flask/flask_op.c b/xen/xsm/flask/flask_op.c
index bb3bebc30e01..2d7ca3abaecd 100644
--- a/xen/xsm/flask/flask_op.c
+++ b/xen/xsm/flask/flask_op.c
@@ -28,8 +28,6 @@
#define _copy_from_guest copy_from_guest
enum flask_bootparam_t __read_mostly flask_bootparam = FLASK_BOOTPARAM_ENFORCING;
-static int parse_flask_param(const char *s);
-custom_param("flask", parse_flask_param);
bool __read_mostly flask_enforcing = true;
@@ -60,7 +58,7 @@ static int flask_security_make_bools(void);
extern int ss_initialized;
-static int __init parse_flask_param(const char *s)
+static int __init cf_check parse_flask_param(const char *s)
{
if ( !strcmp(s, "enforcing") )
flask_bootparam = FLASK_BOOTPARAM_ENFORCING;
@@ -75,6 +73,7 @@ static int __init parse_flask_param(const char *s)
return (flask_bootparam == FLASK_BOOTPARAM_INVALID) ? -EINVAL : 0;
}
+custom_param("flask", parse_flask_param);
static int domain_has_security(struct domain *d, u32 perms)
{
diff --git a/xen/xsm/xsm_core.c b/xen/xsm/xsm_core.c
index 5fc3a5f75478..2286a502e3e8 100644
--- a/xen/xsm/xsm_core.c
+++ b/xen/xsm/xsm_core.c
@@ -55,7 +55,7 @@ static enum xsm_bootparam __initdata xsm_bootparam =
XSM_BOOTPARAM_DUMMY;
#endif
-static int __init parse_xsm_param(const char *s)
+static int __init cf_check parse_xsm_param(const char *s)
{
int rc = 0;
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 12/70] xen: CFI hardening for continue_hypercall_on_cpu()
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (4 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 08/70] xen: CFI hardening for custom_param() Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 26/70] xen/iommu: CFI hardening Andrew Cooper
` (9 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3:
* Fix !CONFIG_PV_SHIM build. Annotate pv_shim_cpu_{up,down}() stubs.
---
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/cpu/microcode/core.c | 2 +-
xen/arch/x86/include/asm/pv/shim.h | 8 ++++----
xen/arch/x86/include/asm/smp.h | 6 +++---
xen/arch/x86/platform_hypercall.c | 4 ++--
xen/arch/x86/pv/shim.c | 4 ++--
xen/arch/x86/smp.c | 4 ++--
xen/arch/x86/sysctl.c | 2 +-
xen/common/core_parking.c | 2 +-
xen/common/kexec.c | 2 +-
xen/common/sched/cpupool.c | 2 +-
11 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 912d4c4d62f4..c4e7e8698920 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -326,7 +326,7 @@ static int enter_state(u32 state)
return error;
}
-static long enter_state_helper(void *data)
+static long cf_check enter_state_helper(void *data)
{
struct acpi_sleep_info *sinfo = (struct acpi_sleep_info *)data;
return enter_state(sinfo->sleep_state);
diff --git a/xen/arch/x86/cpu/microcode/core.c b/xen/arch/x86/cpu/microcode/core.c
index 46f55fe7f191..9631042190ab 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -558,7 +558,7 @@ struct ucode_buf {
char buffer[];
};
-static long microcode_update_helper(void *data)
+static long cf_check microcode_update_helper(void *data)
{
int ret;
struct ucode_buf *buffer = data;
diff --git a/xen/arch/x86/include/asm/pv/shim.h b/xen/arch/x86/include/asm/pv/shim.h
index 6415f8068e5c..a43c3689b48a 100644
--- a/xen/arch/x86/include/asm/pv/shim.h
+++ b/xen/arch/x86/include/asm/pv/shim.h
@@ -38,8 +38,8 @@ void pv_shim_setup_dom(struct domain *d, l4_pgentry_t *l4start,
start_info_t *si);
int pv_shim_shutdown(uint8_t reason);
void pv_shim_inject_evtchn(unsigned int port);
-long pv_shim_cpu_up(void *data);
-long pv_shim_cpu_down(void *data);
+long cf_check pv_shim_cpu_up(void *data);
+long cf_check pv_shim_cpu_down(void *data);
void pv_shim_online_memory(unsigned int nr, unsigned int order);
void pv_shim_offline_memory(unsigned int nr, unsigned int order);
domid_t get_initial_domain_id(void);
@@ -69,12 +69,12 @@ static inline void pv_shim_inject_evtchn(unsigned int port)
{
ASSERT_UNREACHABLE();
}
-static inline long pv_shim_cpu_up(void *data)
+static inline long cf_check pv_shim_cpu_up(void *data)
{
ASSERT_UNREACHABLE();
return 0;
}
-static inline long pv_shim_cpu_down(void *data)
+static inline long cf_check pv_shim_cpu_down(void *data)
{
ASSERT_UNREACHABLE();
return 0;
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index f7485f602efa..1747772d232e 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -57,10 +57,10 @@ int cpu_add(uint32_t apic_id, uint32_t acpi_id, uint32_t pxm);
void __stop_this_cpu(void);
-long cpu_up_helper(void *data);
-long cpu_down_helper(void *data);
+long cf_check cpu_up_helper(void *data);
+long cf_check cpu_down_helper(void *data);
-long core_parking_helper(void *data);
+long cf_check core_parking_helper(void *data);
bool core_parking_remove(unsigned int cpu);
uint32_t get_cur_idle_nums(void);
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index 84566bbfaa3d..f5d7adc1e802 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -45,7 +45,7 @@ struct resource_access {
xenpf_resource_entry_t *entries;
};
-long cpu_frequency_change_helper(void *);
+long cf_check cpu_frequency_change_helper(void *);
void check_resource_access(struct resource_access *);
void resource_access(void *);
@@ -59,7 +59,7 @@ DEFINE_SPINLOCK(xenpf_lock);
# undef guest_from_compat_handle
# define guest_from_compat_handle(x,y) ((x)=(y))
-long cpu_frequency_change_helper(void *data)
+long cf_check cpu_frequency_change_helper(void *data)
{
return cpu_frequency_change((uint64_t)data);
}
diff --git a/xen/arch/x86/pv/shim.c b/xen/arch/x86/pv/shim.c
index ae4d8913faa1..2ee290a3920d 100644
--- a/xen/arch/x86/pv/shim.c
+++ b/xen/arch/x86/pv/shim.c
@@ -845,7 +845,7 @@ int cf_check compat_grant_table_op(
#endif
#endif
-long pv_shim_cpu_up(void *data)
+long cf_check pv_shim_cpu_up(void *data)
{
struct vcpu *v = data;
struct domain *d = v->domain;
@@ -883,7 +883,7 @@ long pv_shim_cpu_up(void *data)
return 0;
}
-long pv_shim_cpu_down(void *data)
+long cf_check pv_shim_cpu_down(void *data)
{
struct vcpu *v = data;
long rc;
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index eef0f9c6cbf4..f4952a6bf9a5 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -399,7 +399,7 @@ void call_function_interrupt(struct cpu_user_regs *regs)
smp_call_function_interrupt();
}
-long cpu_up_helper(void *data)
+long cf_check cpu_up_helper(void *data)
{
unsigned int cpu = (unsigned long)data;
int ret = cpu_up(cpu);
@@ -422,7 +422,7 @@ long cpu_up_helper(void *data)
return ret;
}
-long cpu_down_helper(void *data)
+long cf_check cpu_down_helper(void *data)
{
int cpu = (unsigned long)data;
int ret = cpu_down(cpu);
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index aff52a13f373..1772f51f8f6e 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -79,7 +79,7 @@ static void l3_cache_get(void *arg)
l3_info->size = info.size / 1024; /* in KB unit */
}
-static long smt_up_down_helper(void *data)
+static long cf_check smt_up_down_helper(void *data)
{
bool up = (bool)data;
unsigned int cpu, sibling_mask = boot_cpu_data.x86_num_siblings - 1;
diff --git a/xen/common/core_parking.c b/xen/common/core_parking.c
index 44a907abfd7f..4afad04f2f68 100644
--- a/xen/common/core_parking.c
+++ b/xen/common/core_parking.c
@@ -169,7 +169,7 @@ static unsigned int core_parking_power(unsigned int event)
return cpu;
}
-long core_parking_helper(void *data)
+long cf_check core_parking_helper(void *data)
{
uint32_t idle_nums = (unsigned long)data;
unsigned int cpu;
diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index 3b223cd03d75..b222a5fd782e 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -395,7 +395,7 @@ void kexec_crash(enum crash_reason reason)
BUG();
}
-static long kexec_reboot(void *_image)
+static long cf_check kexec_reboot(void *_image)
{
struct kexec_image *image = _image;
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index e5cfb03b857e..b9d4babd0d8a 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -544,7 +544,7 @@ static int cpupool_unassign_cpu_start(struct cpupool *c, unsigned int cpu)
return ret;
}
-static long cpupool_unassign_cpu_helper(void *info)
+static long cf_check cpupool_unassign_cpu_helper(void *info)
{
struct cpupool *c = info;
long ret;
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 26/70] xen/iommu: CFI hardening
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (5 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 12/70] xen: CFI hardening for continue_hypercall_on_cpu() Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 27/70] xen/video: " Andrew Cooper
` (8 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
AMD's parse_ppr_log_entry() has no external callers, so becomes static.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3:
* Rebase over recent commits
---
xen/common/compat/memory.c | 4 +-
xen/drivers/passthrough/amd/iommu.h | 41 ++++++++--------
xen/drivers/passthrough/amd/iommu_init.c | 22 ++++-----
xen/drivers/passthrough/amd/iommu_intr.c | 18 +++----
xen/drivers/passthrough/amd/iommu_map.c | 22 +++++----
xen/drivers/passthrough/amd/pci_amd_iommu.c | 32 ++++++-------
xen/drivers/passthrough/pci.c | 7 +--
xen/drivers/passthrough/vtd/dmar.c | 3 +-
xen/drivers/passthrough/vtd/extern.h | 36 +++++++-------
xen/drivers/passthrough/vtd/intremap.c | 14 +++---
xen/drivers/passthrough/vtd/iommu.c | 73 +++++++++++++++--------------
xen/drivers/passthrough/vtd/qinval.c | 28 +++++------
12 files changed, 152 insertions(+), 148 deletions(-)
diff --git a/xen/common/compat/memory.c b/xen/common/compat/memory.c
index ec8ba54bb66e..077ded4a754c 100644
--- a/xen/common/compat/memory.c
+++ b/xen/common/compat/memory.c
@@ -23,8 +23,8 @@ struct get_reserved_device_memory {
unsigned int used_entries;
};
-static int get_reserved_device_memory(xen_pfn_t start, xen_ulong_t nr,
- u32 id, void *ctxt)
+static int cf_check get_reserved_device_memory(
+ xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt)
{
struct get_reserved_device_memory *grdm = ctxt;
uint32_t sbdf = PCI_SBDF3(grdm->map.dev.pci.seg, grdm->map.dev.pci.bus,
diff --git a/xen/drivers/passthrough/amd/iommu.h b/xen/drivers/passthrough/amd/iommu.h
index 99be9aafcc53..03811fedea57 100644
--- a/xen/drivers/passthrough/amd/iommu.h
+++ b/xen/drivers/passthrough/amd/iommu.h
@@ -236,25 +236,26 @@ int amd_iommu_init_late(void);
int amd_iommu_update_ivrs_mapping_acpi(void);
int cf_check iov_adjust_irq_affinities(void);
-int amd_iommu_quarantine_init(struct domain *d);
+int cf_check amd_iommu_quarantine_init(struct domain *d);
/* mapping functions */
-int __must_check amd_iommu_map_page(struct domain *d, dfn_t dfn,
- mfn_t mfn, unsigned int flags,
- unsigned int *flush_flags);
-int __must_check amd_iommu_unmap_page(struct domain *d, dfn_t dfn,
- unsigned int *flush_flags);
+int __must_check cf_check amd_iommu_map_page(
+ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags,
+ unsigned int *flush_flags);
+int __must_check cf_check amd_iommu_unmap_page(
+ struct domain *d, dfn_t dfn, unsigned int *flush_flags);
int __must_check amd_iommu_alloc_root(struct domain *d);
int amd_iommu_reserve_domain_unity_map(struct domain *domain,
const struct ivrs_unity_map *map,
unsigned int flag);
int amd_iommu_reserve_domain_unity_unmap(struct domain *d,
const struct ivrs_unity_map *map);
-int amd_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt);
-int __must_check amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
- unsigned long page_count,
- unsigned int flush_flags);
-int __must_check amd_iommu_flush_iotlb_all(struct domain *d);
+int cf_check amd_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt);
+int __must_check cf_check amd_iommu_flush_iotlb_pages(
+ struct domain *d, dfn_t dfn, unsigned long page_count,
+ unsigned int flush_flags);
+int __must_check cf_check amd_iommu_flush_iotlb_all(struct domain *d);
/* device table functions */
int get_dma_requestor_id(uint16_t seg, uint16_t bdf);
@@ -282,21 +283,21 @@ void amd_iommu_flush_all_caches(struct amd_iommu *iommu);
struct amd_iommu *find_iommu_for_device(int seg, int bdf);
/* interrupt remapping */
-bool iov_supports_xt(void);
+bool cf_check iov_supports_xt(void);
int amd_iommu_setup_ioapic_remapping(void);
void *amd_iommu_alloc_intremap_table(
const struct amd_iommu *, unsigned long **, unsigned int nr);
-int amd_iommu_free_intremap_table(
+int cf_check amd_iommu_free_intremap_table(
const struct amd_iommu *, struct ivrs_mappings *, uint16_t);
unsigned int amd_iommu_intremap_table_order(
const void *irt, const struct amd_iommu *iommu);
-void amd_iommu_ioapic_update_ire(
+void cf_check amd_iommu_ioapic_update_ire(
unsigned int apic, unsigned int reg, unsigned int value);
-unsigned int amd_iommu_read_ioapic_from_ire(
+unsigned int cf_check amd_iommu_read_ioapic_from_ire(
unsigned int apic, unsigned int reg);
-int amd_iommu_msi_msg_update_ire(
+int cf_check amd_iommu_msi_msg_update_ire(
struct msi_desc *msi_desc, struct msi_msg *msg);
-int amd_setup_hpet_msi(struct msi_desc *msi_desc);
+int cf_check amd_setup_hpet_msi(struct msi_desc *msi_desc);
void cf_check amd_iommu_dump_intremap_tables(unsigned char key);
extern struct ioapic_sbdf {
@@ -327,9 +328,9 @@ extern void *shared_intremap_table;
extern unsigned long *shared_intremap_inuse;
/* power management support */
-void amd_iommu_resume(void);
-int __must_check amd_iommu_suspend(void);
-void amd_iommu_crash_shutdown(void);
+void cf_check amd_iommu_resume(void);
+int __must_check cf_check amd_iommu_suspend(void);
+void cf_check amd_iommu_crash_shutdown(void);
/* guest iommu support */
#ifdef CONFIG_HVM
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 34a9e49f1c5a..06b4d2b1fea0 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -258,8 +258,8 @@ static void register_iommu_exclusion_range(struct amd_iommu *iommu)
writel(entry, iommu->mmio_base+IOMMU_EXCLUSION_BASE_LOW_OFFSET);
}
-static void set_iommu_event_log_control(struct amd_iommu *iommu,
- bool enable)
+static void cf_check set_iommu_event_log_control(
+ struct amd_iommu *iommu, bool enable)
{
/* Reset head and tail pointer manually before enablement */
if ( enable )
@@ -275,8 +275,8 @@ static void set_iommu_event_log_control(struct amd_iommu *iommu,
writeq(iommu->ctrl.raw, iommu->mmio_base + IOMMU_CONTROL_MMIO_OFFSET);
}
-static void set_iommu_ppr_log_control(struct amd_iommu *iommu,
- bool enable)
+static void cf_check set_iommu_ppr_log_control(
+ struct amd_iommu *iommu, bool enable)
{
/* Reset head and tail pointer manually before enablement */
if ( enable )
@@ -527,7 +527,7 @@ static hw_irq_controller iommu_x2apic_type = {
.set_affinity = set_x2apic_affinity,
};
-static void parse_event_log_entry(struct amd_iommu *iommu, u32 entry[])
+static void cf_check parse_event_log_entry(struct amd_iommu *iommu, u32 entry[])
{
u32 code;
static const char *const event_str[] = {
@@ -628,7 +628,7 @@ static void iommu_check_event_log(struct amd_iommu *iommu)
spin_unlock_irqrestore(&iommu->lock, flags);
}
-void parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[])
+static void cf_check parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[])
{
u16 device_id;
@@ -1243,7 +1243,7 @@ static int __init alloc_ivrs_mappings(u16 seg)
return 0;
}
-static int __init amd_iommu_setup_device_table(
+static int __init cf_check amd_iommu_setup_device_table(
u16 seg, struct ivrs_mappings *ivrs_mappings)
{
struct amd_iommu_dte *dt = IVRS_MAPPINGS_DEVTAB(ivrs_mappings);
@@ -1543,7 +1543,7 @@ static void invalidate_all_domain_pages(void)
amd_iommu_flush_all_pages(d);
}
-static int _invalidate_all_devices(
+static int cf_check _invalidate_all_devices(
u16 seg, struct ivrs_mappings *ivrs_mappings)
{
unsigned int bdf;
@@ -1569,14 +1569,14 @@ static void invalidate_all_devices(void)
iterate_ivrs_mappings(_invalidate_all_devices);
}
-int amd_iommu_suspend(void)
+int cf_check amd_iommu_suspend(void)
{
amd_iommu_crash_shutdown();
return 0;
}
-void amd_iommu_crash_shutdown(void)
+void cf_check amd_iommu_crash_shutdown(void)
{
struct amd_iommu *iommu;
@@ -1584,7 +1584,7 @@ void amd_iommu_crash_shutdown(void)
disable_iommu(iommu);
}
-void amd_iommu_resume(void)
+void cf_check amd_iommu_resume(void)
{
struct amd_iommu *iommu;
diff --git a/xen/drivers/passthrough/amd/iommu_intr.c b/xen/drivers/passthrough/amd/iommu_intr.c
index e7804413c7f4..cebf9ceca74e 100644
--- a/xen/drivers/passthrough/amd/iommu_intr.c
+++ b/xen/drivers/passthrough/amd/iommu_intr.c
@@ -349,7 +349,7 @@ static int update_intremap_entry_from_ioapic(
return 0;
}
-void amd_iommu_ioapic_update_ire(
+void cf_check amd_iommu_ioapic_update_ire(
unsigned int apic, unsigned int reg, unsigned int value)
{
struct IO_APIC_route_entry old_rte = { 0 };
@@ -455,7 +455,7 @@ void amd_iommu_ioapic_update_ire(
}
}
-unsigned int amd_iommu_read_ioapic_from_ire(
+unsigned int cf_check amd_iommu_read_ioapic_from_ire(
unsigned int apic, unsigned int reg)
{
unsigned int idx;
@@ -608,7 +608,7 @@ static struct amd_iommu *_find_iommu_for_device(int seg, int bdf)
return ERR_PTR(-EINVAL);
}
-int amd_iommu_msi_msg_update_ire(
+int cf_check amd_iommu_msi_msg_update_ire(
struct msi_desc *msi_desc, struct msi_msg *msg)
{
struct pci_dev *pdev = msi_desc->dev;
@@ -653,7 +653,7 @@ int amd_iommu_msi_msg_update_ire(
return rc;
}
-int amd_iommu_free_intremap_table(
+int cf_check amd_iommu_free_intremap_table(
const struct amd_iommu *iommu, struct ivrs_mappings *ivrs_mapping,
uint16_t bdf)
{
@@ -727,7 +727,7 @@ void *amd_iommu_alloc_intremap_table(
return tb;
}
-bool __init iov_supports_xt(void)
+bool __init cf_check iov_supports_xt(void)
{
unsigned int apic;
@@ -756,7 +756,7 @@ bool __init iov_supports_xt(void)
return true;
}
-int __init amd_setup_hpet_msi(struct msi_desc *msi_desc)
+int __init cf_check amd_setup_hpet_msi(struct msi_desc *msi_desc)
{
const struct amd_iommu *iommu;
spinlock_t *lock;
@@ -826,9 +826,9 @@ static void dump_intremap_table(const struct amd_iommu *iommu,
}
}
-static int dump_intremap_mapping(const struct amd_iommu *iommu,
- struct ivrs_mappings *ivrs_mapping,
- uint16_t unused)
+static int cf_check dump_intremap_mapping(
+ const struct amd_iommu *iommu, struct ivrs_mappings *ivrs_mapping,
+ uint16_t unused)
{
unsigned long flags;
diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c
index b0330157eab5..bf5df5fe5d9a 100644
--- a/xen/drivers/passthrough/amd/iommu_map.c
+++ b/xen/drivers/passthrough/amd/iommu_map.c
@@ -276,8 +276,9 @@ static int iommu_pde_from_dfn(struct domain *d, unsigned long dfn,
return 0;
}
-int amd_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
- unsigned int flags, unsigned int *flush_flags)
+int cf_check amd_iommu_map_page(
+ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags,
+ unsigned int *flush_flags)
{
struct domain_iommu *hd = dom_iommu(d);
int rc;
@@ -326,8 +327,8 @@ int amd_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
return 0;
}
-int amd_iommu_unmap_page(struct domain *d, dfn_t dfn,
- unsigned int *flush_flags)
+int cf_check amd_iommu_unmap_page(
+ struct domain *d, dfn_t dfn, unsigned int *flush_flags)
{
unsigned long pt_mfn = 0;
struct domain_iommu *hd = dom_iommu(d);
@@ -370,9 +371,9 @@ static unsigned long flush_count(unsigned long dfn, unsigned long page_count,
return end - start;
}
-int amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
- unsigned long page_count,
- unsigned int flush_flags)
+int cf_check amd_iommu_flush_iotlb_pages(
+ struct domain *d, dfn_t dfn, unsigned long page_count,
+ unsigned int flush_flags)
{
unsigned long dfn_l = dfn_x(dfn);
@@ -410,7 +411,7 @@ int amd_iommu_flush_iotlb_pages(struct domain *d, dfn_t dfn,
return 0;
}
-int amd_iommu_flush_iotlb_all(struct domain *d)
+int cf_check amd_iommu_flush_iotlb_all(struct domain *d)
{
amd_iommu_flush_all_pages(d);
@@ -462,7 +463,8 @@ int amd_iommu_reserve_domain_unity_unmap(struct domain *d,
return rc;
}
-int amd_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
+int cf_check amd_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt)
{
unsigned int seg = 0 /* XXX */, bdf;
const struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(seg);
@@ -537,7 +539,7 @@ int amd_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
return 0;
}
-int __init amd_iommu_quarantine_init(struct domain *d)
+int __init cf_check amd_iommu_quarantine_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
unsigned long end_gfn =
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 9642bba43a26..e57f555d00d1 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -197,7 +197,7 @@ int __init acpi_ivrs_init(void)
return 0;
}
-static int __init iov_detect(void)
+static int __init cf_check iov_detect(void)
{
if ( !iommu_enable && !iommu_intremap )
return 0;
@@ -217,7 +217,7 @@ static int __init iov_detect(void)
return 0;
}
-static int iov_enable_xt(void)
+static int cf_check iov_enable_xt(void)
{
int rc;
@@ -253,7 +253,7 @@ int amd_iommu_alloc_root(struct domain *d)
unsigned int __read_mostly amd_iommu_max_paging_mode = 6;
int __read_mostly amd_iommu_min_paging_mode = 1;
-static int amd_iommu_domain_init(struct domain *d)
+static int cf_check amd_iommu_domain_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -275,9 +275,9 @@ static int amd_iommu_domain_init(struct domain *d)
return 0;
}
-static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev);
+static int cf_check amd_iommu_add_device(u8 devfn, struct pci_dev *pdev);
-static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
+static void __hwdom_init cf_check amd_iommu_hwdom_init(struct domain *d)
{
const struct amd_iommu *iommu;
@@ -350,8 +350,9 @@ static void amd_iommu_disable_domain_device(const struct domain *domain,
spin_unlock_irqrestore(&iommu->lock, flags);
}
-static int reassign_device(struct domain *source, struct domain *target,
- u8 devfn, struct pci_dev *pdev)
+static int cf_check reassign_device(
+ struct domain *source, struct domain *target, u8 devfn,
+ struct pci_dev *pdev)
{
struct amd_iommu *iommu;
int bdf, rc;
@@ -404,9 +405,8 @@ static int reassign_device(struct domain *source, struct domain *target,
return 0;
}
-static int amd_iommu_assign_device(struct domain *d, u8 devfn,
- struct pci_dev *pdev,
- u32 flag)
+static int cf_check amd_iommu_assign_device(
+ struct domain *d, u8 devfn, struct pci_dev *pdev, u32 flag)
{
struct ivrs_mappings *ivrs_mappings = get_ivrs_mappings(pdev->seg);
int bdf = PCI_BDF2(pdev->bus, devfn);
@@ -435,7 +435,7 @@ static int amd_iommu_assign_device(struct domain *d, u8 devfn,
return rc;
}
-static void amd_iommu_clear_root_pgtable(struct domain *d)
+static void cf_check amd_iommu_clear_root_pgtable(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -444,13 +444,13 @@ static void amd_iommu_clear_root_pgtable(struct domain *d)
spin_unlock(&hd->arch.mapping_lock);
}
-static void amd_iommu_domain_destroy(struct domain *d)
+static void cf_check amd_iommu_domain_destroy(struct domain *d)
{
iommu_identity_map_teardown(d);
ASSERT(!dom_iommu(d)->arch.amd.root_table);
}
-static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
{
struct amd_iommu *iommu;
u16 bdf;
@@ -525,7 +525,7 @@ static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
return amd_iommu_setup_domain_device(pdev->domain, iommu, devfn, pdev);
}
-static int amd_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check amd_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
{
struct amd_iommu *iommu;
u16 bdf;
@@ -562,7 +562,7 @@ static int amd_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
return 0;
}
-static int amd_iommu_group_id(u16 seg, u8 bus, u8 devfn)
+static int cf_check amd_iommu_group_id(u16 seg, u8 bus, u8 devfn)
{
int bdf = PCI_BDF2(bus, devfn);
@@ -616,7 +616,7 @@ static void amd_dump_page_table_level(struct page_info *pg, int level,
unmap_domain_page(table_vaddr);
}
-static void amd_dump_page_tables(struct domain *d)
+static void cf_check amd_dump_page_tables(struct domain *d)
{
const struct domain_iommu *hd = dom_iommu(d);
diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 18af4e5088a0..22cb3872c22c 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1098,7 +1098,7 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn)
* scan pci devices to add all existed PCI devices to alldevs_list,
* and setup pci hierarchy in array bus2bridge.
*/
-static int __init _scan_pci_devices(struct pci_seg *pseg, void *arg)
+static int __init cf_check _scan_pci_devices(struct pci_seg *pseg, void *arg)
{
struct pci_dev *pdev;
int bus, dev, func;
@@ -1176,7 +1176,8 @@ static void __hwdom_init setup_one_hwdom_device(const struct setup_hwdom *ctxt,
ctxt->d->domain_id, err);
}
-static int __hwdom_init _setup_hwdom_pci_devices(struct pci_seg *pseg, void *arg)
+static int __hwdom_init cf_check _setup_hwdom_pci_devices(
+ struct pci_seg *pseg, void *arg)
{
struct setup_hwdom *ctxt = arg;
int bus, devfn;
@@ -1333,7 +1334,7 @@ bool_t pcie_aer_get_firmware_first(const struct pci_dev *pdev)
}
#endif
-static int _dump_pci_devices(struct pci_seg *pseg, void *arg)
+static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg)
{
struct pci_dev *pdev;
diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index b8e91f5be1ae..63f8642e126a 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -1046,7 +1046,8 @@ bool_t __init platform_supports_x2apic(void)
return cpu_has_x2apic && ((dmar_flags & mask) == ACPI_DMAR_INTR_REMAP);
}
-int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt)
+int cf_check intel_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt)
{
struct acpi_rmrr_unit *rmrr, *rmrr_cur = NULL;
unsigned int i;
diff --git a/xen/drivers/passthrough/vtd/extern.h b/xen/drivers/passthrough/vtd/extern.h
index ccf8df7be4f6..e6535548e1c1 100644
--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -33,9 +33,9 @@ void print_iommu_regs(struct acpi_drhd_unit *drhd);
void print_vtd_entries(struct vtd_iommu *iommu, int bus, int devfn, u64 gmfn);
keyhandler_fn_t cf_check vtd_dump_iommu_info;
-bool intel_iommu_supports_eim(void);
-int intel_iommu_enable_eim(void);
-void intel_iommu_disable_eim(void);
+bool cf_check intel_iommu_supports_eim(void);
+int cf_check intel_iommu_enable_eim(void);
+void cf_check intel_iommu_disable_eim(void);
int enable_qinval(struct vtd_iommu *iommu);
void disable_qinval(struct vtd_iommu *iommu);
@@ -51,15 +51,13 @@ int iommu_flush_iec_global(struct vtd_iommu *iommu);
int iommu_flush_iec_index(struct vtd_iommu *iommu, u8 im, u16 iidx);
void clear_fault_bits(struct vtd_iommu *iommu);
-int __must_check vtd_flush_context_reg(struct vtd_iommu *iommu, uint16_t did,
- uint16_t source_id,
- uint8_t function_mask, uint64_t type,
- bool flush_non_present_entry);
-int __must_check vtd_flush_iotlb_reg(struct vtd_iommu *iommu, uint16_t did,
- uint64_t addr, unsigned int size_order,
- uint64_t type,
- bool flush_non_present_entry,
- bool flush_dev_iotlb);
+int __must_check cf_check vtd_flush_context_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint16_t source_id,
+ uint8_t function_mask, uint64_t type, bool flush_non_present_entry);
+int __must_check cf_check vtd_flush_iotlb_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
+ unsigned int size_order, uint64_t type, bool flush_non_present_entry,
+ bool flush_dev_iotlb);
struct vtd_iommu *ioapic_to_iommu(unsigned int apic_id);
struct vtd_iommu *hpet_to_iommu(unsigned int hpet_id);
@@ -86,17 +84,19 @@ int domain_context_mapping_one(struct domain *domain, struct vtd_iommu *iommu,
u8 bus, u8 devfn, const struct pci_dev *);
int domain_context_unmap_one(struct domain *domain, struct vtd_iommu *iommu,
u8 bus, u8 devfn);
-int intel_iommu_get_reserved_device_memory(iommu_grdm_t *func, void *ctxt);
+int cf_check intel_iommu_get_reserved_device_memory(
+ iommu_grdm_t *func, void *ctxt);
-unsigned int io_apic_read_remap_rte(unsigned int apic, unsigned int reg);
-void io_apic_write_remap_rte(unsigned int apic,
- unsigned int reg, unsigned int value);
+unsigned int cf_check io_apic_read_remap_rte(
+ unsigned int apic, unsigned int reg);
+void cf_check io_apic_write_remap_rte(
+ unsigned int apic, unsigned int reg, unsigned int value);
struct msi_desc;
struct msi_msg;
-int msi_msg_write_remap_rte(struct msi_desc *, struct msi_msg *);
+int cf_check msi_msg_write_remap_rte(struct msi_desc *, struct msi_msg *);
-int intel_setup_hpet_msi(struct msi_desc *);
+int cf_check intel_setup_hpet_msi(struct msi_desc *);
int is_igd_vt_enabled_quirk(void);
bool is_azalia_tlb_enabled(const struct acpi_drhd_unit *);
diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c
index 01152f200664..e6ba89591b6f 100644
--- a/xen/drivers/passthrough/vtd/intremap.c
+++ b/xen/drivers/passthrough/vtd/intremap.c
@@ -142,7 +142,7 @@ static void set_hpet_source_id(unsigned int id, struct iremap_entry *ire)
set_ire_sid(ire, SVT_VERIFY_SID_SQ, SQ_13_IGNORE_3, hpetid_to_bdf(id));
}
-bool __init intel_iommu_supports_eim(void)
+bool __init cf_check intel_iommu_supports_eim(void)
{
struct acpi_drhd_unit *drhd;
unsigned int apic;
@@ -414,7 +414,7 @@ static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu,
return 0;
}
-unsigned int io_apic_read_remap_rte(
+unsigned int cf_check io_apic_read_remap_rte(
unsigned int apic, unsigned int reg)
{
unsigned int ioapic_pin = (reg - 0x10) / 2;
@@ -438,7 +438,7 @@ unsigned int io_apic_read_remap_rte(
return (*(((u32 *)&old_rte) + 0));
}
-void io_apic_write_remap_rte(
+void cf_check io_apic_write_remap_rte(
unsigned int apic, unsigned int reg, unsigned int value)
{
unsigned int ioapic_pin = (reg - 0x10) / 2;
@@ -639,7 +639,7 @@ static int msi_msg_to_remap_entry(
return 0;
}
-int msi_msg_write_remap_rte(
+int cf_check msi_msg_write_remap_rte(
struct msi_desc *msi_desc, struct msi_msg *msg)
{
struct pci_dev *pdev = msi_desc->dev;
@@ -651,7 +651,7 @@ int msi_msg_write_remap_rte(
: -EINVAL;
}
-int __init intel_setup_hpet_msi(struct msi_desc *msi_desc)
+int __init cf_check intel_setup_hpet_msi(struct msi_desc *msi_desc)
{
struct vtd_iommu *iommu = hpet_to_iommu(msi_desc->hpet_id);
unsigned long flags;
@@ -802,7 +802,7 @@ void disable_intremap(struct vtd_iommu *iommu)
* This function is used to enable Interrupt remapping when
* enable x2apic
*/
-int intel_iommu_enable_eim(void)
+int cf_check intel_iommu_enable_eim(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -856,7 +856,7 @@ int intel_iommu_enable_eim(void)
* This function is used to disable Interrupt remapping when
* suspend local apic
*/
-void intel_iommu_disable_eim(void)
+void cf_check intel_iommu_disable_eim(void)
{
struct acpi_drhd_unit *drhd;
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 42181e12be5a..1a1cf14785cb 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -59,7 +59,7 @@ static unsigned int __read_mostly nr_iommus;
static struct iommu_ops vtd_ops;
static struct tasklet vtd_fault_tasklet;
-static int setup_hwdom_device(u8 devfn, struct pci_dev *);
+static int cf_check setup_hwdom_device(u8 devfn, struct pci_dev *);
static void setup_hwdom_rmrr(struct domain *d);
static bool domid_mapping(const struct vtd_iommu *iommu)
@@ -426,9 +426,9 @@ static void iommu_flush_write_buffer(struct vtd_iommu *iommu)
}
/* return value determine if we need a write buffer flush */
-int vtd_flush_context_reg(struct vtd_iommu *iommu, uint16_t did,
- uint16_t source_id, uint8_t function_mask,
- uint64_t type, bool flush_non_present_entry)
+int cf_check vtd_flush_context_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint16_t source_id,
+ uint8_t function_mask, uint64_t type, bool flush_non_present_entry)
{
unsigned long flags;
@@ -493,9 +493,10 @@ static int __must_check iommu_flush_context_device(struct vtd_iommu *iommu,
}
/* return value determine if we need a write buffer flush */
-int vtd_flush_iotlb_reg(struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
- unsigned int size_order, uint64_t type,
- bool flush_non_present_entry, bool flush_dev_iotlb)
+int cf_check vtd_flush_iotlb_reg(
+ struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
+ unsigned int size_order, uint64_t type, bool flush_non_present_entry,
+ bool flush_dev_iotlb)
{
int tlb_offset = ecap_iotlb_offset(iommu->ecap);
uint64_t val = type | DMA_TLB_IVT;
@@ -704,10 +705,9 @@ static int __must_check iommu_flush_iotlb(struct domain *d, dfn_t dfn,
return ret;
}
-static int __must_check iommu_flush_iotlb_pages(struct domain *d,
- dfn_t dfn,
- unsigned long page_count,
- unsigned int flush_flags)
+static int __must_check cf_check iommu_flush_iotlb_pages(
+ struct domain *d, dfn_t dfn, unsigned long page_count,
+ unsigned int flush_flags)
{
ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN));
ASSERT(flush_flags);
@@ -716,7 +716,7 @@ static int __must_check iommu_flush_iotlb_pages(struct domain *d,
page_count);
}
-static int __must_check iommu_flush_iotlb_all(struct domain *d)
+static int __must_check cf_check iommu_flush_iotlb_all(struct domain *d)
{
return iommu_flush_iotlb(d, INVALID_DFN, 0, 0);
}
@@ -1345,7 +1345,7 @@ void __init iommu_free(struct acpi_drhd_unit *drhd)
agaw = 64; \
agaw; })
-static int intel_iommu_domain_init(struct domain *d)
+static int cf_check intel_iommu_domain_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -1359,7 +1359,7 @@ static int intel_iommu_domain_init(struct domain *d)
return 0;
}
-static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
+static void __hwdom_init cf_check intel_iommu_hwdom_init(struct domain *d)
{
struct acpi_drhd_unit *drhd;
@@ -1808,7 +1808,7 @@ static int domain_context_unmap(struct domain *domain, u8 devfn,
return ret;
}
-static void iommu_clear_root_pgtable(struct domain *d)
+static void cf_check iommu_clear_root_pgtable(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
@@ -1817,7 +1817,7 @@ static void iommu_clear_root_pgtable(struct domain *d)
spin_unlock(&hd->arch.mapping_lock);
}
-static void iommu_domain_teardown(struct domain *d)
+static void cf_check iommu_domain_teardown(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
const struct acpi_drhd_unit *drhd;
@@ -1835,9 +1835,9 @@ static void iommu_domain_teardown(struct domain *d)
XFREE(hd->arch.vtd.iommu_bitmap);
}
-static int __must_check intel_iommu_map_page(struct domain *d, dfn_t dfn,
- mfn_t mfn, unsigned int flags,
- unsigned int *flush_flags)
+static int __must_check cf_check intel_iommu_map_page(
+ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags,
+ unsigned int *flush_flags)
{
struct domain_iommu *hd = dom_iommu(d);
struct dma_pte *page, *pte, old, new = {};
@@ -1906,8 +1906,8 @@ static int __must_check intel_iommu_map_page(struct domain *d, dfn_t dfn,
return rc;
}
-static int __must_check intel_iommu_unmap_page(struct domain *d, dfn_t dfn,
- unsigned int *flush_flags)
+static int __must_check cf_check intel_iommu_unmap_page(
+ struct domain *d, dfn_t dfn, unsigned int *flush_flags)
{
/* Do nothing if VT-d shares EPT page table */
if ( iommu_use_hap_pt(d) )
@@ -1922,8 +1922,8 @@ static int __must_check intel_iommu_unmap_page(struct domain *d, dfn_t dfn,
return 0;
}
-static int intel_iommu_lookup_page(struct domain *d, dfn_t dfn, mfn_t *mfn,
- unsigned int *flags)
+static int cf_check intel_iommu_lookup_page(
+ struct domain *d, dfn_t dfn, mfn_t *mfn, unsigned int *flags)
{
struct domain_iommu *hd = dom_iommu(d);
struct dma_pte *page, val;
@@ -1975,7 +1975,7 @@ static int __init vtd_ept_page_compatible(struct vtd_iommu *iommu)
(ept_has_1gb(ept_cap) && opt_hap_1gb) <= cap_sps_1gb(vtd_cap);
}
-static int intel_iommu_add_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check intel_iommu_add_device(u8 devfn, struct pci_dev *pdev)
{
struct acpi_rmrr_unit *rmrr;
u16 bdf;
@@ -2018,7 +2018,7 @@ static int intel_iommu_add_device(u8 devfn, struct pci_dev *pdev)
return 0;
}
-static int intel_iommu_enable_device(struct pci_dev *pdev)
+static int cf_check intel_iommu_enable_device(struct pci_dev *pdev)
{
struct acpi_drhd_unit *drhd = acpi_find_matched_drhd_unit(pdev);
int ret = drhd ? ats_device(pdev, drhd) : -ENODEV;
@@ -2033,7 +2033,7 @@ static int intel_iommu_enable_device(struct pci_dev *pdev)
return ret >= 0 ? 0 : ret;
}
-static int intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
+static int cf_check intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
{
struct acpi_rmrr_unit *rmrr;
u16 bdf;
@@ -2060,7 +2060,8 @@ static int intel_iommu_remove_device(u8 devfn, struct pci_dev *pdev)
return domain_context_unmap(pdev->domain, devfn, pdev);
}
-static int __hwdom_init setup_hwdom_device(u8 devfn, struct pci_dev *pdev)
+static int __hwdom_init cf_check setup_hwdom_device(
+ u8 devfn, struct pci_dev *pdev)
{
return domain_context_mapping(pdev->domain, devfn, pdev);
}
@@ -2266,7 +2267,7 @@ static struct iommu_state {
uint32_t fectl;
} *__read_mostly iommu_state;
-static int __init vtd_setup(void)
+static int __init cf_check vtd_setup(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2401,7 +2402,7 @@ static int __init vtd_setup(void)
return ret;
}
-static int reassign_device_ownership(
+static int cf_check reassign_device_ownership(
struct domain *source,
struct domain *target,
u8 devfn, struct pci_dev *pdev)
@@ -2479,7 +2480,7 @@ static int reassign_device_ownership(
return ret;
}
-static int intel_iommu_assign_device(
+static int cf_check intel_iommu_assign_device(
struct domain *d, u8 devfn, struct pci_dev *pdev, u32 flag)
{
struct domain *s = pdev->domain;
@@ -2561,7 +2562,7 @@ static int intel_iommu_assign_device(
return ret;
}
-static int intel_iommu_group_id(u16 seg, u8 bus, u8 devfn)
+static int cf_check intel_iommu_group_id(u16 seg, u8 bus, u8 devfn)
{
u8 secbus;
@@ -2571,7 +2572,7 @@ static int intel_iommu_group_id(u16 seg, u8 bus, u8 devfn)
return PCI_BDF2(bus, devfn);
}
-static int __must_check vtd_suspend(void)
+static int __must_check cf_check vtd_suspend(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2614,7 +2615,7 @@ static int __must_check vtd_suspend(void)
return 0;
}
-static void vtd_crash_shutdown(void)
+static void cf_check vtd_crash_shutdown(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2635,7 +2636,7 @@ static void vtd_crash_shutdown(void)
}
}
-static void vtd_resume(void)
+static void cf_check vtd_resume(void)
{
struct acpi_drhd_unit *drhd;
struct vtd_iommu *iommu;
@@ -2713,7 +2714,7 @@ static void vtd_dump_page_table_level(paddr_t pt_maddr, int level, paddr_t gpa,
unmap_vtd_domain_page(pt_vaddr);
}
-static void vtd_dump_page_tables(struct domain *d)
+static void cf_check vtd_dump_page_tables(struct domain *d)
{
const struct domain_iommu *hd = dom_iommu(d);
@@ -2723,7 +2724,7 @@ static void vtd_dump_page_tables(struct domain *d)
agaw_to_level(hd->arch.vtd.agaw), 0, 0);
}
-static int __init intel_iommu_quarantine_init(struct domain *d)
+static int __init cf_check intel_iommu_quarantine_init(struct domain *d)
{
struct domain_iommu *hd = dom_iommu(d);
struct page_info *pg;
diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c
index 9f291f47e518..beeb65f0deec 100644
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -322,9 +322,9 @@ int iommu_flush_iec_index(struct vtd_iommu *iommu, u8 im, u16 iidx)
return queue_invalidate_iec_sync(iommu, IEC_INDEX_INVL, im, iidx);
}
-static int __must_check flush_context_qi(struct vtd_iommu *iommu, u16 did,
- u16 sid, u8 fm, u64 type,
- bool flush_non_present_entry)
+static int __must_check cf_check flush_context_qi(
+ struct vtd_iommu *iommu, u16 did, u16 sid, u8 fm, u64 type,
+ bool flush_non_present_entry)
{
ASSERT(iommu->qinval_maddr);
@@ -346,11 +346,9 @@ static int __must_check flush_context_qi(struct vtd_iommu *iommu, u16 did,
type >> DMA_CCMD_INVL_GRANU_OFFSET);
}
-static int __must_check flush_iotlb_qi(struct vtd_iommu *iommu, u16 did,
- u64 addr,
- unsigned int size_order, u64 type,
- bool flush_non_present_entry,
- bool flush_dev_iotlb)
+static int __must_check cf_check flush_iotlb_qi(
+ struct vtd_iommu *iommu, u16 did, u64 addr, unsigned int size_order,
+ u64 type, bool flush_non_present_entry, bool flush_dev_iotlb)
{
u8 dr = 0, dw = 0;
int ret = 0, rc;
@@ -461,18 +459,18 @@ int enable_qinval(struct vtd_iommu *iommu)
return 0;
}
-static int vtd_flush_context_noop(struct vtd_iommu *iommu, uint16_t did,
- uint16_t source_id, uint8_t function_mask,
- uint64_t type, bool flush_non_present_entry)
+static int cf_check vtd_flush_context_noop(
+ struct vtd_iommu *iommu, uint16_t did, uint16_t source_id,
+ uint8_t function_mask, uint64_t type, bool flush_non_present_entry)
{
WARN();
return -EIO;
}
-static int vtd_flush_iotlb_noop(struct vtd_iommu *iommu, uint16_t did,
- uint64_t addr, unsigned int size_order,
- uint64_t type, bool flush_non_present_entry,
- bool flush_dev_iotlb)
+static int cf_check vtd_flush_iotlb_noop(
+ struct vtd_iommu *iommu, uint16_t did, uint64_t addr,
+ unsigned int size_order, uint64_t type, bool flush_non_present_entry,
+ bool flush_dev_iotlb)
{
WARN();
return -EIO;
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 27/70] xen/video: CFI hardening
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (6 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 26/70] xen/iommu: CFI hardening Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 29/70] xen/misc: " Andrew Cooper
` (7 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3:
* Rebase over recent commits
---
xen/drivers/video/lfb.c | 4 ++--
xen/drivers/video/lfb.h | 4 ++--
xen/drivers/video/vesa.c | 4 ++--
xen/drivers/video/vga.c | 6 +++---
4 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/xen/drivers/video/lfb.c b/xen/drivers/video/lfb.c
index 9254b5e9022a..a6fb837974f7 100644
--- a/xen/drivers/video/lfb.c
+++ b/xen/drivers/video/lfb.c
@@ -53,7 +53,7 @@ static void lfb_show_line(
}
/* Fast mode which redraws all modified parts of a 2D text buffer. */
-void lfb_redraw_puts(const char *s, size_t nr)
+void cf_check lfb_redraw_puts(const char *s, size_t nr)
{
unsigned int i, min_redraw_y = lfb.ypos;
@@ -98,7 +98,7 @@ void lfb_redraw_puts(const char *s, size_t nr)
}
/* Slower line-based scroll mode which interacts better with dom0. */
-void lfb_scroll_puts(const char *s, size_t nr)
+void cf_check lfb_scroll_puts(const char *s, size_t nr)
{
unsigned int i;
diff --git a/xen/drivers/video/lfb.h b/xen/drivers/video/lfb.h
index e743ccdd6b11..42161402d611 100644
--- a/xen/drivers/video/lfb.h
+++ b/xen/drivers/video/lfb.h
@@ -35,8 +35,8 @@ struct lfb_prop {
unsigned int text_rows;
};
-void lfb_redraw_puts(const char *s, size_t nr);
-void lfb_scroll_puts(const char *s, size_t nr);
+void cf_check lfb_redraw_puts(const char *s, size_t nr);
+void cf_check lfb_scroll_puts(const char *s, size_t nr);
void lfb_carriage_return(void);
void lfb_free(void);
diff --git a/xen/drivers/video/vesa.c b/xen/drivers/video/vesa.c
index c8f81a5cc5fc..c41f6b8d4028 100644
--- a/xen/drivers/video/vesa.c
+++ b/xen/drivers/video/vesa.c
@@ -17,7 +17,7 @@
#define vlfb_info vga_console_info.u.vesa_lfb
-static void lfb_flush(void);
+static void cf_check lfb_flush(void);
static unsigned char *__read_mostly lfb;
static const struct font_desc *__initdata font;
@@ -140,7 +140,7 @@ void __init vesa_init(void)
video_puts = lfb_redraw_puts;
}
-static void lfb_flush(void)
+static void cf_check lfb_flush(void)
{
__asm__ __volatile__ ("sfence" : : : "memory");
}
diff --git a/xen/drivers/video/vga.c b/xen/drivers/video/vga.c
index 5e58f83c97ff..e624ebff4f8c 100644
--- a/xen/drivers/video/vga.c
+++ b/xen/drivers/video/vga.c
@@ -19,8 +19,8 @@ static int vgacon_keep;
static unsigned int xpos, ypos;
static unsigned char *video;
-static void vga_text_puts(const char *s, size_t nr);
-static void vga_noop_puts(const char *s, size_t nr) {}
+static void cf_check vga_text_puts(const char *s, size_t nr);
+static void cf_check vga_noop_puts(const char *s, size_t nr) {}
void (*video_puts)(const char *, size_t nr) = vga_noop_puts;
/*
@@ -179,7 +179,7 @@ void __init video_endboot(void)
}
}
-static void vga_text_puts(const char *s, size_t nr)
+static void cf_check vga_text_puts(const char *s, size_t nr)
{
for ( ; nr > 0; nr--, s++ )
{
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 29/70] xen/misc: CFI hardening
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (7 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 27/70] xen/video: " Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-23 10:25 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 33/70] x86/emul: " Andrew Cooper
` (6 subsequent siblings)
15 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3
* Annotations for gcov
---
xen/arch/x86/mm.c | 6 ++++--
xen/arch/x86/setup.c | 4 ++--
xen/common/coverage/gcov.c | 8 ++++----
xen/common/domain.c | 2 +-
xen/common/gdbstub.c | 5 ++---
xen/common/livepatch.c | 7 +++----
xen/common/memory.c | 4 ++--
xen/common/page_alloc.c | 2 +-
xen/common/radix-tree.c | 4 ++--
xen/common/rangeset.c | 2 +-
xen/common/spinlock.c | 6 +++---
xen/common/vm_event.c | 6 +++---
xen/common/xmalloc_tlsf.c | 4 ++--
xen/drivers/passthrough/amd/iommu_init.c | 2 +-
xen/include/xen/domain.h | 2 +-
15 files changed, 32 insertions(+), 32 deletions(-)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index a1b8737096c4..0665095d2309 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -835,7 +835,8 @@ struct mmio_emul_range_ctxt {
unsigned long mfn;
};
-static int print_mmio_emul_range(unsigned long s, unsigned long e, void *arg)
+static int cf_check print_mmio_emul_range(
+ unsigned long s, unsigned long e, void *arg)
{
const struct mmio_emul_range_ctxt *ctxt = arg;
@@ -4606,7 +4607,8 @@ static int _handle_iomem_range(unsigned long s, unsigned long e,
return 0;
}
-static int handle_iomem_range(unsigned long s, unsigned long e, void *p)
+static int cf_check handle_iomem_range(
+ unsigned long s, unsigned long e, void *p)
{
int err = 0;
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 3a4ec1fcfd04..a9a371336b36 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -2021,8 +2021,8 @@ int __hwdom_init xen_in_range(unsigned long mfn)
return 0;
}
-static int __hwdom_init io_bitmap_cb(unsigned long s, unsigned long e,
- void *ctx)
+static int __hwdom_init cf_check io_bitmap_cb(
+ unsigned long s, unsigned long e, void *ctx)
{
struct domain *d = ctx;
unsigned int i;
diff --git a/xen/common/coverage/gcov.c b/xen/common/coverage/gcov.c
index 3cc98728bfce..327bf8d646c0 100644
--- a/xen/common/coverage/gcov.c
+++ b/xen/common/coverage/gcov.c
@@ -120,7 +120,7 @@ static int gcov_info_dump_payload(const struct gcov_info *info,
}
-static uint32_t gcov_get_size(void)
+static uint32_t cf_check gcov_get_size(void)
{
uint32_t total_size = sizeof(uint32_t); /* Magic number XCOV */
struct gcov_info *info = NULL;
@@ -140,7 +140,7 @@ static uint32_t gcov_get_size(void)
return total_size;
}
-static void gcov_reset_all_counters(void)
+static void cf_check gcov_reset_all_counters(void)
{
struct gcov_info *info = NULL;
@@ -172,8 +172,8 @@ static int gcov_dump_one_record(const struct gcov_info *info,
return gcov_info_dump_payload(info, buffer, off);
}
-static int gcov_dump_all(XEN_GUEST_HANDLE_PARAM(char) buffer,
- uint32_t *buffer_size)
+static int cf_check gcov_dump_all(
+ XEN_GUEST_HANDLE_PARAM(char) buffer, uint32_t *buffer_size)
{
uint32_t off;
uint32_t magic = XEN_GCOV_FORMAT_MAGIC;
diff --git a/xen/common/domain.c b/xen/common/domain.c
index f3d06df76c33..351029f8b239 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -1803,7 +1803,7 @@ static void cf_check _free_pirq_struct(struct rcu_head *head)
xfree(container_of(head, struct pirq, rcu_head));
}
-void free_pirq_struct(void *ptr)
+void cf_check free_pirq_struct(void *ptr)
{
struct pirq *pirq = ptr;
diff --git a/xen/common/gdbstub.c b/xen/common/gdbstub.c
index 079c3ca9616a..d6872721dc0d 100644
--- a/xen/common/gdbstub.c
+++ b/xen/common/gdbstub.c
@@ -69,7 +69,7 @@ static void gdb_smp_resume(void);
static char __initdata opt_gdb[30];
string_param("gdb", opt_gdb);
-static void gdbstub_console_puts(const char *str, size_t nr);
+static void cf_check gdbstub_console_puts(const char *str, size_t nr);
/* value <-> char (de)serialzers */
static char
@@ -546,8 +546,7 @@ __gdb_ctx = {
};
static struct gdb_context *gdb_ctx = &__gdb_ctx;
-static void
-gdbstub_console_puts(const char *str, size_t nr)
+static void cf_check gdbstub_console_puts(const char *str, size_t nr)
{
const char *p;
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index e8714920dc8f..ec301a9f120c 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -157,10 +157,9 @@ unsigned long livepatch_symbols_lookup_by_name(const char *symname)
return 0;
}
-static const char *livepatch_symbols_lookup(unsigned long addr,
- unsigned long *symbolsize,
- unsigned long *offset,
- char *namebuf)
+static const char *cf_check livepatch_symbols_lookup(
+ unsigned long addr, unsigned long *symbolsize, unsigned long *offset,
+ char *namebuf)
{
const struct payload *data;
unsigned int i, best;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index ede45c4af9db..69b0cd1e50de 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1051,8 +1051,8 @@ struct get_reserved_device_memory {
unsigned int used_entries;
};
-static int get_reserved_device_memory(xen_pfn_t start, xen_ulong_t nr,
- u32 id, void *ctxt)
+static int cf_check get_reserved_device_memory(
+ xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt)
{
struct get_reserved_device_memory *grdm = ctxt;
uint32_t sbdf = PCI_SBDF3(grdm->map.dev.pci.seg, grdm->map.dev.pci.bus,
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 3caf5c954b24..46357182375a 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1238,7 +1238,7 @@ struct scrub_wait_state {
bool drop;
};
-static void scrub_continue(void *data)
+static void cf_check scrub_continue(void *data)
{
struct scrub_wait_state *st = data;
diff --git a/xen/common/radix-tree.c b/xen/common/radix-tree.c
index 33b47748ae49..adc3034222dc 100644
--- a/xen/common/radix-tree.c
+++ b/xen/common/radix-tree.c
@@ -52,7 +52,7 @@ struct rcu_node {
struct rcu_head rcu_head;
};
-static struct radix_tree_node *rcu_node_alloc(void *arg)
+static struct radix_tree_node *cf_check rcu_node_alloc(void *arg)
{
struct rcu_node *rcu_node = xmalloc(struct rcu_node);
return rcu_node ? &rcu_node->node : NULL;
@@ -65,7 +65,7 @@ static void cf_check _rcu_node_free(struct rcu_head *head)
xfree(rcu_node);
}
-static void rcu_node_free(struct radix_tree_node *node, void *arg)
+static void cf_check rcu_node_free(struct radix_tree_node *node, void *arg)
{
struct rcu_node *rcu_node = container_of(node, struct rcu_node, node);
call_rcu(&rcu_node->rcu_head, _rcu_node_free);
diff --git a/xen/common/rangeset.c b/xen/common/rangeset.c
index 885b6b15c229..a6ef2640462a 100644
--- a/xen/common/rangeset.c
+++ b/xen/common/rangeset.c
@@ -384,7 +384,7 @@ int rangeset_consume_ranges(struct rangeset *r,
return rc;
}
-static int merge(unsigned long s, unsigned long e, void *data)
+static int cf_check merge(unsigned long s, unsigned long e, void *data)
{
struct rangeset *r = data;
diff --git a/xen/common/spinlock.c b/xen/common/spinlock.c
index 25bfbf3c47f7..62c83aaa6a73 100644
--- a/xen/common/spinlock.c
+++ b/xen/common/spinlock.c
@@ -375,7 +375,7 @@ static void spinlock_profile_iterate(lock_profile_subfunc *sub, void *par)
spin_unlock(&lock_profile_lock);
}
-static void spinlock_profile_print_elem(struct lock_profile *data,
+static void cf_check spinlock_profile_print_elem(struct lock_profile *data,
int32_t type, int32_t idx, void *par)
{
struct spinlock *lock = data->lock;
@@ -404,7 +404,7 @@ void cf_check spinlock_profile_printall(unsigned char key)
spinlock_profile_iterate(spinlock_profile_print_elem, NULL);
}
-static void spinlock_profile_reset_elem(struct lock_profile *data,
+static void cf_check spinlock_profile_reset_elem(struct lock_profile *data,
int32_t type, int32_t idx, void *par)
{
data->lock_cnt = 0;
@@ -428,7 +428,7 @@ typedef struct {
int rc;
} spinlock_profile_ucopy_t;
-static void spinlock_profile_ucopy_elem(struct lock_profile *data,
+static void cf_check spinlock_profile_ucopy_elem(struct lock_profile *data,
int32_t type, int32_t idx, void *par)
{
spinlock_profile_ucopy_t *p = par;
diff --git a/xen/common/vm_event.c b/xen/common/vm_event.c
index 70ab3ba406ff..84cf52636bc4 100644
--- a/xen/common/vm_event.c
+++ b/xen/common/vm_event.c
@@ -523,21 +523,21 @@ int __vm_event_claim_slot(struct domain *d, struct vm_event_domain *ved,
#ifdef CONFIG_MEM_PAGING
/* Registered with Xen-bound event channel for incoming notifications. */
-static void mem_paging_notification(struct vcpu *v, unsigned int port)
+static void cf_check mem_paging_notification(struct vcpu *v, unsigned int port)
{
vm_event_resume(v->domain, v->domain->vm_event_paging);
}
#endif
/* Registered with Xen-bound event channel for incoming notifications. */
-static void monitor_notification(struct vcpu *v, unsigned int port)
+static void cf_check monitor_notification(struct vcpu *v, unsigned int port)
{
vm_event_resume(v->domain, v->domain->vm_event_monitor);
}
#ifdef CONFIG_MEM_SHARING
/* Registered with Xen-bound event channel for incoming notifications. */
-static void mem_sharing_notification(struct vcpu *v, unsigned int port)
+static void cf_check mem_sharing_notification(struct vcpu *v, unsigned int port)
{
vm_event_resume(v->domain, v->domain->vm_event_share);
}
diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c
index e3f6886e6b62..d2ad909502d0 100644
--- a/xen/common/xmalloc_tlsf.c
+++ b/xen/common/xmalloc_tlsf.c
@@ -512,13 +512,13 @@ int xmem_pool_maxalloc(struct xmem_pool *pool)
static struct xmem_pool *xenpool;
-static void *xmalloc_pool_get(unsigned long size)
+static void *cf_check xmalloc_pool_get(unsigned long size)
{
ASSERT(size == PAGE_SIZE);
return alloc_xenheap_page();
}
-static void xmalloc_pool_put(void *p)
+static void cf_check xmalloc_pool_put(void *p)
{
free_xenheap_page(p);
}
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 06b4d2b1fea0..cebcd68a6c04 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1073,7 +1073,7 @@ static void * __init allocate_ppr_log(struct amd_iommu *iommu)
#define IVRS_MAPPINGS_DEVTAB(m) (m)[ivrs_bdf_entries].intremap_table
/* Gets passed to radix_tree_destroy(), so its param needs to be void *. */
-static void __init free_ivrs_mapping_callback(void *ptr)
+static void __init cf_check free_ivrs_mapping_callback(void *ptr)
{
const struct ivrs_mappings *ivrs_mappings = ptr;
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index 24eb4cc7d37e..1c3c88a14d6f 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -52,7 +52,7 @@ void free_vcpu_struct(struct vcpu *v);
#ifndef alloc_pirq_struct
struct pirq *alloc_pirq_struct(struct domain *);
#endif
-void free_pirq_struct(void *);
+void cf_check free_pirq_struct(void *);
/*
* Initialise/destroy arch-specific details of a VCPU.
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 33/70] x86/emul: CFI hardening
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (8 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 29/70] xen/misc: " Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 46/70] x86/logdirty: " Andrew Cooper
` (5 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
pv_emul_is_mem_write() is only used in a single file. Move it out of its
header file, so it doesn't risk being duplicated in multiple translation
units.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v2:
* Correct details in commit message.
v2:
* Reword again.
---
xen/arch/x86/hvm/emulate.c | 72 +++++++++++++++++-----------------
xen/arch/x86/hvm/hvm.c | 8 ++--
xen/arch/x86/hvm/svm/svm.c | 4 +-
xen/arch/x86/include/asm/hvm/emulate.h | 8 ++--
xen/arch/x86/include/asm/mm.h | 16 +++-----
xen/arch/x86/mm.c | 4 +-
xen/arch/x86/mm/shadow/hvm.c | 8 ++--
xen/arch/x86/pv/emul-gate-op.c | 9 +++--
xen/arch/x86/pv/emul-priv-op.c | 64 +++++++++++++++---------------
xen/arch/x86/pv/emulate.h | 7 ----
xen/arch/x86/pv/ro-page-fault.c | 31 +++++++++------
xen/arch/x86/x86_emulate.c | 21 +++++-----
xen/arch/x86/x86_emulate/x86_emulate.c | 10 ++---
xen/arch/x86/x86_emulate/x86_emulate.h | 33 ++++++++--------
14 files changed, 148 insertions(+), 147 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 39dac7fd9d6d..e8d510e0be91 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -1272,7 +1272,7 @@ static int __hvmemul_read(
return linear_read(addr, bytes, p_data, pfec, hvmemul_ctxt);
}
-static int hvmemul_read(
+static int cf_check hvmemul_read(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1290,7 +1290,7 @@ static int hvmemul_read(
container_of(ctxt, struct hvm_emulate_ctxt, ctxt));
}
-int hvmemul_insn_fetch(
+int cf_check hvmemul_insn_fetch(
unsigned long offset,
void *p_data,
unsigned int bytes,
@@ -1336,7 +1336,7 @@ int hvmemul_insn_fetch(
return X86EMUL_OKAY;
}
-static int hvmemul_write(
+static int cf_check hvmemul_write(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1384,7 +1384,7 @@ static int hvmemul_write(
return X86EMUL_OKAY;
}
-static int hvmemul_rmw(
+static int cf_check hvmemul_rmw(
enum x86_segment seg,
unsigned long offset,
unsigned int bytes,
@@ -1437,7 +1437,7 @@ static int hvmemul_rmw(
return rc;
}
-static int hvmemul_blk(
+static int cf_check hvmemul_blk(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1478,7 +1478,7 @@ static int hvmemul_blk(
return rc;
}
-static int hvmemul_write_discard(
+static int cf_check hvmemul_write_discard(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -1489,7 +1489,7 @@ static int hvmemul_write_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_ins_discard(
+static int cf_check hvmemul_rep_ins_discard(
uint16_t src_port,
enum x86_segment dst_seg,
unsigned long dst_offset,
@@ -1500,7 +1500,7 @@ static int hvmemul_rep_ins_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_movs_discard(
+static int cf_check hvmemul_rep_movs_discard(
enum x86_segment src_seg,
unsigned long src_offset,
enum x86_segment dst_seg,
@@ -1512,7 +1512,7 @@ static int hvmemul_rep_movs_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_stos_discard(
+static int cf_check hvmemul_rep_stos_discard(
void *p_data,
enum x86_segment seg,
unsigned long offset,
@@ -1523,7 +1523,7 @@ static int hvmemul_rep_stos_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_rep_outs_discard(
+static int cf_check hvmemul_rep_outs_discard(
enum x86_segment src_seg,
unsigned long src_offset,
uint16_t dst_port,
@@ -1534,7 +1534,7 @@ static int hvmemul_rep_outs_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_cmpxchg_discard(
+static int cf_check hvmemul_cmpxchg_discard(
enum x86_segment seg,
unsigned long offset,
void *p_old,
@@ -1546,7 +1546,7 @@ static int hvmemul_cmpxchg_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_read_io_discard(
+static int cf_check hvmemul_read_io_discard(
unsigned int port,
unsigned int bytes,
unsigned long *val,
@@ -1555,7 +1555,7 @@ static int hvmemul_read_io_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_write_io_discard(
+static int cf_check hvmemul_write_io_discard(
unsigned int port,
unsigned int bytes,
unsigned long val,
@@ -1564,7 +1564,7 @@ static int hvmemul_write_io_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_write_msr_discard(
+static int cf_check hvmemul_write_msr_discard(
unsigned int reg,
uint64_t val,
struct x86_emulate_ctxt *ctxt)
@@ -1572,7 +1572,7 @@ static int hvmemul_write_msr_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_cache_op_discard(
+static int cf_check hvmemul_cache_op_discard(
enum x86emul_cache_op op,
enum x86_segment seg,
unsigned long offset,
@@ -1581,7 +1581,7 @@ static int hvmemul_cache_op_discard(
return X86EMUL_OKAY;
}
-static int hvmemul_cmpxchg(
+static int cf_check hvmemul_cmpxchg(
enum x86_segment seg,
unsigned long offset,
void *p_old,
@@ -1675,7 +1675,7 @@ static int hvmemul_cmpxchg(
return rc;
}
-static int hvmemul_validate(
+static int cf_check hvmemul_validate(
const struct x86_emulate_state *state,
struct x86_emulate_ctxt *ctxt)
{
@@ -1688,7 +1688,7 @@ static int hvmemul_validate(
? X86EMUL_OKAY : X86EMUL_UNHANDLEABLE;
}
-static int hvmemul_rep_ins(
+static int cf_check hvmemul_rep_ins(
uint16_t src_port,
enum x86_segment dst_seg,
unsigned long dst_offset,
@@ -1766,7 +1766,7 @@ static int hvmemul_rep_outs_set_context(
return rc;
}
-static int hvmemul_rep_outs(
+static int cf_check hvmemul_rep_outs(
enum x86_segment src_seg,
unsigned long src_offset,
uint16_t dst_port,
@@ -1807,7 +1807,7 @@ static int hvmemul_rep_outs(
!!(ctxt->regs->eflags & X86_EFLAGS_DF), gpa);
}
-static int hvmemul_rep_movs(
+static int cf_check hvmemul_rep_movs(
enum x86_segment src_seg,
unsigned long src_offset,
enum x86_segment dst_seg,
@@ -1977,7 +1977,7 @@ static int hvmemul_rep_movs(
return X86EMUL_UNHANDLEABLE;
}
-static int hvmemul_rep_stos(
+static int cf_check hvmemul_rep_stos(
void *p_data,
enum x86_segment seg,
unsigned long offset,
@@ -2105,7 +2105,7 @@ static int hvmemul_rep_stos(
}
}
-static int hvmemul_read_segment(
+static int cf_check hvmemul_read_segment(
enum x86_segment seg,
struct segment_register *reg,
struct x86_emulate_ctxt *ctxt)
@@ -2122,7 +2122,7 @@ static int hvmemul_read_segment(
return X86EMUL_OKAY;
}
-static int hvmemul_write_segment(
+static int cf_check hvmemul_write_segment(
enum x86_segment seg,
const struct segment_register *reg,
struct x86_emulate_ctxt *ctxt)
@@ -2141,7 +2141,7 @@ static int hvmemul_write_segment(
return X86EMUL_OKAY;
}
-static int hvmemul_read_io(
+static int cf_check hvmemul_read_io(
unsigned int port,
unsigned int bytes,
unsigned long *val,
@@ -2158,7 +2158,7 @@ static int hvmemul_read_io(
return hvmemul_do_pio_buffer(port, bytes, IOREQ_READ, val);
}
-static int hvmemul_write_io(
+static int cf_check hvmemul_write_io(
unsigned int port,
unsigned int bytes,
unsigned long val,
@@ -2167,7 +2167,7 @@ static int hvmemul_write_io(
return hvmemul_do_pio_buffer(port, bytes, IOREQ_WRITE, &val);
}
-static int hvmemul_read_cr(
+static int cf_check hvmemul_read_cr(
unsigned int reg,
unsigned long *val,
struct x86_emulate_ctxt *ctxt)
@@ -2188,7 +2188,7 @@ static int hvmemul_read_cr(
return X86EMUL_UNHANDLEABLE;
}
-static int hvmemul_write_cr(
+static int cf_check hvmemul_write_cr(
unsigned int reg,
unsigned long val,
struct x86_emulate_ctxt *ctxt)
@@ -2232,7 +2232,7 @@ static int hvmemul_write_cr(
return rc;
}
-static int hvmemul_read_xcr(
+static int cf_check hvmemul_read_xcr(
unsigned int reg,
uint64_t *val,
struct x86_emulate_ctxt *ctxt)
@@ -2245,7 +2245,7 @@ static int hvmemul_read_xcr(
return rc;
}
-static int hvmemul_write_xcr(
+static int cf_check hvmemul_write_xcr(
unsigned int reg,
uint64_t val,
struct x86_emulate_ctxt *ctxt)
@@ -2255,7 +2255,7 @@ static int hvmemul_write_xcr(
return x86emul_write_xcr(reg, val, ctxt);
}
-static int hvmemul_read_msr(
+static int cf_check hvmemul_read_msr(
unsigned int reg,
uint64_t *val,
struct x86_emulate_ctxt *ctxt)
@@ -2268,7 +2268,7 @@ static int hvmemul_read_msr(
return rc;
}
-static int hvmemul_write_msr(
+static int cf_check hvmemul_write_msr(
unsigned int reg,
uint64_t val,
struct x86_emulate_ctxt *ctxt)
@@ -2281,7 +2281,7 @@ static int hvmemul_write_msr(
return rc;
}
-static int hvmemul_cache_op(
+static int cf_check hvmemul_cache_op(
enum x86emul_cache_op op,
enum x86_segment seg,
unsigned long offset,
@@ -2353,7 +2353,7 @@ static int hvmemul_cache_op(
return X86EMUL_OKAY;
}
-static int hvmemul_get_fpu(
+static int cf_check hvmemul_get_fpu(
enum x86_emulate_fpu_type type,
struct x86_emulate_ctxt *ctxt)
{
@@ -2395,7 +2395,7 @@ static int hvmemul_get_fpu(
return X86EMUL_OKAY;
}
-static void hvmemul_put_fpu(
+static void cf_check hvmemul_put_fpu(
struct x86_emulate_ctxt *ctxt,
enum x86_emulate_fpu_type backout,
const struct x86_emul_fpu_aux *aux)
@@ -2482,7 +2482,7 @@ static void hvmemul_put_fpu(
}
}
-static int hvmemul_tlb_op(
+static int cf_check hvmemul_tlb_op(
enum x86emul_tlb_op op,
unsigned long addr,
unsigned long aux,
@@ -2539,7 +2539,7 @@ static int hvmemul_tlb_op(
return rc;
}
-static int hvmemul_vmfunc(
+static int cf_check hvmemul_vmfunc(
struct x86_emulate_ctxt *ctxt)
{
int rc;
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9e4924649077..e87e809a945d 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3755,8 +3755,8 @@ void hvm_set_reg(struct vcpu *v, unsigned int reg, uint64_t val)
}
}
-static bool is_sysdesc_access(const struct x86_emulate_state *state,
- const struct x86_emulate_ctxt *ctxt)
+static bool cf_check is_sysdesc_access(
+ const struct x86_emulate_state *state, const struct x86_emulate_ctxt *ctxt)
{
unsigned int ext;
int mode = x86_insn_modrm(state, NULL, &ext);
@@ -3796,8 +3796,8 @@ int hvm_descriptor_access_intercept(uint64_t exit_info,
return X86EMUL_OKAY;
}
-static bool is_cross_vendor(const struct x86_emulate_state *state,
- const struct x86_emulate_ctxt *ctxt)
+static bool cf_check is_cross_vendor(
+ const struct x86_emulate_state *state, const struct x86_emulate_ctxt *ctxt)
{
switch ( ctxt->opcode )
{
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 4c4ebda5e6e4..dedb2848e6a1 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2447,8 +2447,8 @@ static void svm_invlpg_intercept(unsigned long linear)
paging_invlpg(current, linear);
}
-static bool is_invlpg(const struct x86_emulate_state *state,
- const struct x86_emulate_ctxt *ctxt)
+static bool cf_check is_invlpg(
+ const struct x86_emulate_state *state, const struct x86_emulate_ctxt *ctxt)
{
unsigned int ext;
diff --git a/xen/arch/x86/include/asm/hvm/emulate.h b/xen/arch/x86/include/asm/hvm/emulate.h
index e67004060345..d8ba2df4e4a2 100644
--- a/xen/arch/x86/include/asm/hvm/emulate.h
+++ b/xen/arch/x86/include/asm/hvm/emulate.h
@@ -92,10 +92,10 @@ static inline bool handle_mmio(void)
return hvm_emulate_one_insn(x86_insn_is_mem_access, "MMIO");
}
-int hvmemul_insn_fetch(unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt);
+int cf_check hvmemul_insn_fetch(
+ unsigned long offset, void *p_data, unsigned int bytes,
+ struct x86_emulate_ctxt *ctxt);
+
int hvmemul_do_pio_buffer(uint16_t port,
unsigned int size,
uint8_t dir,
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index bdde24d2cec3..f2f7b6902ce4 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -538,16 +538,12 @@ struct mmio_ro_emulate_ctxt {
unsigned int seg, bdf;
};
-extern int mmio_ro_emulated_write(enum x86_segment seg,
- unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt);
-extern int mmcfg_intercept_write(enum x86_segment seg,
- unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt);
+int cf_check mmio_ro_emulated_write(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt);
+int cf_check mmcfg_intercept_write(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt);
int audit_adjust_pgtables(struct domain *d, int dir, int noisy);
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 0665095d2309..2befd0c191ae 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4852,7 +4852,7 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
return 0;
}
-int mmio_ro_emulated_write(
+int cf_check mmio_ro_emulated_write(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -4873,7 +4873,7 @@ int mmio_ro_emulated_write(
return X86EMUL_OKAY;
}
-int mmcfg_intercept_write(
+int cf_check mmcfg_intercept_write(
enum x86_segment seg,
unsigned long offset,
void *p_data,
diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c
index f2991bc176f0..c90d326becb3 100644
--- a/xen/arch/x86/mm/shadow/hvm.c
+++ b/xen/arch/x86/mm/shadow/hvm.c
@@ -148,7 +148,7 @@ hvm_read(enum x86_segment seg,
return X86EMUL_UNHANDLEABLE;
}
-static int
+static int cf_check
hvm_emulate_read(enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -161,7 +161,7 @@ hvm_emulate_read(enum x86_segment seg,
container_of(ctxt, struct sh_emulate_ctxt, ctxt));
}
-static int
+static int cf_check
hvm_emulate_insn_fetch(unsigned long offset,
void *p_data,
unsigned int bytes,
@@ -181,7 +181,7 @@ hvm_emulate_insn_fetch(unsigned long offset,
return X86EMUL_OKAY;
}
-static int
+static int cf_check
hvm_emulate_write(enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -234,7 +234,7 @@ hvm_emulate_write(enum x86_segment seg,
return X86EMUL_OKAY;
}
-static int
+static int cf_check
hvm_emulate_cmpxchg(enum x86_segment seg,
unsigned long offset,
void *p_old,
diff --git a/xen/arch/x86/pv/emul-gate-op.c b/xen/arch/x86/pv/emul-gate-op.c
index 68ec4d11f6bb..758a20ad9df4 100644
--- a/xen/arch/x86/pv/emul-gate-op.c
+++ b/xen/arch/x86/pv/emul-gate-op.c
@@ -96,8 +96,9 @@ struct gate_op_ctxt {
bool insn_fetch;
};
-static int read_mem(enum x86_segment seg, unsigned long offset, void *p_data,
- unsigned int bytes, struct x86_emulate_ctxt *ctxt)
+static int cf_check read_mem(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
const struct gate_op_ctxt *goc =
container_of(ctxt, struct gate_op_ctxt, ctxt);
@@ -163,8 +164,8 @@ static int read_mem(enum x86_segment seg, unsigned long offset, void *p_data,
return X86EMUL_OKAY;
}
-static int fetch(unsigned long offset, void *p_data,
- unsigned int bytes, struct x86_emulate_ctxt *ctxt)
+static int cf_check fetch(unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
return read_mem(x86_seg_cs, offset, p_data, bytes, ctxt);
}
diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c
index c78be6d92b21..c46c072f93db 100644
--- a/xen/arch/x86/pv/emul-priv-op.c
+++ b/xen/arch/x86/pv/emul-priv-op.c
@@ -358,8 +358,9 @@ static unsigned int check_guest_io_breakpoint(struct vcpu *v,
return match;
}
-static int read_io(unsigned int port, unsigned int bytes,
- unsigned long *val, struct x86_emulate_ctxt *ctxt)
+static int cf_check read_io(
+ unsigned int port, unsigned int bytes, unsigned long *val,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -462,8 +463,9 @@ static void guest_io_write(unsigned int port, unsigned int bytes,
}
}
-static int write_io(unsigned int port, unsigned int bytes,
- unsigned long val, struct x86_emulate_ctxt *ctxt)
+static int cf_check write_io(
+ unsigned int port, unsigned int bytes, unsigned long val,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -493,9 +495,9 @@ static int write_io(unsigned int port, unsigned int bytes,
return X86EMUL_OKAY;
}
-static int read_segment(enum x86_segment seg,
- struct segment_register *reg,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check read_segment(
+ enum x86_segment seg, struct segment_register *reg,
+ struct x86_emulate_ctxt *ctxt)
{
/* Check if this is an attempt to access the I/O bitmap. */
if ( seg == x86_seg_tr )
@@ -607,10 +609,10 @@ static int pv_emul_virt_to_linear(unsigned long base, unsigned long offset,
return rc;
}
-static int rep_ins(uint16_t port,
- enum x86_segment seg, unsigned long offset,
- unsigned int bytes_per_rep, unsigned long *reps,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check rep_ins(
+ uint16_t port, enum x86_segment seg, unsigned long offset,
+ unsigned int bytes_per_rep, unsigned long *reps,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -675,10 +677,10 @@ static int rep_ins(uint16_t port,
return X86EMUL_OKAY;
}
-static int rep_outs(enum x86_segment seg, unsigned long offset,
- uint16_t port,
- unsigned int bytes_per_rep, unsigned long *reps,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check rep_outs(
+ enum x86_segment seg, unsigned long offset, uint16_t port,
+ unsigned int bytes_per_rep, unsigned long *reps,
+ struct x86_emulate_ctxt *ctxt)
{
struct priv_op_ctxt *poc = container_of(ctxt, struct priv_op_ctxt, ctxt);
struct vcpu *curr = current;
@@ -744,8 +746,8 @@ static int rep_outs(enum x86_segment seg, unsigned long offset,
return X86EMUL_OKAY;
}
-static int read_cr(unsigned int reg, unsigned long *val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check read_cr(
+ unsigned int reg, unsigned long *val, struct x86_emulate_ctxt *ctxt)
{
const struct vcpu *curr = current;
@@ -787,8 +789,8 @@ static int read_cr(unsigned int reg, unsigned long *val,
return X86EMUL_UNHANDLEABLE;
}
-static int write_cr(unsigned int reg, unsigned long val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check write_cr(
+ unsigned int reg, unsigned long val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
@@ -871,8 +873,8 @@ static uint64_t guest_efer(const struct domain *d)
return val;
}
-static int read_msr(unsigned int reg, uint64_t *val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check read_msr(
+ unsigned int reg, uint64_t *val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
const struct domain *currd = curr->domain;
@@ -1020,8 +1022,8 @@ static int read_msr(unsigned int reg, uint64_t *val,
return ret;
}
-static int write_msr(unsigned int reg, uint64_t val,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check write_msr(
+ unsigned int reg, uint64_t val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
const struct domain *currd = curr->domain;
@@ -1188,8 +1190,9 @@ static int write_msr(unsigned int reg, uint64_t val,
return X86EMUL_UNHANDLEABLE;
}
-static int cache_op(enum x86emul_cache_op op, enum x86_segment seg,
- unsigned long offset, struct x86_emulate_ctxt *ctxt)
+static int cf_check cache_op(
+ enum x86emul_cache_op op, enum x86_segment seg,
+ unsigned long offset, struct x86_emulate_ctxt *ctxt)
{
ASSERT(op == x86emul_wbinvd || op == x86emul_wbnoinvd);
@@ -1208,8 +1211,8 @@ static int cache_op(enum x86emul_cache_op op, enum x86_segment seg,
return X86EMUL_OKAY;
}
-static int validate(const struct x86_emulate_state *state,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check validate(
+ const struct x86_emulate_state *state, struct x86_emulate_ctxt *ctxt)
{
switch ( ctxt->opcode )
{
@@ -1258,10 +1261,9 @@ static int validate(const struct x86_emulate_state *state,
return X86EMUL_UNHANDLEABLE;
}
-static int insn_fetch(unsigned long offset,
- void *p_data,
- unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check insn_fetch(
+ unsigned long offset, void *p_data, unsigned int bytes,
+ struct x86_emulate_ctxt *ctxt)
{
const struct priv_op_ctxt *poc =
container_of(ctxt, struct priv_op_ctxt, ctxt);
diff --git a/xen/arch/x86/pv/emulate.h b/xen/arch/x86/pv/emulate.h
index 4b845b08e372..49a4d34832df 100644
--- a/xen/arch/x86/pv/emulate.h
+++ b/xen/arch/x86/pv/emulate.h
@@ -12,13 +12,6 @@ int pv_emul_read_descriptor(unsigned int sel, const struct vcpu *v,
void pv_emul_instruction_done(struct cpu_user_regs *regs, unsigned long rip);
-static inline int pv_emul_is_mem_write(const struct x86_emulate_state *state,
- struct x86_emulate_ctxt *ctxt)
-{
- return x86_insn_is_mem_write(state, ctxt) ? X86EMUL_OKAY
- : X86EMUL_UNHANDLEABLE;
-}
-
/* Return a pointer to the GDT/LDT descriptor referenced by sel. */
static inline const seg_desc_t *gdt_ldt_desc_ptr(unsigned int sel)
{
diff --git a/xen/arch/x86/pv/ro-page-fault.c b/xen/arch/x86/pv/ro-page-fault.c
index ef4d146c1d9e..5963f5ee2d51 100644
--- a/xen/arch/x86/pv/ro-page-fault.c
+++ b/xen/arch/x86/pv/ro-page-fault.c
@@ -26,6 +26,13 @@
#include "emulate.h"
#include "mm.h"
+static int cf_check pv_emul_is_mem_write(
+ const struct x86_emulate_state *state, struct x86_emulate_ctxt *ctxt)
+{
+ return x86_insn_is_mem_write(state, ctxt) ? X86EMUL_OKAY
+ : X86EMUL_UNHANDLEABLE;
+}
+
/*********************
* Writable Pagetables
*/
@@ -35,9 +42,9 @@ struct ptwr_emulate_ctxt {
l1_pgentry_t pte;
};
-static int ptwr_emulated_read(enum x86_segment seg, unsigned long offset,
- void *p_data, unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_read(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
unsigned int rc = bytes;
unsigned long addr = offset;
@@ -52,9 +59,9 @@ static int ptwr_emulated_read(enum x86_segment seg, unsigned long offset,
return X86EMUL_OKAY;
}
-static int ptwr_emulated_insn_fetch(unsigned long offset,
- void *p_data, unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_insn_fetch(
+ unsigned long offset, void *p_data, unsigned int bytes,
+ struct x86_emulate_ctxt *ctxt)
{
unsigned int rc = copy_from_guest_pv(p_data, (void *)offset, bytes);
@@ -218,9 +225,9 @@ static int ptwr_emulated_update(unsigned long addr, intpte_t *p_old,
return X86EMUL_OKAY;
}
-static int ptwr_emulated_write(enum x86_segment seg, unsigned long offset,
- void *p_data, unsigned int bytes,
- struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_write(
+ enum x86_segment seg, unsigned long offset, void *p_data,
+ unsigned int bytes, struct x86_emulate_ctxt *ctxt)
{
intpte_t val = 0;
@@ -236,9 +243,9 @@ static int ptwr_emulated_write(enum x86_segment seg, unsigned long offset,
return ptwr_emulated_update(offset, NULL, val, bytes, ctxt);
}
-static int ptwr_emulated_cmpxchg(enum x86_segment seg, unsigned long offset,
- void *p_old, void *p_new, unsigned int bytes,
- bool lock, struct x86_emulate_ctxt *ctxt)
+static int cf_check ptwr_emulated_cmpxchg(
+ enum x86_segment seg, unsigned long offset, void *p_old, void *p_new,
+ unsigned int bytes, bool lock, struct x86_emulate_ctxt *ctxt)
{
intpte_t old = 0, new = 0;
int rc;
diff --git a/xen/arch/x86/x86_emulate.c b/xen/arch/x86/x86_emulate.c
index 1e082e6f3b2d..60191a94dc18 100644
--- a/xen/arch/x86/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate.c
@@ -53,8 +53,8 @@
#include "x86_emulate/x86_emulate.c"
-int x86emul_read_xcr(unsigned int reg, uint64_t *val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_read_xcr(
+ unsigned int reg, uint64_t *val, struct x86_emulate_ctxt *ctxt)
{
switch ( reg )
{
@@ -77,8 +77,8 @@ int x86emul_read_xcr(unsigned int reg, uint64_t *val,
}
/* Note: May be called with ctxt=NULL. */
-int x86emul_write_xcr(unsigned int reg, uint64_t val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_write_xcr(
+ unsigned int reg, uint64_t val, struct x86_emulate_ctxt *ctxt)
{
switch ( reg )
{
@@ -100,8 +100,8 @@ int x86emul_write_xcr(unsigned int reg, uint64_t val,
#ifdef CONFIG_PV
/* Called with NULL ctxt in hypercall context. */
-int x86emul_read_dr(unsigned int reg, unsigned long *val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_read_dr(
+ unsigned int reg, unsigned long *val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
@@ -143,8 +143,8 @@ int x86emul_read_dr(unsigned int reg, unsigned long *val,
return X86EMUL_OKAY;
}
-int x86emul_write_dr(unsigned int reg, unsigned long val,
- struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_write_dr(
+ unsigned int reg, unsigned long val, struct x86_emulate_ctxt *ctxt)
{
struct vcpu *curr = current;
@@ -167,8 +167,9 @@ int x86emul_write_dr(unsigned int reg, unsigned long val,
}
#endif /* CONFIG_PV */
-int x86emul_cpuid(uint32_t leaf, uint32_t subleaf,
- struct cpuid_leaf *res, struct x86_emulate_ctxt *ctxt)
+int cf_check x86emul_cpuid(
+ uint32_t leaf, uint32_t subleaf, struct cpuid_leaf *res,
+ struct x86_emulate_ctxt *ctxt)
{
guest_cpuid(current, leaf, subleaf, res);
diff --git a/xen/arch/x86/x86_emulate/x86_emulate.c b/xen/arch/x86/x86_emulate/x86_emulate.c
index 2ba54c61511c..6c0d18954a5f 100644
--- a/xen/arch/x86/x86_emulate/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
@@ -2524,7 +2524,7 @@ static void adjust_bnd(struct x86_emulate_ctxt *ctxt,
done:;
}
-int x86emul_unhandleable_rw(
+int cf_check x86emul_unhandleable_rw(
enum x86_segment seg,
unsigned long offset,
void *p_data,
@@ -12320,7 +12320,7 @@ x86_insn_operand_ea(const struct x86_emulate_state *state,
* memory operand (like POP), but it does not mean e.g. segment selector
* loads, where the descriptor table access is considered an implicit one.
*/
-bool
+bool cf_check
x86_insn_is_mem_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
@@ -12412,7 +12412,7 @@ x86_insn_is_mem_access(const struct x86_emulate_state *state,
* loads, where the (possible) descriptor table write is considered an
* implicit access.
*/
-bool
+bool cf_check
x86_insn_is_mem_write(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
@@ -12584,7 +12584,7 @@ x86_insn_is_mem_write(const struct x86_emulate_state *state,
return false;
}
-bool
+bool cf_check
x86_insn_is_portio(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
@@ -12599,7 +12599,7 @@ x86_insn_is_portio(const struct x86_emulate_state *state,
return false;
}
-bool
+bool cf_check
x86_insn_is_cr_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt)
{
diff --git a/xen/arch/x86/x86_emulate/x86_emulate.h b/xen/arch/x86/x86_emulate/x86_emulate.h
index 419def8790a0..4732855c40ed 100644
--- a/xen/arch/x86/x86_emulate/x86_emulate.h
+++ b/xen/arch/x86/x86_emulate/x86_emulate.h
@@ -737,7 +737,7 @@ static inline unsigned long *decode_gpr(struct cpu_user_regs *regs,
}
/* Unhandleable read, write or instruction fetch */
-int
+int cf_check
x86emul_unhandleable_rw(
enum x86_segment seg,
unsigned long offset,
@@ -766,16 +766,16 @@ x86_insn_immediate(const struct x86_emulate_state *state,
unsigned int
x86_insn_length(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_mem_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_mem_write(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_portio(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
-bool
+bool cf_check
x86_insn_is_cr_access(const struct x86_emulate_state *state,
const struct x86_emulate_ctxt *ctxt);
@@ -787,17 +787,18 @@ void x86_emulate_free_state(struct x86_emulate_state *state);
#ifdef __XEN__
-int x86emul_read_xcr(unsigned int reg, uint64_t *val,
- struct x86_emulate_ctxt *ctxt);
-int x86emul_write_xcr(unsigned int reg, uint64_t val,
- struct x86_emulate_ctxt *ctxt);
-
-int x86emul_read_dr(unsigned int reg, unsigned long *val,
- struct x86_emulate_ctxt *ctxt);
-int x86emul_write_dr(unsigned int reg, unsigned long val,
- struct x86_emulate_ctxt *ctxt);
-int x86emul_cpuid(uint32_t leaf, uint32_t subleaf,
- struct cpuid_leaf *res, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_read_xcr(
+ unsigned int reg, uint64_t *val, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_write_xcr(
+ unsigned int reg, uint64_t val, struct x86_emulate_ctxt *ctxt);
+
+int cf_check x86emul_read_dr(
+ unsigned int reg, unsigned long *val, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_write_dr(
+ unsigned int reg, unsigned long val, struct x86_emulate_ctxt *ctxt);
+int cf_check x86emul_cpuid(
+ uint32_t leaf, uint32_t subleaf, struct cpuid_leaf *res,
+ struct x86_emulate_ctxt *ctxt);
#endif
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 46/70] x86/logdirty: CFI hardening
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (9 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 33/70] x86/emul: " Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 47/70] x86/shadow: " Andrew Cooper
` (4 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3:
* Fix !CONFIG_SHADOW build. Annotate targets in none.c
---
xen/arch/x86/mm/hap/hap.c | 6 +++---
xen/arch/x86/mm/shadow/common.c | 12 ++++++------
xen/arch/x86/mm/shadow/none.c | 6 +++---
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/mm/hap/hap.c b/xen/arch/x86/mm/hap/hap.c
index de4b13565ab4..ed5112b00b63 100644
--- a/xen/arch/x86/mm/hap/hap.c
+++ b/xen/arch/x86/mm/hap/hap.c
@@ -180,7 +180,7 @@ int hap_track_dirty_vram(struct domain *d,
* NB: Domain that having device assigned should not set log_global. Because
* there is no way to track the memory updating from device.
*/
-static int hap_enable_log_dirty(struct domain *d, bool_t log_global)
+static int cf_check hap_enable_log_dirty(struct domain *d, bool log_global)
{
struct p2m_domain *p2m = p2m_get_hostp2m(d);
@@ -211,7 +211,7 @@ static int hap_enable_log_dirty(struct domain *d, bool_t log_global)
return 0;
}
-static int hap_disable_log_dirty(struct domain *d)
+static int cf_check hap_disable_log_dirty(struct domain *d)
{
paging_lock(d);
d->arch.paging.mode &= ~PG_log_dirty;
@@ -228,7 +228,7 @@ static int hap_disable_log_dirty(struct domain *d)
return 0;
}
-static void hap_clean_dirty_bitmap(struct domain *d)
+static void cf_check hap_clean_dirty_bitmap(struct domain *d)
{
/*
* Switch to log-dirty mode, either by setting l1e entries of P2M table to
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 83dedc8870aa..071a19adce82 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -40,9 +40,9 @@
DEFINE_PER_CPU(uint32_t,trace_shadow_path_flags);
-static int sh_enable_log_dirty(struct domain *, bool log_global);
-static int sh_disable_log_dirty(struct domain *);
-static void sh_clean_dirty_bitmap(struct domain *);
+static int cf_check sh_enable_log_dirty(struct domain *, bool log_global);
+static int cf_check sh_disable_log_dirty(struct domain *);
+static void cf_check sh_clean_dirty_bitmap(struct domain *);
/* Set up the shadow-specific parts of a domain struct at start of day.
* Called for every domain from arch_domain_create() */
@@ -3016,7 +3016,7 @@ static int shadow_test_disable(struct domain *d)
/* Shadow specific code which is called in paging_log_dirty_enable().
* Return 0 if no problem found.
*/
-static int sh_enable_log_dirty(struct domain *d, bool log_global)
+static int cf_check sh_enable_log_dirty(struct domain *d, bool log_global)
{
int ret;
@@ -3044,7 +3044,7 @@ static int sh_enable_log_dirty(struct domain *d, bool log_global)
}
/* shadow specfic code which is called in paging_log_dirty_disable() */
-static int sh_disable_log_dirty(struct domain *d)
+static int cf_check sh_disable_log_dirty(struct domain *d)
{
int ret;
@@ -3058,7 +3058,7 @@ static int sh_disable_log_dirty(struct domain *d)
/* This function is called when we CLEAN log dirty bitmap. See
* paging_log_dirty_op() for details.
*/
-static void sh_clean_dirty_bitmap(struct domain *d)
+static void cf_check sh_clean_dirty_bitmap(struct domain *d)
{
paging_lock(d);
/* Need to revoke write access to the domain's pages again.
diff --git a/xen/arch/x86/mm/shadow/none.c b/xen/arch/x86/mm/shadow/none.c
index 79889b926a89..463a0e3e89c3 100644
--- a/xen/arch/x86/mm/shadow/none.c
+++ b/xen/arch/x86/mm/shadow/none.c
@@ -1,19 +1,19 @@
#include <xen/mm.h>
#include <asm/shadow.h>
-static int _enable_log_dirty(struct domain *d, bool log_global)
+static int cf_check _enable_log_dirty(struct domain *d, bool log_global)
{
ASSERT(is_pv_domain(d));
return -EOPNOTSUPP;
}
-static int _disable_log_dirty(struct domain *d)
+static int cf_check _disable_log_dirty(struct domain *d)
{
ASSERT(is_pv_domain(d));
return -EOPNOTSUPP;
}
-static void _clean_dirty_bitmap(struct domain *d)
+static void cf_check _clean_dirty_bitmap(struct domain *d)
{
ASSERT(is_pv_domain(d));
}
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 47/70] x86/shadow: CFI hardening
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (10 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 46/70] x86/logdirty: " Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 59/70] x86: Use control flow typechecking where possible Andrew Cooper
` (3 subsequent siblings)
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper
Control Flow Integrity schemes use toolchain and optionally hardware support
to help protect against call/jump/return oriented programming attacks.
Use cf_check to annotate function pointer targets for the toolchain.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
v3:
* Fix !CONFIG_SHADOW build. Annotate targets in none.c
---
xen/arch/x86/mm/shadow/common.c | 11 +++---
xen/arch/x86/mm/shadow/hvm.c | 8 ++--
xen/arch/x86/mm/shadow/multi.c | 80 ++++++++++++++++++++--------------------
xen/arch/x86/mm/shadow/multi.h | 20 +++++-----
xen/arch/x86/mm/shadow/none.c | 14 +++----
xen/arch/x86/mm/shadow/private.h | 12 +++---
xen/arch/x86/mm/shadow/pv.c | 4 +-
7 files changed, 74 insertions(+), 75 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 071a19adce82..8f111901730f 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1215,7 +1215,7 @@ void shadow_free(struct domain *d, mfn_t smfn)
* This action is irreversible: the p2m mapping only ever grows.
* That's OK because the p2m table only exists for translated domains,
* and those domains can't ever turn off shadow mode. */
-static struct page_info *
+static struct page_info *cf_check
shadow_alloc_p2m_page(struct domain *d)
{
struct page_info *pg;
@@ -1251,7 +1251,7 @@ shadow_alloc_p2m_page(struct domain *d)
return pg;
}
-static void
+static void cf_check
shadow_free_p2m_page(struct domain *d, struct page_info *pg)
{
struct domain *owner = page_get_owner(pg);
@@ -2290,7 +2290,8 @@ void shadow_prepare_page_type_change(struct domain *d, struct page_info *page,
/* Reset the up-pointers of every L3 shadow to 0.
* This is called when l3 shadows stop being pinnable, to clear out all
* the list-head bits so the up-pointer field is properly inititalised. */
-static int sh_clear_up_pointer(struct vcpu *v, mfn_t smfn, mfn_t unused)
+static int cf_check sh_clear_up_pointer(
+ struct vcpu *v, mfn_t smfn, mfn_t unused)
{
mfn_to_page(smfn)->up = 0;
return 0;
@@ -2490,7 +2491,7 @@ static void sh_update_paging_modes(struct vcpu *v)
v->arch.paging.mode->update_cr3(v, 0, false);
}
-void shadow_update_paging_modes(struct vcpu *v)
+void cf_check shadow_update_paging_modes(struct vcpu *v)
{
paging_lock(v->domain);
sh_update_paging_modes(v);
@@ -3075,7 +3076,7 @@ static bool flush_vcpu(const struct vcpu *v, const unsigned long *vcpu_bitmap)
}
/* Flush TLB of selected vCPUs. NULL for all. */
-bool shadow_flush_tlb(const unsigned long *vcpu_bitmap)
+bool cf_check shadow_flush_tlb(const unsigned long *vcpu_bitmap)
{
static DEFINE_PER_CPU(cpumask_t, flush_cpumask);
cpumask_t *mask = &this_cpu(flush_cpumask);
diff --git a/xen/arch/x86/mm/shadow/hvm.c b/xen/arch/x86/mm/shadow/hvm.c
index c90d326becb3..27dd99f1a12e 100644
--- a/xen/arch/x86/mm/shadow/hvm.c
+++ b/xen/arch/x86/mm/shadow/hvm.c
@@ -794,9 +794,9 @@ sh_remove_all_shadows_and_parents(struct domain *d, mfn_t gmfn)
* It means extra emulated writes and slows down removal of mappings. */
}
-static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
- l1_pgentry_t old, l1_pgentry_t new,
- unsigned int level)
+static void cf_check sh_unshadow_for_p2m_change(
+ struct domain *d, unsigned long gfn, l1_pgentry_t old, l1_pgentry_t new,
+ unsigned int level)
{
mfn_t omfn = l1e_get_mfn(old);
unsigned int oflags = l1e_get_flags(old);
@@ -879,7 +879,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
}
#if (SHADOW_OPTIMIZATIONS & SHOPT_FAST_FAULT_PATH)
-static void
+static void cf_check
sh_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
{
struct domain *d = p2m->domain;
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index bddef53163f5..b0b1c31ee033 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -369,7 +369,7 @@ static void sh_audit_gw(struct vcpu *v, const walk_t *gw)
#if GUEST_PAGING_LEVELS == 2
/* From one page of a multi-page shadow, find the next one */
-static inline mfn_t sh_next_page(mfn_t smfn)
+static inline mfn_t cf_check sh_next_page(mfn_t smfn)
{
struct page_info *pg = mfn_to_page(smfn), *next;
struct page_list_head h = PAGE_LIST_HEAD_INIT(h);
@@ -399,8 +399,7 @@ guest_index(void *ptr)
return (u32)((unsigned long)ptr & ~PAGE_MASK) / sizeof(guest_l1e_t);
}
-static u32
-shadow_l1_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l1_index(mfn_t *smfn, u32 guest_index)
{
#if (GUEST_PAGING_LEVELS == 2)
ASSERT(mfn_to_page(*smfn)->u.sh.head);
@@ -412,8 +411,7 @@ shadow_l1_index(mfn_t *smfn, u32 guest_index)
#endif
}
-static u32
-shadow_l2_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l2_index(mfn_t *smfn, u32 guest_index)
{
#if (GUEST_PAGING_LEVELS == 2)
int i;
@@ -432,14 +430,12 @@ shadow_l2_index(mfn_t *smfn, u32 guest_index)
#if GUEST_PAGING_LEVELS >= 4
-static u32
-shadow_l3_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l3_index(mfn_t *smfn, u32 guest_index)
{
return guest_index;
}
-static u32
-shadow_l4_index(mfn_t *smfn, u32 guest_index)
+static u32 cf_check shadow_l4_index(mfn_t *smfn, u32 guest_index)
{
return guest_index;
}
@@ -924,7 +920,7 @@ do { \
/**************************************************************************/
/* Create a shadow of a given guest page.
*/
-static mfn_t
+static mfn_t cf_check
sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
{
struct domain *d = v->domain;
@@ -1459,7 +1455,8 @@ void sh_unhook_64b_mappings(struct domain *d, mfn_t sl4mfn, int user_only)
*/
#if GUEST_PAGING_LEVELS >= 4
-static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
+static int cf_check validate_gl4e(
+ struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
{
shadow_l4e_t new_sl4e;
guest_l4e_t new_gl4e = *(guest_l4e_t *)new_ge;
@@ -1518,7 +1515,8 @@ static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
}
-static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
+static int cf_check validate_gl3e(
+ struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
{
struct domain *d = v->domain;
shadow_l3e_t new_sl3e;
@@ -1552,7 +1550,8 @@ static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
}
#endif // GUEST_PAGING_LEVELS >= 4
-static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
+static int cf_check validate_gl2e(
+ struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
{
struct domain *d = v->domain;
shadow_l2e_t new_sl2e;
@@ -1599,7 +1598,8 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
return result;
}
-static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
+static int cf_check validate_gl1e(
+ struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
{
struct domain *d = v->domain;
shadow_l1e_t new_sl1e;
@@ -2089,8 +2089,8 @@ static DEFINE_PER_CPU(int,trace_extra_emulation_count);
#endif
static DEFINE_PER_CPU(guest_pa_t,trace_emulate_write_val);
-static void trace_emulate_write_val(const void *ptr, unsigned long vaddr,
- const void *src, unsigned int bytes)
+static void cf_check trace_emulate_write_val(
+ const void *ptr, unsigned long vaddr, const void *src, unsigned int bytes)
{
#if GUEST_PAGING_LEVELS == 3
if ( vaddr == this_cpu(trace_emulate_initial_va) )
@@ -2144,9 +2144,8 @@ static inline void trace_shadow_emulate(guest_l1e_t gl1e, unsigned long va)
* shadow code (and the guest should retry) or 0 if it is not (and the
* fault should be handled elsewhere or passed to the guest). */
-static int sh_page_fault(struct vcpu *v,
- unsigned long va,
- struct cpu_user_regs *regs)
+static int cf_check sh_page_fault(
+ struct vcpu *v, unsigned long va, struct cpu_user_regs *regs)
{
struct domain *d = v->domain;
walk_t gw;
@@ -2898,7 +2897,7 @@ static int sh_page_fault(struct vcpu *v,
* instruction should be issued on the hardware, or false if it's safe not
* to do so.
*/
-static bool sh_invlpg(struct vcpu *v, unsigned long linear)
+static bool cf_check sh_invlpg(struct vcpu *v, unsigned long linear)
{
mfn_t sl1mfn;
shadow_l2e_t sl2e;
@@ -3030,9 +3029,8 @@ static bool sh_invlpg(struct vcpu *v, unsigned long linear)
#ifdef CONFIG_HVM
-static unsigned long
-sh_gva_to_gfn(struct vcpu *v, struct p2m_domain *p2m,
- unsigned long va, uint32_t *pfec)
+static unsigned long cf_check sh_gva_to_gfn(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long va, uint32_t *pfec)
/* Called to translate a guest virtual address to what the *guest*
* pagetables would map it to. */
{
@@ -3196,8 +3194,7 @@ sh_update_linear_entries(struct vcpu *v)
* Removes v->arch.paging.shadow.shadow_table[].
* Does all appropriate management/bookkeeping/refcounting/etc...
*/
-static void
-sh_detach_old_tables(struct vcpu *v)
+static void cf_check sh_detach_old_tables(struct vcpu *v)
{
struct domain *d = v->domain;
mfn_t smfn;
@@ -3216,8 +3213,7 @@ sh_detach_old_tables(struct vcpu *v)
}
}
-static void
-sh_update_cr3(struct vcpu *v, int do_locking, bool noflush)
+static void cf_check sh_update_cr3(struct vcpu *v, int do_locking, bool noflush)
/* Updates vcpu->arch.cr3 after the guest has changed CR3.
* Paravirtual guests should set v->arch.guest_table (and guest_table_user,
* if appropriate).
@@ -3525,7 +3521,8 @@ int sh_rm_write_access_from_sl1p(struct domain *d, mfn_t gmfn,
#endif /* OOS */
#if defined(CONFIG_HVM) && (SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC)
-static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
+static int cf_check sh_guess_wrmap(
+ struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
/* Look up this vaddr in the current shadow and see if it's a writeable
* mapping of this gmfn. If so, remove it. Returns 1 if it worked. */
{
@@ -3589,8 +3586,8 @@ static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
}
#endif
-int sh_rm_write_access_from_l1(struct domain *d, mfn_t sl1mfn,
- mfn_t readonly_mfn)
+int cf_check sh_rm_write_access_from_l1(
+ struct domain *d, mfn_t sl1mfn, mfn_t readonly_mfn)
/* Excises all writeable mappings to readonly_mfn from this l1 shadow table */
{
shadow_l1e_t *sl1e;
@@ -3626,7 +3623,8 @@ int sh_rm_write_access_from_l1(struct domain *d, mfn_t sl1mfn,
}
-int sh_rm_mappings_from_l1(struct domain *d, mfn_t sl1mfn, mfn_t target_mfn)
+int cf_check sh_rm_mappings_from_l1(
+ struct domain *d, mfn_t sl1mfn, mfn_t target_mfn)
/* Excises all mappings to guest frame from this shadow l1 table */
{
shadow_l1e_t *sl1e;
@@ -3677,7 +3675,7 @@ void sh_clear_shadow_entry(struct domain *d, void *ep, mfn_t smfn)
}
}
-int sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
+int cf_check sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
/* Remove all mappings of this l1 shadow from this l2 shadow */
{
shadow_l2e_t *sl2e;
@@ -3700,7 +3698,7 @@ int sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
}
#if GUEST_PAGING_LEVELS >= 4
-int sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
+int cf_check sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
/* Remove all mappings of this l2 shadow from this l3 shadow */
{
shadow_l3e_t *sl3e;
@@ -3722,7 +3720,7 @@ int sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
return done;
}
-int sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
+int cf_check sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
/* Remove all mappings of this l3 shadow from this l4 shadow */
{
shadow_l4e_t *sl4e;
@@ -3752,7 +3750,7 @@ int sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
* and in the meantime we unhook its top-level user-mode entries. */
#if GUEST_PAGING_LEVELS == 3
-static void sh_pagetable_dying(paddr_t gpa)
+static void cf_check sh_pagetable_dying(paddr_t gpa)
{
struct vcpu *v = current;
struct domain *d = v->domain;
@@ -3833,7 +3831,7 @@ static void sh_pagetable_dying(paddr_t gpa)
put_gfn(d, l3gfn);
}
#else
-static void sh_pagetable_dying(paddr_t gpa)
+static void cf_check sh_pagetable_dying(paddr_t gpa)
{
struct vcpu *v = current;
struct domain *d = v->domain;
@@ -3932,7 +3930,7 @@ static const char *sh_audit_flags(struct vcpu *v, int level,
return NULL;
}
-int sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
+int cf_check sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
{
guest_l1e_t *gl1e, *gp;
shadow_l1e_t *sl1e;
@@ -4000,7 +3998,7 @@ int sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
return done;
}
-int sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
+int cf_check sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
{
guest_l1e_t *gl1e, e;
shadow_l1e_t *sl1e;
@@ -4026,7 +4024,7 @@ int sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
return 0;
}
-int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
+int cf_check sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
{
struct domain *d = v->domain;
guest_l2e_t *gl2e, *gp;
@@ -4078,7 +4076,7 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
}
#if GUEST_PAGING_LEVELS >= 4
-int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
+int cf_check sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
{
struct domain *d = v->domain;
guest_l3e_t *gl3e, *gp;
@@ -4126,7 +4124,7 @@ int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
return 0;
}
-int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
+int cf_check sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
{
struct domain *d = v->domain;
guest_l4e_t *gl4e, *gp;
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 0bd6a2d5b787..5bcd6ae1a8da 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -59,10 +59,10 @@ extern void
SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, GUEST_LEVELS)
(struct domain *d, mfn_t sl4mfn, int user_only);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, GUEST_LEVELS)
(struct domain *d, mfn_t sl1mfn, mfn_t readonly_mfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, GUEST_LEVELS)
(struct domain *d, mfn_t sl1mfn, mfn_t target_mfn);
@@ -70,30 +70,30 @@ extern void
SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, GUEST_LEVELS)
(struct domain *d, void *ep, mfn_t smfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_remove_l1_shadow, GUEST_LEVELS)
(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_remove_l2_shadow, GUEST_LEVELS)
(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn);
-extern int
+int cf_check
SHADOW_INTERNAL_NAME(sh_remove_l3_shadow, GUEST_LEVELS)
(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn);
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l1_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_fl1_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l2_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl2mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l3_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl3mfn, mfn_t x);
-int
+int cf_check
SHADOW_INTERNAL_NAME(sh_audit_l4_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl4mfn, mfn_t x);
#endif
diff --git a/xen/arch/x86/mm/shadow/none.c b/xen/arch/x86/mm/shadow/none.c
index 463a0e3e89c3..eaaa874b119f 100644
--- a/xen/arch/x86/mm/shadow/none.c
+++ b/xen/arch/x86/mm/shadow/none.c
@@ -30,34 +30,34 @@ int shadow_domain_init(struct domain *d)
return is_hvm_domain(d) ? -EOPNOTSUPP : 0;
}
-static int _page_fault(struct vcpu *v, unsigned long va,
- struct cpu_user_regs *regs)
+static int cf_check _page_fault(
+ struct vcpu *v, unsigned long va, struct cpu_user_regs *regs)
{
ASSERT_UNREACHABLE();
return 0;
}
-static bool _invlpg(struct vcpu *v, unsigned long linear)
+static bool cf_check _invlpg(struct vcpu *v, unsigned long linear)
{
ASSERT_UNREACHABLE();
return true;
}
#ifdef CONFIG_HVM
-static unsigned long _gva_to_gfn(struct vcpu *v, struct p2m_domain *p2m,
- unsigned long va, uint32_t *pfec)
+static unsigned long cf_check _gva_to_gfn(
+ struct vcpu *v, struct p2m_domain *p2m, unsigned long va, uint32_t *pfec)
{
ASSERT_UNREACHABLE();
return gfn_x(INVALID_GFN);
}
#endif
-static void _update_cr3(struct vcpu *v, int do_locking, bool noflush)
+static void cf_check _update_cr3(struct vcpu *v, int do_locking, bool noflush)
{
ASSERT_UNREACHABLE();
}
-static void _update_paging_modes(struct vcpu *v)
+static void cf_check _update_paging_modes(struct vcpu *v)
{
ASSERT_UNREACHABLE();
}
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index e4db8d32546a..3dc024e30f20 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -420,15 +420,15 @@ static inline int sh_remove_write_access(struct domain *d, mfn_t readonly_mfn,
#endif
/* Functions that atomically write PV guest PT entries */
-void sh_write_guest_entry(struct vcpu *v, intpte_t *p, intpte_t new,
- mfn_t gmfn);
-intpte_t sh_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p, intpte_t old,
- intpte_t new, mfn_t gmfn);
+void cf_check sh_write_guest_entry(
+ struct vcpu *v, intpte_t *p, intpte_t new, mfn_t gmfn);
+intpte_t cf_check sh_cmpxchg_guest_entry(
+ struct vcpu *v, intpte_t *p, intpte_t old, intpte_t new, mfn_t gmfn);
/* Update all the things that are derived from the guest's CR0/CR3/CR4.
* Called to initialize paging structures if the paging mode
* has changed, and when bringing up a VCPU for the first time. */
-void shadow_update_paging_modes(struct vcpu *v);
+void cf_check shadow_update_paging_modes(struct vcpu *v);
/* Unhook the non-Xen mappings in this top-level shadow mfn.
* With user_only == 1, unhooks only the user-mode mappings. */
@@ -922,7 +922,7 @@ static inline int sh_check_page_has_no_refs(struct page_info *page)
}
/* Flush the TLB of the selected vCPUs. */
-bool shadow_flush_tlb(const unsigned long *vcpu_bitmap);
+bool cf_check shadow_flush_tlb(const unsigned long *vcpu_bitmap);
#endif /* _XEN_SHADOW_PRIVATE_H */
diff --git a/xen/arch/x86/mm/shadow/pv.c b/xen/arch/x86/mm/shadow/pv.c
index f51f980f2694..ed10d5479c5e 100644
--- a/xen/arch/x86/mm/shadow/pv.c
+++ b/xen/arch/x86/mm/shadow/pv.c
@@ -28,7 +28,7 @@
* Write a new value into the guest pagetable, and update the shadows
* appropriately.
*/
-void
+void cf_check
sh_write_guest_entry(struct vcpu *v, intpte_t *p, intpte_t new, mfn_t gmfn)
{
paging_lock(v->domain);
@@ -42,7 +42,7 @@ sh_write_guest_entry(struct vcpu *v, intpte_t *p, intpte_t new, mfn_t gmfn)
* appropriately. Returns the previous entry found, which the caller is
* expected to check to see if the cmpxchg was successful.
*/
-intpte_t
+intpte_t cf_check
sh_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p, intpte_t old,
intpte_t new, mfn_t gmfn)
{
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 59/70] x86: Use control flow typechecking where possible
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (11 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 47/70] x86/shadow: " Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-23 14:21 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
` (2 subsequent siblings)
15 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
Now all callees have been annotated, turn on typechecking to catch issues in
the future.
This extension isn't in a released version of GCC yet, so provide a container
to use with the extention included, and add it to CI. RANDCONFIG is necessary
because some stubs for compiled-out subsystems are used as function pointer
targets.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v3:
* Provide container. Already pushed to Gitlab.
---
automation/build/debian/buster-gcc-ibt.dockerfile | 66 +++++++++++++++++++++++
automation/gitlab-ci/build.yaml | 6 +++
automation/scripts/containerize | 1 +
xen/arch/x86/arch.mk | 1 +
4 files changed, 74 insertions(+)
create mode 100644 automation/build/debian/buster-gcc-ibt.dockerfile
diff --git a/automation/build/debian/buster-gcc-ibt.dockerfile b/automation/build/debian/buster-gcc-ibt.dockerfile
new file mode 100644
index 000000000000..441d9a9ab37a
--- /dev/null
+++ b/automation/build/debian/buster-gcc-ibt.dockerfile
@@ -0,0 +1,66 @@
+FROM debian:buster-slim AS builder
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN apt-get update && \
+ apt-get --quiet --yes install \
+ bison \
+ build-essential \
+ flex \
+ libc6-dev-i386 \
+ libgmp-dev \
+ libisl-dev \
+ libmpc-dev \
+ libmpfr-dev \
+ patch \
+ wget
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN wget -q https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz -O - | tar xJ --strip=1
+RUN wget -q https://xenbits.xen.org/people/andrewcoop/gcc-11.2-Add-fcf-check-attribute-yes-no.patch -O - | patch -p1
+RUN ./configure \
+ --prefix=/opt/gcc-11-ibt \
+ --enable-languages=c \
+ --disable-nls \
+ --disable-threads \
+ --disable-bootstrap \
+ --disable-shared \
+ --disable-libmudflap \
+ --disable-libssp \
+ --disable-libgomp \
+ --disable-decimal-float \
+ --disable-libquadmath \
+ --disable-libatomic \
+ --disable-libcc1 \
+ --disable-libmpx
+RUN make -j`nproc` && make -j`nproc` install
+
+
+FROM debian:buster-slim
+COPY --from=builder /opt/gcc-11-ibt /opt/gcc-11-ibt
+
+LABEL maintainer.name="The Xen Project" \
+ maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+ENV PATH="/opt/gcc-11-ibt/bin:${PATH}"
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN apt-get update && \
+ apt-get --quiet --yes install \
+ bison \
+ checkpolicy \
+ flex \
+ gawk \
+ make \
+ python3 \
+ && \
+ apt-get autoremove -y && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index fdd5c76582b3..cc36428cf55b 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -294,6 +294,12 @@ debian-stretch-32-gcc-debug:
variables:
CONTAINER: debian:stretch-i386
+debian-buster-gcc-ibt:
+ extends: .gcc-x86-64-build
+ variables:
+ CONTAINER: debian:buster-gcc-ibt
+ RANDCONFIG: y
+
debian-unstable-clang:
extends: .clang-x86-64-build
variables:
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index 7682ccd34759..8992c67278ae 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -33,6 +33,7 @@ case "_${CONTAINER}" in
_focal) CONTAINER="${BASE}/ubuntu:focal" ;;
_jessie) CONTAINER="${BASE}/debian:jessie" ;;
_stretch|_) CONTAINER="${BASE}/debian:stretch" ;;
+ _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
_unstable|_) CONTAINER="${BASE}/debian:unstable" ;;
_trusty) CONTAINER="${BASE}/ubuntu:trusty" ;;
_xenial) CONTAINER="${BASE}/ubuntu:xenial" ;;
diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
index f780c912a9cf..92fd19811013 100644
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -54,6 +54,7 @@ endif
ifdef CONFIG_XEN_IBT
CFLAGS += -fcf-protection=branch -mmanual-endbr
+$(call cc-option-add,CFLAGS,CC,-fcf-check-attribute=no)
else
$(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
endif
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (12 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 59/70] x86: Use control flow typechecking where possible Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-23 11:31 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 64/70] x86: Introduce helpers/checks for " Andrew Cooper
2022-02-22 15:29 ` [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Jan Beulich
15 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel
Cc: Marek Marczykowski-Górecki, Andrew Cooper, Jan Beulich,
Roger Pau Monné,
Wei Liu
From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
An interesting corner case occurs when the byte sequence making up endb64 ends
up on a non-instruction boundary. Such embedded instructions mark legal
indirect branch targets as far as the CPU is concerned, which aren't legal as
far as the logic is concerned.
When CET-IBT is active, check for embedded byte sequences. Example failures
look like:
check-endbr.sh xen-syms Fail: Found 2 embedded endbr64 instructions
0xffff82d040325677: test_endbr64 at /local/xen.git/xen/arch/x86/x86_64/entry.S:28
0xffff82d040352da6: init_done at /local/xen.git/xen/arch/x86/setup.c:675
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v2:
* New
v3:
* Reposition to the end of the cf_check-ing, to retain bisectability
* Reword commit message to explain 'embedded'
* Use ${ADDR2LINE} if present in the environment
* Use objdump -w
* Explain the use of octal
* Check the EFI build too. Reposition to be last action, so all build
artefacts remain in a failure case
* Check for grep support and warn if missing
* Replace strtonum() with int() to avoid gaining a gawk dependency
* Replace `join` with `sort | uniq` to avoid adding a coreutils dependency
---
README | 1 +
xen/arch/x86/Makefile | 6 ++++
xen/tools/check-endbr.sh | 85 ++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 92 insertions(+)
create mode 100755 xen/tools/check-endbr.sh
diff --git a/README b/README
index 562b80808033..5e55047ffd9e 100644
--- a/README
+++ b/README
@@ -68,6 +68,7 @@ provided by your OS distributor:
In addition to the above there are a number of optional build
prerequisites. Omitting these will cause the related features to be
disabled at compile time:
+ * Binary-search capable grep (if building Xen with CET support)
* Development install of Ocaml (e.g. ocaml-nox and
ocaml-findlib). Required to build ocaml components which
includes the alternative ocaml xenstored.
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index db97ae8c07f0..b90146b75636 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -142,6 +142,9 @@ $(TARGET)-syms: $(BASEDIR)/prelink.o $(obj)/xen.lds
| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort \
>$(@D)/$(@F).map
rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
+ifeq ($(CONFIG_XEN_IBT),y)
+ $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
+endif
$(obj)/note.o: $(TARGET)-syms
$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $< $@.bin
@@ -212,6 +215,9 @@ endif
$(NM) -pa --format=sysv $(@D)/$(@F) \
| $(BASEDIR)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
+ifeq ($(CONFIG_XEN_IBT),y)
+ $(SHELL) $(BASEDIR)/tools/check-endbr.sh $@
+endif
else
$(TARGET).efi: FORCE
rm -f $@
diff --git a/xen/tools/check-endbr.sh b/xen/tools/check-endbr.sh
new file mode 100755
index 000000000000..85878353112a
--- /dev/null
+++ b/xen/tools/check-endbr.sh
@@ -0,0 +1,85 @@
+#!/bin/sh
+#
+# Usage ./$0 xen-syms
+#
+set -e
+
+# Prettyprint parameters a little for message
+MSG_PFX="${0##*/} ${1##*/}"
+
+OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
+OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
+ADDR2LINE="${ADDR2LINE:-addr2line}"
+
+D=$(mktemp -d)
+trap "rm -rf $D" EXIT
+
+TEXT_BIN=$D/xen-syms.text
+VALID=$D/valid-addrs
+ALL=$D/all-addrs
+BAD=$D/bad-addrs
+
+# Check that grep can do binary searches. Some, e.g. busybox, can't. Leave a
+# warning but don't fail the build.
+echo "X" | grep -aob "X" -q 2>/dev/null ||
+ { echo "$MSG_PFX Warning: grep can't do binary searches" >&2; exit 0; }
+
+#
+# First, look for all the valid endbr64 instructions.
+# A worst-case disassembly, viewed through cat -A, may look like:
+#
+# ffff82d040337bd4 <endbr64>:$
+# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
+# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
+# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
+#
+# Want to grab the address of endbr64 instructions only, ignoring function
+# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
+# number of trailing spaces before the end of the line.
+#
+${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
+
+#
+# Second, look for any endbr64 byte sequence
+# This has a couple of complications:
+#
+# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
+# the grep offset to be from the start of .text.
+#
+# 2) dash's printf doesn't understand hex escapes, hence the use of octal.
+#
+# 3) AWK can't add 64bit integers, because internally all numbers are doubles.
+# When the upper bits are set, the exponents worth of precision is lost in
+# the lower bits, rounding integers to the nearest 4k.
+#
+# Instead, use the fact that Xen's .text is within a 1G aligned region, and
+# split the VMA in half so AWK's numeric addition is only working on 32 bit
+# numbers, which don't lose precision.
+#
+eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
+
+${OBJCOPY} -O binary $TEXT_BIN
+grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
+ awk -F':' '{printf "%s%x\n", "'$vma_hi'", int(0x'$vma_lo') + $1}' > $ALL
+
+# Wait for $VALID to become complete
+wait
+
+# Sanity check $VALID and $ALL, in case the string parsing bitrots
+val_sz=$(stat -c '%s' $VALID)
+all_sz=$(stat -c '%s' $ALL)
+[ "$val_sz" -eq 0 ] && { echo "$MSG_PFX Error: Empty valid-addrs" >&2; exit 1; }
+[ "$all_sz" -eq 0 ] && { echo "$MSG_PFX Error: Empty all-addrs" >&2; exit 1; }
+[ "$all_sz" -lt "$val_sz" ] && { echo "$MSG_PFX Error: More valid-addrs than all-addrs" >&2; exit 1; }
+
+# $BAD = $ALL - $VALID
+sort $VALID $ALL | uniq -u > $BAD
+nr_bad=$(wc -l < $BAD)
+
+# Success
+[ "$nr_bad" -eq 0 ] && exit 0
+
+# Failure
+echo "$MSG_PFX Fail: Found ${nr_bad} embedded endbr64 instructions" >&2
+${ADDR2LINE} -afip -e $1 < $BAD >&2
+exit 1
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH v3 64/70] x86: Introduce helpers/checks for endbr64 instructions
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (13 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
@ 2022-02-22 15:26 ` Andrew Cooper
2022-02-22 15:29 ` [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Jan Beulich
15 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:26 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Jan Beulich, Roger Pau Monné, Wei Liu
... to prevent the optimiser creating unsafe code. See the code comment for
full details.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>
v3:
* Introduce ENDBR64_LEN
v2:
* Fix include to let the header be standalone
* Add earlyclobber to asm
v1.1:
* New
---
xen/arch/x86/include/asm/endbr.h | 55 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 xen/arch/x86/include/asm/endbr.h
diff --git a/xen/arch/x86/include/asm/endbr.h b/xen/arch/x86/include/asm/endbr.h
new file mode 100644
index 000000000000..6090afeb0bd8
--- /dev/null
+++ b/xen/arch/x86/include/asm/endbr.h
@@ -0,0 +1,55 @@
+/******************************************************************************
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Copyright (c) 2021-2022 Citrix Systems Ltd.
+ */
+#ifndef XEN_ASM_ENDBR_H
+#define XEN_ASM_ENDBR_H
+
+#include <xen/types.h>
+
+#define ENDBR64_LEN 4
+
+/*
+ * In some cases we need to inspect/insert endbr64 instructions.
+ *
+ * The naive way, mem{cmp,cpy}(ptr, "\xf3\x0f\x1e\xfa", 4), optimises unsafely
+ * by placing 0xfa1e0ff3 in an imm32 operand, and marks a legal indirect
+ * branch target as far as the CPU is concerned.
+ *
+ * gen_endbr64() is written deliberately to avoid the problematic operand, and
+ * marked __const__ as it is safe for the optimiser to hoist/merge/etc.
+ */
+static inline uint32_t __attribute_const__ gen_endbr64(void)
+{
+ uint32_t res;
+
+ asm ( "mov $~0xfa1e0ff3, %[res]\n\t"
+ "not %[res]\n\t"
+ : [res] "=&r" (res) );
+
+ return res;
+}
+
+static inline bool is_endbr64(const void *ptr)
+{
+ return *(const uint32_t *)ptr == gen_endbr64();
+}
+
+static inline void place_endbr64(void *ptr)
+{
+ *(uint32_t *)ptr = gen_endbr64();
+}
+
+#endif /* XEN_ASM_ENDBR_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
` (14 preceding siblings ...)
2022-02-22 15:26 ` [PATCH v3 64/70] x86: Introduce helpers/checks for " Andrew Cooper
@ 2022-02-22 15:29 ` Jan Beulich
2022-02-22 15:41 ` Andrew Cooper
15 siblings, 1 reply; 28+ messages in thread
From: Jan Beulich @ 2022-02-22 15:29 UTC (permalink / raw)
To: Andrew Cooper
Cc: Stefano Stabellini, Wei Liu, Julien Grall, Roger Pau Monné,
Juergen Gross, Marek Marczykowski-Górecki, Xen-devel
On 22.02.2022 16:26, Andrew Cooper wrote:
> To avoid spamming everyone, I have only re-sent patches with changes in v3.
Could you enumerate which ones these are? Otherwise it's hard to tell
whether everything you did send did arrive in the recipients' mailboxes.
Thanks, Jan
> CET Indirect Branch Tracking is a hardware feature designed to protect against
> forward-edge control flow hijacking (Call/Jump oriented programming), and is a
> companion feature to CET Shadow Stacks added in Xen 4.14.
>
> Patches 1 thru 5 are prerequisites. Patches 6 thru 59 are fairly mechanical
> annotations of function pointer targets. Patches 60 thru 70 are the final
> enablement of CET-IBT.
>
> This series functions correctly with GCC 9 and later, although an experimental
> GCC patch is required to get more helpful typechecking at build time. A
> container with this fix has been added to CI.
>
> Tested on a TigerLake NUC by me, and by Marek also.
>
> CI pipelines:
> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/476819536
> https://cirrus-ci.com/build/4634902334275584
>
> Andrew Cooper (67):
> xen/sort: Switch to an extern inline implementation
> xen/xsm: Move {do,compat}_flask_op() declarations into a header
> x86/kexec: Annotate embedded data with ELF metadata
> x86: Introduce support for CET-IBT
> xen: CFI hardening for x86 hypercalls
> xen: CFI hardening for custom_param()
> xen: CFI hardening for __initcall()
> xen: CFI hardening for notifier callbacks
> xen: CFI hardening for acpi_table_parse()
> xen: CFI hardening for continue_hypercall_on_cpu()
> xen: CFI hardening for init_timer()
> xen: CFI hardening for call_rcu()
> xen: CFI hardening for IPIs
> xen: CFI hardening for open_softirq()
> xsm/flask/ss: CFI hardening
> xsm: CFI hardening
> xen/sched: CFI hardening
> xen/evtchn: CFI hardening
> xen/hypfs: CFI hardening
> xen/tasklet: CFI hardening
> xen/keyhandler: CFI hardening
> xen/vpci: CFI hardening
> xen/decompress: CFI hardening
> xen/iommu: CFI hardening
> xen/video: CFI hardening
> xen/console: CFI hardening
> xen/misc: CFI hardening
> x86: CFI hardening for request_irq()
> x86/hvm: CFI hardening for hvm_funcs
> x86/hvm: CFI hardening for device emulation
> x86/emul: CFI hardening
> x86/ucode: CFI hardening
> x86/power: CFI hardening
> x86/apic: CFI hardening
> x86/nmi: CFI hardening
> x86/mtrr: CFI hardening
> x86/idle: CFI hardening
> x86/quirks: CFI hardening
> x86/hvmsave: CFI hardening
> x86/mce: CFI hardening
> x86/pmu: CFI hardening
> x86/cpu: CFI hardening
> x86/guest: CFI hardening
> x86/logdirty: CFI hardening
> x86/shadow: CFI hardening
> x86/hap: CFI hardening
> x86/p2m: CFI hardening
> x86/irq: CFI hardening
> x86/apei: CFI hardening
> x86/psr: CFI hardening
> x86/dpci: CFI hardening
> x86/pt: CFI hardening
> x86/time: CFI hardening
> x86/misc: CFI hardening
> x86/stack: CFI hardening
> x86/bugframe: CFI hardening
> x86: Use control flow typechecking where possible
> x86/setup: Read CR4 earlier in __start_xen()
> x86/alternatives: Clear CR4.CET when clearing CR0.WP
> x86/traps: Rework write_stub_trampoline() to not hardcode the jmp
> x86: Introduce helpers/checks for endbr64 instructions
> x86/emul: Update emulation stubs to be CET-IBT compatible
> x86/entry: Make syscall/sysenter entrypoints CET-IBT compatible
> x86/entry: Make IDT entrypoints CET-IBT compatible
> x86/setup: Rework MSR_S_CET handling for CET-IBT
> x86/efi: Disable CET-IBT around Runtime Services calls
> x86: Enable CET Indirect Branch Tracking
>
> Juergen Gross (2):
> x86/pv-shim: Don't modify the hypercall table
> x86: Don't use the hypercall table for calling compat hypercalls
>
> Marek Marczykowski-Górecki (1):
> x86: Build check for embedded endbr64 instructions
>
> Config.mk | 1 -
> README | 1 +
> automation/build/debian/buster-gcc-ibt.dockerfile | 66 ++++
> automation/gitlab-ci/build.yaml | 6 +
> automation/scripts/containerize | 1 +
> docs/misc/xen-command-line.pandoc | 16 +-
> tools/firmware/Makefile | 2 +
> tools/libs/guest/xg_dom_decompress_unsafe.h | 2 +
> tools/tests/x86_emulator/x86-emulate.h | 2 +
> xen/arch/arm/bootfdt.c | 9 +-
> xen/arch/arm/io.c | 9 +-
> xen/arch/x86/Kconfig | 17 +
> xen/arch/x86/Makefile | 6 +
> xen/arch/x86/acpi/boot.c | 24 +-
> xen/arch/x86/acpi/cpu_idle.c | 43 ++-
> xen/arch/x86/acpi/cpufreq/cpufreq.c | 24 +-
> xen/arch/x86/acpi/cpufreq/powernow.c | 21 +-
> xen/arch/x86/acpi/cpuidle_menu.c | 6 +-
> xen/arch/x86/acpi/lib.c | 2 +-
> xen/arch/x86/acpi/power.c | 4 +-
> xen/arch/x86/acpi/wakeup_prot.S | 38 +-
> xen/arch/x86/alternative.c | 13 +-
> xen/arch/x86/apic.c | 12 +-
> xen/arch/x86/arch.mk | 7 +
> xen/arch/x86/boot/x86_64.S | 30 +-
> xen/arch/x86/compat.c | 21 +-
> xen/arch/x86/configs/pvshim_defconfig | 1 +
> xen/arch/x86/cpu/amd.c | 8 +-
> xen/arch/x86/cpu/centaur.c | 2 +-
> xen/arch/x86/cpu/common.c | 3 +-
> xen/arch/x86/cpu/cpu.h | 2 +-
> xen/arch/x86/cpu/hygon.c | 2 +-
> xen/arch/x86/cpu/intel.c | 6 +-
> xen/arch/x86/cpu/mcheck/amd_nonfatal.c | 4 +-
> xen/arch/x86/cpu/mcheck/mce.c | 22 +-
> xen/arch/x86/cpu/mcheck/mce.h | 2 +-
> xen/arch/x86/cpu/mcheck/mce_amd.c | 9 +-
> xen/arch/x86/cpu/mcheck/mce_amd.h | 4 +-
> xen/arch/x86/cpu/mcheck/mce_intel.c | 49 ++-
> xen/arch/x86/cpu/mcheck/non-fatal.c | 6 +-
> xen/arch/x86/cpu/mcheck/vmce.c | 4 +-
> xen/arch/x86/cpu/microcode/amd.c | 9 +-
> xen/arch/x86/cpu/microcode/core.c | 15 +-
> xen/arch/x86/cpu/microcode/intel.c | 10 +-
> xen/arch/x86/cpu/mtrr/generic.c | 20 +-
> xen/arch/x86/cpu/mtrr/main.c | 4 +-
> xen/arch/x86/cpu/mtrr/mtrr.h | 8 +-
> xen/arch/x86/cpu/mwait-idle.c | 12 +-
> xen/arch/x86/cpu/shanghai.c | 2 +-
> xen/arch/x86/cpu/vpmu.c | 13 +-
> xen/arch/x86/cpu/vpmu_amd.c | 16 +-
> xen/arch/x86/cpu/vpmu_intel.c | 16 +-
> xen/arch/x86/cpuid.c | 8 +-
> xen/arch/x86/crash.c | 7 +-
> xen/arch/x86/dmi_scan.c | 10 +-
> xen/arch/x86/dom0_build.c | 8 +-
> xen/arch/x86/domain.c | 16 +-
> xen/arch/x86/emul-i8254.c | 14 +-
> xen/arch/x86/extable.c | 18 +-
> xen/arch/x86/genapic/bigsmp.c | 4 +-
> xen/arch/x86/genapic/delivery.c | 12 +-
> xen/arch/x86/genapic/probe.c | 2 +-
> xen/arch/x86/genapic/x2apic.c | 18 +-
> xen/arch/x86/guest/hyperv/hyperv.c | 10 +-
> xen/arch/x86/guest/xen/xen.c | 15 +-
> xen/arch/x86/hpet.c | 29 +-
> xen/arch/x86/hvm/dm.c | 5 +-
> xen/arch/x86/hvm/dom0_build.c | 16 +-
> xen/arch/x86/hvm/emulate.c | 93 +++--
> xen/arch/x86/hvm/hpet.c | 12 +-
> xen/arch/x86/hvm/hvm.c | 47 +--
> xen/arch/x86/hvm/hypercall.c | 5 +-
> xen/arch/x86/hvm/intercept.c | 28 +-
> xen/arch/x86/hvm/io.c | 38 +-
> xen/arch/x86/hvm/ioreq.c | 2 +-
> xen/arch/x86/hvm/irq.c | 16 +-
> xen/arch/x86/hvm/mtrr.c | 8 +-
> xen/arch/x86/hvm/nestedhvm.c | 6 +-
> xen/arch/x86/hvm/pmtimer.c | 10 +-
> xen/arch/x86/hvm/quirks.c | 4 +-
> xen/arch/x86/hvm/rtc.c | 18 +-
> xen/arch/x86/hvm/stdvga.c | 19 +-
> xen/arch/x86/hvm/svm/nestedsvm.c | 22 +-
> xen/arch/x86/hvm/svm/svm.c | 404 +++++++++++-----------
> xen/arch/x86/hvm/svm/vmcb.c | 2 +-
> xen/arch/x86/hvm/vioapic.c | 12 +-
> xen/arch/x86/hvm/viridian/time.c | 2 +-
> xen/arch/x86/hvm/viridian/viridian.c | 17 +-
> xen/arch/x86/hvm/vlapic.c | 25 +-
> xen/arch/x86/hvm/vmsi.c | 16 +-
> xen/arch/x86/hvm/vmx/intr.c | 2 +-
> xen/arch/x86/hvm/vmx/vmcs.c | 22 +-
> xen/arch/x86/hvm/vmx/vmx.c | 155 +++++----
> xen/arch/x86/hvm/vmx/vvmx.c | 16 +-
> xen/arch/x86/hvm/vpic.c | 8 +-
> xen/arch/x86/hvm/vpt.c | 2 +-
> xen/arch/x86/i8259.c | 10 +-
> xen/arch/x86/include/asm/asm-defns.h | 6 +
> xen/arch/x86/include/asm/bug.h | 10 +-
> xen/arch/x86/include/asm/cpufeature.h | 1 +
> xen/arch/x86/include/asm/cpufeatures.h | 1 +
> xen/arch/x86/include/asm/cpuidle.h | 4 +-
> xen/arch/x86/include/asm/current.h | 6 +-
> xen/arch/x86/include/asm/endbr.h | 55 +++
> xen/arch/x86/include/asm/flushtlb.h | 2 +-
> xen/arch/x86/include/asm/genapic.h | 18 +-
> xen/arch/x86/include/asm/hpet.h | 8 +-
> xen/arch/x86/include/asm/hvm/emulate.h | 8 +-
> xen/arch/x86/include/asm/hvm/save.h | 2 +-
> xen/arch/x86/include/asm/hvm/svm/nestedsvm.h | 18 +-
> xen/arch/x86/include/asm/hvm/svm/svm.h | 1 -
> xen/arch/x86/include/asm/hvm/vioapic.h | 2 +-
> xen/arch/x86/include/asm/hvm/vmx/vmcs.h | 8 +-
> xen/arch/x86/include/asm/hvm/vmx/vmx.h | 4 +-
> xen/arch/x86/include/asm/hvm/vmx/vvmx.h | 18 +-
> xen/arch/x86/include/asm/hypercall.h | 81 +++--
> xen/arch/x86/include/asm/irq.h | 24 +-
> xen/arch/x86/include/asm/machine_kexec.h | 2 +-
> xen/arch/x86/include/asm/mm.h | 16 +-
> xen/arch/x86/include/asm/msi.h | 8 +-
> xen/arch/x86/include/asm/msr-index.h | 1 +
> xen/arch/x86/include/asm/mtrr.h | 2 +-
> xen/arch/x86/include/asm/p2m.h | 4 +-
> xen/arch/x86/include/asm/paging.h | 2 +-
> xen/arch/x86/include/asm/processor.h | 4 +-
> xen/arch/x86/include/asm/pv/domain.h | 4 +-
> xen/arch/x86/include/asm/pv/shim.h | 11 +-
> xen/arch/x86/include/asm/shadow.h | 2 +-
> xen/arch/x86/include/asm/smp.h | 6 +-
> xen/arch/x86/include/asm/tboot.h | 2 +-
> xen/arch/x86/include/asm/time.h | 6 +-
> xen/arch/x86/io_apic.c | 28 +-
> xen/arch/x86/ioport_emulate.c | 4 +-
> xen/arch/x86/irq.c | 28 +-
> xen/arch/x86/livepatch.c | 2 +-
> xen/arch/x86/machine_kexec.c | 2 +-
> xen/arch/x86/mm.c | 35 +-
> xen/arch/x86/mm/hap/guest_walk.c | 4 +-
> xen/arch/x86/mm/hap/hap.c | 29 +-
> xen/arch/x86/mm/hap/nested_hap.c | 2 +-
> xen/arch/x86/mm/hap/private.h | 30 +-
> xen/arch/x86/mm/mem_sharing.c | 2 +-
> xen/arch/x86/mm/p2m-ept.c | 34 +-
> xen/arch/x86/mm/p2m-pt.c | 19 +-
> xen/arch/x86/mm/paging.c | 3 +-
> xen/arch/x86/mm/shadow/common.c | 33 +-
> xen/arch/x86/mm/shadow/hvm.c | 16 +-
> xen/arch/x86/mm/shadow/multi.c | 80 +++--
> xen/arch/x86/mm/shadow/multi.h | 20 +-
> xen/arch/x86/mm/shadow/none.c | 20 +-
> xen/arch/x86/mm/shadow/private.h | 12 +-
> xen/arch/x86/mm/shadow/pv.c | 4 +-
> xen/arch/x86/msi.c | 18 +-
> xen/arch/x86/nmi.c | 16 +-
> xen/arch/x86/numa.c | 10 +-
> xen/arch/x86/oprofile/nmi_int.c | 16 +-
> xen/arch/x86/oprofile/op_model_athlon.c | 18 +-
> xen/arch/x86/oprofile/op_model_p4.c | 14 +-
> xen/arch/x86/oprofile/op_model_ppro.c | 26 +-
> xen/arch/x86/percpu.c | 6 +-
> xen/arch/x86/physdev.c | 2 +-
> xen/arch/x86/platform_hypercall.c | 11 +-
> xen/arch/x86/psr.c | 41 +--
> xen/arch/x86/pv/callback.c | 25 +-
> xen/arch/x86/pv/descriptor-tables.c | 14 +-
> xen/arch/x86/pv/domain.c | 12 +-
> xen/arch/x86/pv/emul-gate-op.c | 9 +-
> xen/arch/x86/pv/emul-priv-op.c | 71 ++--
> xen/arch/x86/pv/emulate.h | 7 -
> xen/arch/x86/pv/hypercall.c | 11 +-
> xen/arch/x86/pv/iret.c | 4 +-
> xen/arch/x86/pv/misc-hypercalls.c | 10 +-
> xen/arch/x86/pv/ro-page-fault.c | 31 +-
> xen/arch/x86/pv/shim.c | 60 ++--
> xen/arch/x86/pv/traps.c | 2 +-
> xen/arch/x86/setup.c | 80 ++++-
> xen/arch/x86/shutdown.c | 10 +-
> xen/arch/x86/smp.c | 20 +-
> xen/arch/x86/smpboot.c | 2 +-
> xen/arch/x86/spec_ctrl.c | 6 +-
> xen/arch/x86/srat.c | 4 +-
> xen/arch/x86/sysctl.c | 4 +-
> xen/arch/x86/tboot.c | 2 +-
> xen/arch/x86/time.c | 68 ++--
> xen/arch/x86/traps.c | 8 +-
> xen/arch/x86/tsx.c | 2 +-
> xen/arch/x86/x86_64/acpi_mmcfg.c | 2 +-
> xen/arch/x86/x86_64/compat.c | 1 -
> xen/arch/x86/x86_64/compat/entry.S | 1 +
> xen/arch/x86/x86_64/compat/mm.c | 7 +-
> xen/arch/x86/x86_64/entry.S | 49 ++-
> xen/arch/x86/x86_64/kexec_reloc.S | 23 +-
> xen/arch/x86/x86_64/mmconfig-shared.c | 10 +-
> xen/arch/x86/x86_64/mmconfig.h | 2 +-
> xen/arch/x86/x86_64/platform_hypercall.c | 2 +-
> xen/arch/x86/x86_64/traps.c | 42 ++-
> xen/arch/x86/x86_emulate.c | 34 +-
> xen/arch/x86/x86_emulate/x86_emulate.c | 10 +-
> xen/arch/x86/x86_emulate/x86_emulate.h | 33 +-
> xen/arch/x86/xen.lds.S | 3 +-
> xen/common/argo.c | 6 +-
> xen/common/bunzip2.c | 2 +-
> xen/common/compat/domain.c | 3 +-
> xen/common/compat/grant_table.c | 5 +-
> xen/common/compat/kernel.c | 2 +-
> xen/common/compat/memory.c | 7 +-
> xen/common/compat/multicall.c | 3 +-
> xen/common/core_parking.c | 10 +-
> xen/common/coverage/gcov.c | 8 +-
> xen/common/cpu.c | 4 +-
> xen/common/debugtrace.c | 10 +-
> xen/common/decompress.c | 2 +-
> xen/common/dm.c | 6 +-
> xen/common/domain.c | 15 +-
> xen/common/domctl.c | 2 +-
> xen/common/efi/boot.c | 6 +-
> xen/common/efi/runtime.c | 18 +
> xen/common/event_2l.c | 21 +-
> xen/common/event_channel.c | 18 +-
> xen/common/event_fifo.c | 30 +-
> xen/common/gdbstub.c | 9 +-
> xen/common/grant_table.c | 29 +-
> xen/common/hypfs.c | 63 ++--
> xen/common/irq.c | 6 +-
> xen/common/kernel.c | 6 +-
> xen/common/kexec.c | 18 +-
> xen/common/keyhandler.c | 47 +--
> xen/common/livepatch.c | 15 +-
> xen/common/memory.c | 8 +-
> xen/common/multicall.c | 2 +-
> xen/common/page_alloc.c | 14 +-
> xen/common/perfc.c | 4 +-
> xen/common/radix-tree.c | 8 +-
> xen/common/random.c | 2 +-
> xen/common/rangeset.c | 2 +-
> xen/common/rcupdate.c | 8 +-
> xen/common/sched/arinc653.c | 20 +-
> xen/common/sched/compat.c | 2 +-
> xen/common/sched/core.c | 40 +--
> xen/common/sched/cpupool.c | 35 +-
> xen/common/sched/credit.c | 59 ++--
> xen/common/sched/credit2.c | 55 ++-
> xen/common/sched/null.c | 60 ++--
> xen/common/sched/rt.c | 47 +--
> xen/common/spinlock.c | 12 +-
> xen/common/stop_machine.c | 6 +-
> xen/common/sysctl.c | 2 +-
> xen/common/tasklet.c | 4 +-
> xen/common/timer.c | 6 +-
> xen/common/trace.c | 4 +-
> xen/common/unlzma.c | 2 +-
> xen/common/vm_event.c | 6 +-
> xen/common/xenoprof.c | 2 +-
> xen/common/xmalloc_tlsf.c | 4 +-
> xen/common/zstd/zstd_common.c | 4 +-
> xen/common/zstd/zstd_internal.h | 4 +-
> xen/drivers/acpi/apei/apei-base.c | 32 +-
> xen/drivers/acpi/apei/apei-internal.h | 20 +-
> xen/drivers/acpi/apei/erst.c | 57 ++-
> xen/drivers/acpi/apei/hest.c | 4 +-
> xen/drivers/acpi/numa.c | 10 +-
> xen/drivers/acpi/tables.c | 2 +-
> xen/drivers/char/console.c | 36 +-
> xen/drivers/char/ehci-dbgp.c | 28 +-
> xen/drivers/char/ns16550.c | 34 +-
> xen/drivers/cpufreq/cpufreq.c | 6 +-
> xen/drivers/cpufreq/cpufreq_misc_governors.c | 22 +-
> xen/drivers/cpufreq/cpufreq_ondemand.c | 10 +-
> xen/drivers/passthrough/amd/iommu.h | 45 +--
> xen/drivers/passthrough/amd/iommu_acpi.c | 15 +-
> xen/drivers/passthrough/amd/iommu_guest.c | 12 +-
> xen/drivers/passthrough/amd/iommu_init.c | 49 +--
> xen/drivers/passthrough/amd/iommu_intr.c | 20 +-
> xen/drivers/passthrough/amd/iommu_map.c | 22 +-
> xen/drivers/passthrough/amd/pci_amd_iommu.c | 32 +-
> xen/drivers/passthrough/iommu.c | 56 ++-
> xen/drivers/passthrough/pci.c | 18 +-
> xen/drivers/passthrough/vtd/dmar.c | 7 +-
> xen/drivers/passthrough/vtd/extern.h | 38 +-
> xen/drivers/passthrough/vtd/intremap.c | 14 +-
> xen/drivers/passthrough/vtd/iommu.c | 94 ++---
> xen/drivers/passthrough/vtd/qinval.c | 28 +-
> xen/drivers/passthrough/vtd/quirks.c | 2 +-
> xen/drivers/passthrough/vtd/utils.c | 2 +-
> xen/drivers/passthrough/vtd/x86/hvm.c | 4 +-
> xen/drivers/passthrough/x86/hvm.c | 14 +-
> xen/drivers/video/lfb.c | 4 +-
> xen/drivers/video/lfb.h | 4 +-
> xen/drivers/video/vesa.c | 6 +-
> xen/drivers/video/vga.c | 6 +-
> xen/drivers/vpci/header.c | 18 +-
> xen/drivers/vpci/msi.c | 42 +--
> xen/drivers/vpci/msix.c | 20 +-
> xen/drivers/vpci/vpci.c | 16 +-
> xen/include/acpi/cpufreq/cpufreq.h | 1 -
> xen/include/xen/acpi.h | 2 +-
> xen/include/xen/compiler.h | 6 +
> xen/include/xen/domain.h | 2 +-
> xen/include/xen/hypercall.h | 69 ++--
> xen/include/xen/hypfs.h | 49 ++-
> xen/include/xen/irq.h | 6 +-
> xen/include/xen/lib.h | 2 +-
> xen/include/xen/perfc.h | 4 +-
> xen/include/xen/sched.h | 2 +-
> xen/include/xen/sort.h | 55 ++-
> xen/include/xen/spinlock.h | 4 +-
> xen/include/xen/vpci.h | 8 +-
> xen/include/xsm/dummy.h | 211 +++++------
> xen/lib/sort.c | 80 +----
> xen/tools/check-endbr.sh | 85 +++++
> xen/xsm/flask/avc.c | 2 +-
> xen/xsm/flask/flask_op.c | 8 +-
> xen/xsm/flask/hooks.c | 236 +++++++------
> xen/xsm/flask/private.h | 9 +
> xen/xsm/flask/ss/avtab.c | 4 +-
> xen/xsm/flask/ss/conditional.c | 10 +-
> xen/xsm/flask/ss/conditional.h | 6 +-
> xen/xsm/flask/ss/policydb.c | 53 +--
> xen/xsm/flask/ss/services.c | 6 +-
> xen/xsm/flask/ss/symtab.c | 5 +-
> xen/xsm/silo.c | 24 +-
> xen/xsm/xsm_core.c | 6 +-
> 322 files changed, 3316 insertions(+), 2739 deletions(-)
> create mode 100644 automation/build/debian/buster-gcc-ibt.dockerfile
> create mode 100644 xen/arch/x86/include/asm/endbr.h
> create mode 100755 xen/tools/check-endbr.sh
> create mode 100644 xen/xsm/flask/private.h
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking
2022-02-22 15:29 ` [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Jan Beulich
@ 2022-02-22 15:41 ` Andrew Cooper
0 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-22 15:41 UTC (permalink / raw)
To: Jan Beulich
Cc: Stefano Stabellini, Wei Liu, Julien Grall, Roger Pau Monne,
Juergen Gross, Marek Marczykowski-Górecki, Xen-devel
On 22/02/2022 15:29, Jan Beulich wrote:
> On 22.02.2022 16:26, Andrew Cooper wrote:
>> To avoid spamming everyone, I have only re-sent patches with changes in v3.
> Could you enumerate which ones these are? Otherwise it's hard to tell
> whether everything you did send did arrive in the recipients' mailboxes.
Oops sorry. 1, 3, 5, 6, 8, 12, 26, 27, 29, 33, 46, 47, 59, 60, 64.
All that I'm expecting to see have appeared on the list.
~Andrew
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation
2022-02-22 15:26 ` [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
@ 2022-02-22 15:52 ` Julien Grall
0 siblings, 0 replies; 28+ messages in thread
From: Julien Grall @ 2022-02-22 15:52 UTC (permalink / raw)
To: Andrew Cooper, Xen-devel
Cc: Jan Beulich, Roger Pau Monné,
Wei Liu, Stefano Stabellini, Volodymyr Babchuk, Bertrand Marquis
Hi,
On 22/02/2022 15:26, Andrew Cooper wrote:
> There are exactly 3 callers of sort() in the hypervisor. Callbacks in a tight
> loop like this are problematic for performance, especially with Spectre v2
> protections, which is why extern inline is used commonly by libraries.
>
> Both ARM callers pass in NULL for the swap function, and while this might seem
> like an attractive option at first, it causes generic_swap() to be used, which
> forced a byte-wise copy. Provide real swap functions so the compiler can
> optimise properly, which is very important for ARM downstreams where
> milliseconds until the system is up matters.
After the previous discussion, I was expecting the sentence "Provide
real..." to be completely dropped. Instead the change should be
justified with...
> This is also important for Control Flow Integrity schemes (e.g. x86 CET-IBT,
> ARM BTI), because tagged function(s) performing an arbitrary length swap of
> two arbitrary pointers is a very valuable gadget for an attacker.
... this one as this is the real reason of the change. Not the
performance (unless you have numbers proving it).
>
> No functional change.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
To be pedantic, my Nacked-by hasn't been yet revoked (see [1]). So you
should have kept it in the new version.
Anyway, given that the patch makes sense for BTI, I am willing to
replace the Nacked-by with an Acked-by:
Acked-by: Julien Grall <jgrall@amazon.com>
[1]
https://lore.kernel.org/xen-devel/70824a0c-cc48-b064-695c-35c2d06c0ad1@xen.org/
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 03/70] x86/pv-shim: Don't modify the hypercall table
2022-02-22 15:26 ` [PATCH v3 03/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
@ 2022-02-22 16:24 ` Jan Beulich
0 siblings, 0 replies; 28+ messages in thread
From: Jan Beulich @ 2022-02-22 16:24 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Juergen Gross, Xen-devel
On 22.02.2022 16:26, Andrew Cooper wrote:
> From: Juergen Gross <jgross@suse.com>
>
> When running as pv-shim the hypercall is modified today in order to
> replace the functions for __HYPERVISOR_event_channel_op and
> __HYPERVISOR_grant_table_op hypercalls.
>
> Change this to call the related functions from the normal handlers
> instead when running as shim. The performance implications are not
> really relevant, as a normal production hypervisor will not be
> configured to support shim mode, so the related calls will be dropped
> due to optimisation of the compiler.
>
> Note that for the CONFIG_PV_SHIM_EXCLUSIVE case there is a dummy
> wrapper do_grant_table_op() needed, as in this case grant_table.c
> isn't being built.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> [Split out of series. Make compile in isolation]
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
As said, somewhat hesitantly
Reviewed-by: Jan Beulich <jbeulich@suse.com>
"Make compile in isolation", while describing the changes which weren't
in Jürgen's original patch, is quite terse. But anyway.
Jan
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 29/70] xen/misc: CFI hardening
2022-02-22 15:26 ` [PATCH v3 29/70] xen/misc: " Andrew Cooper
@ 2022-02-23 10:25 ` Jan Beulich
2022-02-23 10:34 ` Andrew Cooper
0 siblings, 1 reply; 28+ messages in thread
From: Jan Beulich @ 2022-02-23 10:25 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Xen-devel
On 22.02.2022 16:26, Andrew Cooper wrote:
> Control Flow Integrity schemes use toolchain and optionally hardware support
> to help protect against call/jump/return oriented programming attacks.
>
> Use cf_check to annotate function pointer targets for the toolchain.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> ---
> v3
> * Annotations for gcov
Hmm, ...
> ---
> xen/arch/x86/mm.c | 6 ++++--
> xen/arch/x86/setup.c | 4 ++--
> xen/common/coverage/gcov.c | 8 ++++----
... what about llvm.c then?
Jan
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 29/70] xen/misc: CFI hardening
2022-02-23 10:25 ` Jan Beulich
@ 2022-02-23 10:34 ` Andrew Cooper
0 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-23 10:34 UTC (permalink / raw)
To: Jan Beulich; +Cc: Xen-devel
On 23/02/2022 10:25, Jan Beulich wrote:
> On 22.02.2022 16:26, Andrew Cooper wrote:
>> Control Flow Integrity schemes use toolchain and optionally hardware support
>> to help protect against call/jump/return oriented programming attacks.
>>
>> Use cf_check to annotate function pointer targets for the toolchain.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> v3
>> * Annotations for gcov
> Hmm, ...
>
>> ---
>> xen/arch/x86/mm.c | 6 ++++--
>> xen/arch/x86/setup.c | 4 ++--
>> xen/common/coverage/gcov.c | 8 ++++----
> ... what about llvm.c then?
Good point. I'll fix up.
There's currently no Clang toolchain capable of spotting/complaining at
this, but the Clang devs are working on this.
~Andrew
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions
2022-02-22 15:26 ` [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
@ 2022-02-23 11:31 ` Jan Beulich
2022-02-23 12:05 ` Andrew Cooper
0 siblings, 1 reply; 28+ messages in thread
From: Jan Beulich @ 2022-02-23 11:31 UTC (permalink / raw)
To: Andrew Cooper
Cc: Marek Marczykowski-Górecki, Roger Pau Monné,
Wei Liu, Xen-devel
On 22.02.2022 16:26, Andrew Cooper wrote:
> From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>
> An interesting corner case occurs when the byte sequence making up endb64 ends
Nit: For grep-ability it would be nice to spell this "endbr64".
> up on a non-instruction boundary. Such embedded instructions mark legal
> indirect branch targets as far as the CPU is concerned, which aren't legal as
> far as the logic is concerned.
Thinking about it: Wouldn't it be yet slightly more reassuring to also
look for ENDBR32?
> When CET-IBT is active, check for embedded byte sequences. Example failures
> look like:
>
> check-endbr.sh xen-syms Fail: Found 2 embedded endbr64 instructions
> 0xffff82d040325677: test_endbr64 at /local/xen.git/xen/arch/x86/x86_64/entry.S:28
> 0xffff82d040352da6: init_done at /local/xen.git/xen/arch/x86/setup.c:675
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
> --- a/README
> +++ b/README
> @@ -68,6 +68,7 @@ provided by your OS distributor:
> In addition to the above there are a number of optional build
> prerequisites. Omitting these will cause the related features to be
> disabled at compile time:
> + * Binary-search capable grep (if building Xen with CET support)
Nit: With this (maybe this was the case already earlier though)
s/will/may/ in the previous sentence?
> --- /dev/null
> +++ b/xen/tools/check-endbr.sh
> @@ -0,0 +1,85 @@
> +#!/bin/sh
> +#
> +# Usage ./$0 xen-syms
> +#
> +set -e
> +
> +# Prettyprint parameters a little for message
> +MSG_PFX="${0##*/} ${1##*/}"
> +
> +OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
> +OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
While embedding the arguments here shortens the lines where these are
used, the appearance especially of $OBJCOPY with a single file name
argument ...
> +ADDR2LINE="${ADDR2LINE:-addr2line}"
> +
> +D=$(mktemp -d)
> +trap "rm -rf $D" EXIT
> +
> +TEXT_BIN=$D/xen-syms.text
> +VALID=$D/valid-addrs
> +ALL=$D/all-addrs
> +BAD=$D/bad-addrs
> +
> +# Check that grep can do binary searches. Some, e.g. busybox, can't. Leave a
> +# warning but don't fail the build.
> +echo "X" | grep -aob "X" -q 2>/dev/null ||
> + { echo "$MSG_PFX Warning: grep can't do binary searches" >&2; exit 0; }
> +
> +#
> +# First, look for all the valid endbr64 instructions.
> +# A worst-case disassembly, viewed through cat -A, may look like:
> +#
> +# ffff82d040337bd4 <endbr64>:$
> +# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
> +# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
> +# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
> +#
> +# Want to grab the address of endbr64 instructions only, ignoring function
> +# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
> +# number of trailing spaces before the end of the line.
> +#
> +${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
> +
> +#
> +# Second, look for any endbr64 byte sequence
> +# This has a couple of complications:
> +#
> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
> +# the grep offset to be from the start of .text.
> +#
> +# 2) dash's printf doesn't understand hex escapes, hence the use of octal.
> +#
> +# 3) AWK can't add 64bit integers, because internally all numbers are doubles.
> +# When the upper bits are set, the exponents worth of precision is lost in
> +# the lower bits, rounding integers to the nearest 4k.
> +#
> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
> +# numbers, which don't lose precision.
> +#
> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
> +
> +${OBJCOPY} -O binary $TEXT_BIN
..., like here, is then somewhat misleading considering that the tool
can take one or two filenames as arguments.
Jan
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions
2022-02-23 11:31 ` Jan Beulich
@ 2022-02-23 12:05 ` Andrew Cooper
2022-02-23 14:29 ` Jan Beulich
0 siblings, 1 reply; 28+ messages in thread
From: Andrew Cooper @ 2022-02-23 12:05 UTC (permalink / raw)
To: Jan Beulich
Cc: Marek Marczykowski-Górecki, Roger Pau Monne, Wei Liu, Xen-devel
On 23/02/2022 11:31, Jan Beulich wrote:
> On 22.02.2022 16:26, Andrew Cooper wrote:
>> up on a non-instruction boundary. Such embedded instructions mark legal
>> indirect branch targets as far as the CPU is concerned, which aren't legal as
>> far as the logic is concerned.
> Thinking about it: Wouldn't it be yet slightly more reassuring to also
> look for ENDBR32?
I considered that, but it's awkward to do and doubles the length of this
already ~0.7s (x2 for efi because this step isn't performed in parallel)
delay to the build.
We do not have __HYPERVISOR_CS32, so ENDBR32 will yield #CP[endbr] if
encountered.
If an attacker has managed to edit the GDT to insert a compatibility
code segment, and hijacked a far transfer to use it, then the absence of
ENDBR32's in the binary isn't going to be an impediment.
>
>> When CET-IBT is active, check for embedded byte sequences. Example failures
>> look like:
>>
>> check-endbr.sh xen-syms Fail: Found 2 embedded endbr64 instructions
>> 0xffff82d040325677: test_endbr64 at /local/xen.git/xen/arch/x86/x86_64/entry.S:28
>> 0xffff82d040352da6: init_done at /local/xen.git/xen/arch/x86/setup.c:675
>>
>> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
Thanks.
>
>> --- a/README
>> +++ b/README
>> @@ -68,6 +68,7 @@ provided by your OS distributor:
>> In addition to the above there are a number of optional build
>> prerequisites. Omitting these will cause the related features to be
>> disabled at compile time:
>> + * Binary-search capable grep (if building Xen with CET support)
> Nit: With this (maybe this was the case already earlier though)
> s/will/may/ in the previous sentence?
I'm planning a separate overhaul to README because bits of it are quite
wrong, including lots of this section. This was the lead bad addition I
could come up with that didn't involve a major rewrite.
>> --- /dev/null
>> +++ b/xen/tools/check-endbr.sh
>> @@ -0,0 +1,85 @@
>> +#!/bin/sh
>> +#
>> +# Usage ./$0 xen-syms
>> +#
>> +set -e
>> +
>> +# Prettyprint parameters a little for message
>> +MSG_PFX="${0##*/} ${1##*/}"
>> +
>> +OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
>> +OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
> While embedding the arguments here shortens the lines where these are
> used, the appearance especially of $OBJCOPY with a single file name
> argument ...
>
>> +ADDR2LINE="${ADDR2LINE:-addr2line}"
>> +
>> +D=$(mktemp -d)
>> +trap "rm -rf $D" EXIT
>> +
>> +TEXT_BIN=$D/xen-syms.text
>> +VALID=$D/valid-addrs
>> +ALL=$D/all-addrs
>> +BAD=$D/bad-addrs
>> +
>> +# Check that grep can do binary searches. Some, e.g. busybox, can't. Leave a
>> +# warning but don't fail the build.
>> +echo "X" | grep -aob "X" -q 2>/dev/null ||
>> + { echo "$MSG_PFX Warning: grep can't do binary searches" >&2; exit 0; }
>> +
>> +#
>> +# First, look for all the valid endbr64 instructions.
>> +# A worst-case disassembly, viewed through cat -A, may look like:
>> +#
>> +# ffff82d040337bd4 <endbr64>:$
>> +# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
>> +# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
>> +# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
>> +#
>> +# Want to grab the address of endbr64 instructions only, ignoring function
>> +# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
>> +# number of trailing spaces before the end of the line.
>> +#
>> +${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
>> +
>> +#
>> +# Second, look for any endbr64 byte sequence
>> +# This has a couple of complications:
>> +#
>> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
>> +# the grep offset to be from the start of .text.
>> +#
>> +# 2) dash's printf doesn't understand hex escapes, hence the use of octal.
>> +#
>> +# 3) AWK can't add 64bit integers, because internally all numbers are doubles.
>> +# When the upper bits are set, the exponents worth of precision is lost in
>> +# the lower bits, rounding integers to the nearest 4k.
>> +#
>> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
>> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
>> +# numbers, which don't lose precision.
>> +#
>> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
>> +
>> +${OBJCOPY} -O binary $TEXT_BIN
> ..., like here, is then somewhat misleading considering that the tool
> can take one or two filenames as arguments.
I can re-expand them if you'd prefer. This would be the delta:
diff --git a/xen/tools/check-endbr.sh b/xen/tools/check-endbr.sh
index 85878353112a..3019ca1c7db0 100755
--- a/xen/tools/check-endbr.sh
+++ b/xen/tools/check-endbr.sh
@@ -7,8 +7,8 @@ set -e
# Prettyprint parameters a little for message
MSG_PFX="${0##*/} ${1##*/}"
-OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
-OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
+OBJCOPY="${OBJCOPY:-objcopy}"
+OBJDUMP="${OBJDUMP:-objdump}"
ADDR2LINE="${ADDR2LINE:-addr2line}"
D=$(mktemp -d)
@@ -37,7 +37,7 @@ echo "X" | grep -aob "X" -q 2>/dev/null ||
# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and
with any
# number of trailing spaces before the end of the line.
#
-${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
+${OBJDUMP} -j .text $1 -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' >
$VALID &
#
# Second, look for any endbr64 byte sequence
@@ -56,9 +56,10 @@ ${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1
-d ':' > $VALID &
# split the VMA in half so AWK's numeric addition is only working on
32 bit
# numbers, which don't lose precision.
#
-eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf
"vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
+eval $(${OBJDUMP} -j .text $1 -h |
+ awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1,
8), substr($4, 9, 16)}')
-${OBJCOPY} -O binary $TEXT_BIN
+${OBJCOPY} -j .text $1 -O binary $TEXT_BIN
grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
awk -F':' '{printf "%s%x\n", "'$vma_hi'", int(0x'$vma_lo') + $1}' >
$ALL
~Andrew
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH v3 59/70] x86: Use control flow typechecking where possible
2022-02-22 15:26 ` [PATCH v3 59/70] x86: Use control flow typechecking where possible Andrew Cooper
@ 2022-02-23 14:21 ` Jan Beulich
2022-02-23 14:28 ` Andrew Cooper
0 siblings, 1 reply; 28+ messages in thread
From: Jan Beulich @ 2022-02-23 14:21 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 22.02.2022 16:26, Andrew Cooper wrote:
> Now all callees have been annotated, turn on typechecking to catch issues in
> the future.
>
> This extension isn't in a released version of GCC yet, so provide a container
> to use with the extention included, and add it to CI. RANDCONFIG is necessary
> because some stubs for compiled-out subsystems are used as function pointer
> targets.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
I certainly see that we will want to have this extra level of safety.
Hence feel free to add
Acked-by: Jan Beulich <jbeulich@suse.com>
But I'd like us to form a plan in case the change doesn't get committed
relatively soon after gcc re-opens for general development after gcc12
was branched off. We don't want to get into the position of needing to
maintain a gcc patch. Do you have any insight as to what is actually
blocking the committing of that change, seeing that it has been around
for a while?
Jan
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
>
> v3:
> * Provide container. Already pushed to Gitlab.
> ---
> automation/build/debian/buster-gcc-ibt.dockerfile | 66 +++++++++++++++++++++++
> automation/gitlab-ci/build.yaml | 6 +++
> automation/scripts/containerize | 1 +
> xen/arch/x86/arch.mk | 1 +
> 4 files changed, 74 insertions(+)
> create mode 100644 automation/build/debian/buster-gcc-ibt.dockerfile
>
> diff --git a/automation/build/debian/buster-gcc-ibt.dockerfile b/automation/build/debian/buster-gcc-ibt.dockerfile
> new file mode 100644
> index 000000000000..441d9a9ab37a
> --- /dev/null
> +++ b/automation/build/debian/buster-gcc-ibt.dockerfile
> @@ -0,0 +1,66 @@
> +FROM debian:buster-slim AS builder
> +
> +ENV DEBIAN_FRONTEND=noninteractive
> +ENV USER root
> +
> +RUN apt-get update && \
> + apt-get --quiet --yes install \
> + bison \
> + build-essential \
> + flex \
> + libc6-dev-i386 \
> + libgmp-dev \
> + libisl-dev \
> + libmpc-dev \
> + libmpfr-dev \
> + patch \
> + wget
> +
> +RUN mkdir /build
> +WORKDIR /build
> +
> +RUN wget -q https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz -O - | tar xJ --strip=1
> +RUN wget -q https://xenbits.xen.org/people/andrewcoop/gcc-11.2-Add-fcf-check-attribute-yes-no.patch -O - | patch -p1
> +RUN ./configure \
> + --prefix=/opt/gcc-11-ibt \
> + --enable-languages=c \
> + --disable-nls \
> + --disable-threads \
> + --disable-bootstrap \
> + --disable-shared \
> + --disable-libmudflap \
> + --disable-libssp \
> + --disable-libgomp \
> + --disable-decimal-float \
> + --disable-libquadmath \
> + --disable-libatomic \
> + --disable-libcc1 \
> + --disable-libmpx
> +RUN make -j`nproc` && make -j`nproc` install
> +
> +
> +FROM debian:buster-slim
> +COPY --from=builder /opt/gcc-11-ibt /opt/gcc-11-ibt
> +
> +LABEL maintainer.name="The Xen Project" \
> + maintainer.email="xen-devel@lists.xenproject.org"
> +
> +ENV DEBIAN_FRONTEND=noninteractive
> +ENV USER root
> +ENV PATH="/opt/gcc-11-ibt/bin:${PATH}"
> +
> +RUN mkdir /build
> +WORKDIR /build
> +
> +RUN apt-get update && \
> + apt-get --quiet --yes install \
> + bison \
> + checkpolicy \
> + flex \
> + gawk \
> + make \
> + python3 \
> + && \
> + apt-get autoremove -y && \
> + apt-get clean && \
> + rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index fdd5c76582b3..cc36428cf55b 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -294,6 +294,12 @@ debian-stretch-32-gcc-debug:
> variables:
> CONTAINER: debian:stretch-i386
>
> +debian-buster-gcc-ibt:
> + extends: .gcc-x86-64-build
> + variables:
> + CONTAINER: debian:buster-gcc-ibt
> + RANDCONFIG: y
> +
> debian-unstable-clang:
> extends: .clang-x86-64-build
> variables:
> diff --git a/automation/scripts/containerize b/automation/scripts/containerize
> index 7682ccd34759..8992c67278ae 100755
> --- a/automation/scripts/containerize
> +++ b/automation/scripts/containerize
> @@ -33,6 +33,7 @@ case "_${CONTAINER}" in
> _focal) CONTAINER="${BASE}/ubuntu:focal" ;;
> _jessie) CONTAINER="${BASE}/debian:jessie" ;;
> _stretch|_) CONTAINER="${BASE}/debian:stretch" ;;
> + _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
> _unstable|_) CONTAINER="${BASE}/debian:unstable" ;;
> _trusty) CONTAINER="${BASE}/ubuntu:trusty" ;;
> _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;;
> diff --git a/xen/arch/x86/arch.mk b/xen/arch/x86/arch.mk
> index f780c912a9cf..92fd19811013 100644
> --- a/xen/arch/x86/arch.mk
> +++ b/xen/arch/x86/arch.mk
> @@ -54,6 +54,7 @@ endif
>
> ifdef CONFIG_XEN_IBT
> CFLAGS += -fcf-protection=branch -mmanual-endbr
> +$(call cc-option-add,CFLAGS,CC,-fcf-check-attribute=no)
> else
> $(call cc-option-add,CFLAGS,CC,-fcf-protection=none)
> endif
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 59/70] x86: Use control flow typechecking where possible
2022-02-23 14:21 ` Jan Beulich
@ 2022-02-23 14:28 ` Andrew Cooper
0 siblings, 0 replies; 28+ messages in thread
From: Andrew Cooper @ 2022-02-23 14:28 UTC (permalink / raw)
To: Jan Beulich; +Cc: Roger Pau Monne, Wei Liu, Xen-devel
On 23/02/2022 14:21, Jan Beulich wrote:
> On 22.02.2022 16:26, Andrew Cooper wrote:
>> Now all callees have been annotated, turn on typechecking to catch issues in
>> the future.
>>
>> This extension isn't in a released version of GCC yet, so provide a container
>> to use with the extention included, and add it to CI. RANDCONFIG is necessary
>> because some stubs for compiled-out subsystems are used as function pointer
>> targets.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> I certainly see that we will want to have this extra level of safety.
> Hence feel free to add
> Acked-by: Jan Beulich <jbeulich@suse.com>
Thanks.
> But I'd like us to form a plan in case the change doesn't get committed
> relatively soon after gcc re-opens for general development after gcc12
> was branched off. We don't want to get into the position of needing to
> maintain a gcc patch. Do you have any insight as to what is actually
> blocking the committing of that change, seeing that it has been around
> for a while?
I'll follow up and try to unblock.
~Andrew
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions
2022-02-23 12:05 ` Andrew Cooper
@ 2022-02-23 14:29 ` Jan Beulich
0 siblings, 0 replies; 28+ messages in thread
From: Jan Beulich @ 2022-02-23 14:29 UTC (permalink / raw)
To: Andrew Cooper
Cc: Marek Marczykowski-Górecki, Roger Pau Monne, Wei Liu, Xen-devel
On 23.02.2022 13:05, Andrew Cooper wrote:
> On 23/02/2022 11:31, Jan Beulich wrote:
>> On 22.02.2022 16:26, Andrew Cooper wrote:
>>> up on a non-instruction boundary. Such embedded instructions mark legal
>>> indirect branch targets as far as the CPU is concerned, which aren't legal as
>>> far as the logic is concerned.
>> Thinking about it: Wouldn't it be yet slightly more reassuring to also
>> look for ENDBR32?
>
> I considered that, but it's awkward to do and doubles the length of this
> already ~0.7s (x2 for efi because this step isn't performed in parallel)
> delay to the build.
(Side note: In general the two linking steps can occur in parallel. An
exception is when the note.o need to be extracted from xen-syms for use
by xen.efi. But that should happen only with old binutils.)
> We do not have __HYPERVISOR_CS32, so ENDBR32 will yield #CP[endbr] if
> encountered.
>
> If an attacker has managed to edit the GDT to insert a compatibility
> code segment, and hijacked a far transfer to use it, then the absence of
> ENDBR32's in the binary isn't going to be an impediment.
True.
>>> --- /dev/null
>>> +++ b/xen/tools/check-endbr.sh
>>> @@ -0,0 +1,85 @@
>>> +#!/bin/sh
>>> +#
>>> +# Usage ./$0 xen-syms
>>> +#
>>> +set -e
>>> +
>>> +# Prettyprint parameters a little for message
>>> +MSG_PFX="${0##*/} ${1##*/}"
>>> +
>>> +OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
>>> +OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
>> While embedding the arguments here shortens the lines where these are
>> used, the appearance especially of $OBJCOPY with a single file name
>> argument ...
>>
>>> +ADDR2LINE="${ADDR2LINE:-addr2line}"
>>> +
>>> +D=$(mktemp -d)
>>> +trap "rm -rf $D" EXIT
>>> +
>>> +TEXT_BIN=$D/xen-syms.text
>>> +VALID=$D/valid-addrs
>>> +ALL=$D/all-addrs
>>> +BAD=$D/bad-addrs
>>> +
>>> +# Check that grep can do binary searches. Some, e.g. busybox, can't. Leave a
>>> +# warning but don't fail the build.
>>> +echo "X" | grep -aob "X" -q 2>/dev/null ||
>>> + { echo "$MSG_PFX Warning: grep can't do binary searches" >&2; exit 0; }
>>> +
>>> +#
>>> +# First, look for all the valid endbr64 instructions.
>>> +# A worst-case disassembly, viewed through cat -A, may look like:
>>> +#
>>> +# ffff82d040337bd4 <endbr64>:$
>>> +# ffff82d040337bd4:^If3 0f 1e fa ^Iendbr64 $
>>> +# ffff82d040337bd8:^Ieb fe ^Ijmp ffff82d040337bd8 <endbr64+0x4>$
>>> +# ffff82d040337bda:^Ib8 f3 0f 1e fa ^Imov $0xfa1e0ff3,%eax$
>>> +#
>>> +# Want to grab the address of endbr64 instructions only, ignoring function
>>> +# names/jump labels/etc, so look for 'endbr64' preceeded by a tab and with any
>>> +# number of trailing spaces before the end of the line.
>>> +#
>>> +${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
>>> +
>>> +#
>>> +# Second, look for any endbr64 byte sequence
>>> +# This has a couple of complications:
>>> +#
>>> +# 1) Grep binary search isn't VMA aware. Copy .text out as binary, causing
>>> +# the grep offset to be from the start of .text.
>>> +#
>>> +# 2) dash's printf doesn't understand hex escapes, hence the use of octal.
>>> +#
>>> +# 3) AWK can't add 64bit integers, because internally all numbers are doubles.
>>> +# When the upper bits are set, the exponents worth of precision is lost in
>>> +# the lower bits, rounding integers to the nearest 4k.
>>> +#
>>> +# Instead, use the fact that Xen's .text is within a 1G aligned region, and
>>> +# split the VMA in half so AWK's numeric addition is only working on 32 bit
>>> +# numbers, which don't lose precision.
>>> +#
>>> +eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
>>> +
>>> +${OBJCOPY} -O binary $TEXT_BIN
>> ..., like here, is then somewhat misleading considering that the tool
>> can take one or two filenames as arguments.
>
> I can re-expand them if you'd prefer. This would be the delta:
I'd actually be happy to keep "-j .text" where you had it, and merely
move the file arguments to the actual invocation lines. But the way
you have it with the incremental diff is of course even less
"unexpected".
Jan
> diff --git a/xen/tools/check-endbr.sh b/xen/tools/check-endbr.sh
> index 85878353112a..3019ca1c7db0 100755
> --- a/xen/tools/check-endbr.sh
> +++ b/xen/tools/check-endbr.sh
> @@ -7,8 +7,8 @@ set -e
> # Prettyprint parameters a little for message
> MSG_PFX="${0##*/} ${1##*/}"
>
> -OBJCOPY="${OBJCOPY:-objcopy} -j .text $1"
> -OBJDUMP="${OBJDUMP:-objdump} -j .text $1"
> +OBJCOPY="${OBJCOPY:-objcopy}"
> +OBJDUMP="${OBJDUMP:-objdump}"
> ADDR2LINE="${ADDR2LINE:-addr2line}"
>
> D=$(mktemp -d)
> @@ -37,7 +37,7 @@ echo "X" | grep -aob "X" -q 2>/dev/null ||
> # names/jump labels/etc, so look for 'endbr64' preceeded by a tab and
> with any
> # number of trailing spaces before the end of the line.
> #
> -${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' > $VALID &
> +${OBJDUMP} -j .text $1 -d -w | grep ' endbr64 *$' | cut -f 1 -d ':' >
> $VALID &
>
> #
> # Second, look for any endbr64 byte sequence
> @@ -56,9 +56,10 @@ ${OBJDUMP} -d -w | grep ' endbr64 *$' | cut -f 1
> -d ':' > $VALID &
> # split the VMA in half so AWK's numeric addition is only working on
> 32 bit
> # numbers, which don't lose precision.
> #
> -eval $(${OBJDUMP} -h | awk '$2 == ".text" {printf
> "vma_hi=%s\nvma_lo=%s\n", substr($4, 1, 8), substr($4, 9, 16)}')
> +eval $(${OBJDUMP} -j .text $1 -h |
> + awk '$2 == ".text" {printf "vma_hi=%s\nvma_lo=%s\n", substr($4, 1,
> 8), substr($4, 9, 16)}')
>
> -${OBJCOPY} -O binary $TEXT_BIN
> +${OBJCOPY} -j .text $1 -O binary $TEXT_BIN
> grep -aob "$(printf '\363\17\36\372')" $TEXT_BIN |
> awk -F':' '{printf "%s%x\n", "'$vma_hi'", int(0x'$vma_lo') + $1}' >
> $ALL
>
>
> ~Andrew
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH v3 06/70] x86: Introduce support for CET-IBT
2022-02-22 15:26 ` [PATCH v3 06/70] x86: Introduce support for CET-IBT Andrew Cooper
@ 2022-02-28 12:58 ` Jan Beulich
0 siblings, 0 replies; 28+ messages in thread
From: Jan Beulich @ 2022-02-28 12:58 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Roger Pau Monné, Wei Liu, Xen-devel
On 22.02.2022 16:26, Andrew Cooper wrote:
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -40,6 +40,11 @@ config HAS_AS_CET_SS
> # binutils >= 2.29 or LLVM >= 6
> def_bool $(as-instr,wrssq %rax$(comma)0;setssbsy)
>
> +config HAS_CC_CET_IBT
> + # GCC >= 9 and binutils >= 2.29
> + # Retpoline check to work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93654
> + def_bool $(cc-option,-fcf-protection=branch -mmanual-endbr -mindirect-branch=thunk-extern) && $(as-instr,endbr64)
Noticed only now: Since the commit message doesn't say anything either
way, was it intentional for this to differ from XEN_SHSTK in the EXPERT
dependency?
Jan
^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2022-02-28 12:58 UTC | newest]
Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-22 15:26 [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 01/70] xen/sort: Switch to an extern inline implementation Andrew Cooper
2022-02-22 15:52 ` Julien Grall
2022-02-22 15:26 ` [PATCH v3 03/70] x86/pv-shim: Don't modify the hypercall table Andrew Cooper
2022-02-22 16:24 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 05/70] x86/kexec: Annotate embedded data with ELF metadata Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 06/70] x86: Introduce support for CET-IBT Andrew Cooper
2022-02-28 12:58 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 08/70] xen: CFI hardening for custom_param() Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 12/70] xen: CFI hardening for continue_hypercall_on_cpu() Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 26/70] xen/iommu: CFI hardening Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 27/70] xen/video: " Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 29/70] xen/misc: " Andrew Cooper
2022-02-23 10:25 ` Jan Beulich
2022-02-23 10:34 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 33/70] x86/emul: " Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 46/70] x86/logdirty: " Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 47/70] x86/shadow: " Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 59/70] x86: Use control flow typechecking where possible Andrew Cooper
2022-02-23 14:21 ` Jan Beulich
2022-02-23 14:28 ` Andrew Cooper
2022-02-22 15:26 ` [PATCH v3 60/70] x86: Build check for embedded endbr64 instructions Andrew Cooper
2022-02-23 11:31 ` Jan Beulich
2022-02-23 12:05 ` Andrew Cooper
2022-02-23 14:29 ` Jan Beulich
2022-02-22 15:26 ` [PATCH v3 64/70] x86: Introduce helpers/checks for " Andrew Cooper
2022-02-22 15:29 ` [PATCH v3 00/70 (not all posted)] x86: Support for CET Indirect Branch Tracking Jan Beulich
2022-02-22 15:41 ` Andrew Cooper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.