From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760204AbcCDUtG (ORCPT ); Fri, 4 Mar 2016 15:49:06 -0500 Received: from mail-bn1bon0069.outbound.protection.outlook.com ([157.56.111.69]:64651 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1760172AbcCDUtB (ORCPT ); Fri, 4 Mar 2016 15:49:01 -0500 Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=amd.com; From: Suravee Suthikulpanit To: , , , , , CC: , , , , Suravee Suthikulpanit , Suravee Suthikulpanit Subject: [PART1 RFC v2 05/10] KVM: x86: Detect and Initialize AVIC support Date: Fri, 4 Mar 2016 14:46:03 -0600 Message-ID: <1457124368-2025-6-git-send-email-Suravee.Suthikulpanit@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1457124368-2025-1-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1457124368-2025-1-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [124.121.8.20] X-ClientProxiedBy: SINPR01CA0045.apcprd01.prod.exchangelabs.com (10.141.109.45) To SN1PR12MB0445.namprd12.prod.outlook.com (25.162.105.139) X-MS-Office365-Filtering-Correlation-Id: 32e10356-5c01-4d74-ee56-08d3446e6837 X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0445;2:jsNS1RS2bIbSrFrLnaHpJZ705aKmrGbTK4x7l7JIazh3PDMLEzFZMScvp82jt4tNT0vGIJGZYJ5hnaElkR0w7EiKfz5MqAmrCuWKQ7ORYZPT4y1+EctuOl2Zvo4wnS+/+LJNzXqJyAJ65lqInk77nXW0HbfHbpopY+OylXh89dE7vRB4FrCxQy4Q606Mgu58;3:MNFUT4619RmcRwnRwrRFqypo2tVHauvrClE4Fr8fVxnurUjCGRsEKmilW2mO1A2xK31OvZPToUYkDMIzVVDzq0Xrd2pAPv5MOz+pGfkb9hiMyi6eI3u0FsamI6fJ4hPc;25:OFdqRN3K9MM8d0WnmGgmuxi9Mo6eFq3AgNaQBCPjTxJv4ZoXNHhHHGsC982068NwGn7/CiNN5FL32b/bYQhtxDeoTObZfffobCblzvbu4taZOhmXFTVjyB3Lw6LKtfZl33YesbUQ97Abf5nOrPY8+iPsOKSslr8CwFtLwUwYfvoZC08R7XRGPH+QZX0mORoL4m4/Bs8V6y1XQOLvXWFkCVHsON7t5vHvO9Wyot5gopmBCGdwETtgA+kc10WUg2yB91azBw6/R+xibfFYJ8dZz3EXydQlm2qV+eq2POka2f1TLBdqtb/RlTCf88UY9p8x22VHbsvBzFAHsyeDhVai5Q== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0445; X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0445;20:jJLk6U3Jxn70h8sJH9gWUe7R322w4TfT/wp3nvGP4/TJ9NNNr4pPsfxHbFd92mHs3/TU/tGuG7fM+2MqZbkTHcirvVc9xdzDK+lf3mT6NAJVFrpt1W6xTJ+KefYE3WzFgv4nMVHasNnVuAiZCpUvx5hEW889fE/Tjq2K6oEQ0PVfreH2D3QpTGZKcHtGb7Cr996xf3cu+0R94SpiuE2c0ny0MuCggAluzaaA5xqB/1cw0IULRxoN3s447i/apABmp996hG3KPslL6Z98UCnNm9FEp1fBEu2oka516UXoxDIuub0buXTdhgOm539d3PPFcWKvz47pfK9omMSNKJZ+xbSyFNssQjx8xjX7wL5gAHMy547drkE6i+lGFNz9+PjiaRkAeI0V6cPQXaiRcntGKZgMD9LDX8ocMivq7Fh7hL7cqRmHnubRpkHl/Gwj3l4OzdHCkjmH9rideGeNpXef1y4wgdUttQIiOqUK6nSDiCoZv4i/I3MkqLo6dFFtu/Pb;4:K5UvAVhavMuMriY/PQWxbfc58cQDyoqrrcB1OkFYvjQkkPsxj1+McMRdzuBhqU0+Mq7f9xIV5wcIk0knLQUbZr0bK8mrFvkKoPpR8Og8eWzCIPaxuzRV+jpth8ts5+vV0Qj3QbOG/CglOTZIuOvciy7fzlQrOj+GrX723g5qCk1ACHmop/JrlXH2MKcVJIZZc/j9zbKbOBjtAOh/SDxU+O4jJM2oklBRgbaaX5HQZyZkHEmOUfUdHBHPtfUDX8L4aN2dIye0sWhrPG0rYM9WHz209BaajYj71NZ6c9rLjtDGcVz5Z5fMDePI2U++2e18VZ7sWC/I9KzMFbDExhjoy/Q0p6fSM4XOhlMCXuMVeOHNcbWT6t3JG6dVuW8iwtQV X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);SRVR:SN1PR12MB0445;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0445; X-Forefront-PRVS: 0871917CDA X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6069001)(6009001)(77096005)(5001770100001)(87976001)(48376002)(47776003)(229853001)(36756003)(2201001)(66066001)(81166005)(50466002)(2950100001)(5004730100002)(189998001)(40100003)(50226001)(5003940100001)(122386002)(5008740100001)(19580405001)(2906002)(19580395003)(86362001)(4326007)(42186005)(50986999)(6116002)(3846002)(1096002)(92566002)(586003)(76176999);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR12MB0445;H:localhost.localdomain;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN1PR12MB0445;23:wDLDytSLx4CMtVVCxP+0UpT3qKXlfes0DwyVKWpfx?= =?us-ascii?Q?1Y2xmyBoSTN74IW9VmoZ1uGSkFy69bBvt4EOkTRcp/xtgmSOaIY6/IxTwdWU?= =?us-ascii?Q?4msid+VVIsW5gud+4Q6mNdAOAg7KUUhXgNxaZvoN3mhOMu8ZxDI/xxZaO7u3?= =?us-ascii?Q?sOz0NaB2AjgJj+sXhtpTlzu2YE+5/yk3fRJMUiJZQweSB6UvbWn1ihCEi4DM?= =?us-ascii?Q?8vIo0apekyyb4E5rleVHpr2kPZkIE3jgygKISux/JALq7NInRrDtu+6hd10J?= =?us-ascii?Q?sBhnv180X2eh+K/XAKqOa6cMFh3z/ZkHYXTCbB0DjYe2AqMhxmCErggVlh9k?= =?us-ascii?Q?DybpRo8bTidy/GaAfC6uxrkOtN2SlolIlvdWORjG+vr96GEY1efiWyEw11Is?= =?us-ascii?Q?6yUS9bFGYMQ1vVaK7nYlPsjYvya1uWEQATKVwp/ftdIFriiHQU0Xvuc98Sdz?= =?us-ascii?Q?PhaAagIXeSiMI3Pg1KK5LNOOLcjqilYee05IXJ9Fo/yz6F70UGLS75Y4Uyxq?= =?us-ascii?Q?KnzGoodq833zZ84Wpwam5Hnd7D74UJLA0Jd217inZC2ehy/ngn6huVD8XC4F?= =?us-ascii?Q?oYHmFibqbdrTCckg7svv4QNm5ilyT8V75GEBld7KZw32srCDiz0ovcz1shKf?= =?us-ascii?Q?gcUHpTLQQnJOj8I1Un+4yOnKEa3qktqu8cn+fRDYt2qUWZo0lBZLds5TSvAN?= =?us-ascii?Q?KCsWbjvKb157xy9cN5bvESOaLWBprxYJl33h2Wuz/JeIju84okh2/vBjrZRr?= =?us-ascii?Q?XqbkCSC3f11AMWg/qnlMMxARyG7auP/RevwZ3K764KPUSkZuh7U3iXe7w91G?= =?us-ascii?Q?r8zxisoddkcIBc6Pu0026MJCbJhXrUFg6UMwOHrwoMR5GzGK/PUGYiJWgbGk?= =?us-ascii?Q?0waUw2kZNbo1cZmkBk/2u0zwK33c5Ii2nGrnO9PWBB8M16NwmBvbJi/JYgCA?= =?us-ascii?Q?tfH8pU1p/5UvFfC5lIg3YPpM8iu9XRpFwRcLHYQGg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0445;5:8l5sXHxHulSem8jpZ1tF1cdmj2c3Sjs72Qz60PyAcplhuZ3i92R9L8pIuBN9cVU8HyqBUzW8BcTlSDscxypMac6g/KRB/xrig6TECujZhPHfuWnmeT7v4sL5SOuPwqLvghUuNFBZ3KZ+sbrDEgO68A==;24:0O0Sy7BWgDEBDkNp2zkWYyZXp6nevE6gp/6X2GzgortSKoferNhBxOG+kAjHYEDKt1dwcjsPL816r9AQD1kWpApOmQ4qtuNAjD6iTZFBCQg=;20:ZLvnuKgAG2ZAni4G2ESPd8PkjYoBwcSr8quXr5P7EnN0kN5JodLib/R1HYeBmXJSD1GjrbKwfpLEETbQ8i5/6QndRWlahRg2jGqgi4KoIFUg1QUUkjnTBPyrlNnyTlmr17FWrsn6qt1nltbFl72uZPcGteyFxeDnR+eFEY6Fz2vAATFyHBZlwXfuSSMkGx1rhiDyFnWbsqSOHngs10E1PDya3O5vpf31oSNQmZn8PrwLIAiq7rRKONkpdg4tFRnd X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2016 20:48:48.6361 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0445 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch introduces AVIC-related data structure, and AVIC initialization code. There are three main data structures for AVIC: * Virtual APIC (vAPIC) backing page (per-VCPU) * Physical APIC ID table (per-VM) * Logical APIC ID table (per-VM) In order to accommodate the new per-VM tables, we introduce a new per-VM arch-specific void pointer, struct kvm_arch.arch_data. This will point to the newly introduced struct svm_vm_data. Currently, AVIC is disabled by default. Users can manually enable AVIC via kernel boot option kvm-amd.avic=1 or during kvm-amd module loading with parameter avic=1. Signed-off-by: Suravee Suthikulpanit --- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/svm.c | 416 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 417 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 9a61669..15f6cd8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -754,6 +754,8 @@ struct kvm_arch { bool irqchip_split; u8 nr_reserved_ioapic_pins; + + void *arch_data; }; struct kvm_vm_stat { diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 6ab66d0..c703149 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -14,6 +14,9 @@ * the COPYING file in the top-level directory. * */ + +#define pr_fmt(fmt) "SVM: " fmt + #include #include "irq.h" @@ -78,6 +81,11 @@ MODULE_DEVICE_TABLE(x86cpu, svm_cpu_id); #define TSC_RATIO_MIN 0x0000000000000001ULL #define TSC_RATIO_MAX 0x000000ffffffffffULL +#define AVIC_HPA_MASK ~((0xFFFULL << 52) || 0xFFF) + +/* NOTE: Current max index allowed for physical APIC ID table is 255 */ +#define AVIC_PHY_APIC_ID_MAX 0xFF + static bool erratum_383_found __read_mostly; static const u32 host_save_user_msrs[] = { @@ -162,6 +170,37 @@ struct vcpu_svm { /* cached guest cpuid flags for faster access */ bool nrips_enabled : 1; + + struct page *avic_bk_page; + void *in_kernel_lapic_regs; +}; + +struct __attribute__ ((__packed__)) +svm_avic_log_ait_entry { + u32 guest_phy_apic_id : 8, + res : 23, + valid : 1; +}; + +struct __attribute__ ((__packed__)) +svm_avic_phy_ait_entry { + u64 host_phy_apic_id : 8, + res1 : 4, + bk_pg_ptr : 40, + res2 : 10, + is_running : 1, + valid : 1; +}; + +/* Note: This structure is per VM */ +struct svm_vm_data { + atomic_t count; + u32 ldr_mode; + u32 avic_max_vcpu_id; + u32 avic_tag; + + struct page *avic_log_ait_page; + struct page *avic_phy_ait_page; }; static DEFINE_PER_CPU(u64, current_tsc_ratio); @@ -205,6 +244,10 @@ module_param(npt, int, S_IRUGO); static int nested = true; module_param(nested, int, S_IRUGO); +/* enable / disable AVIC */ +static int avic; +module_param(avic, int, S_IRUGO); + static void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0); static void svm_flush_tlb(struct kvm_vcpu *vcpu); static void svm_complete_interrupts(struct vcpu_svm *svm); @@ -234,6 +277,13 @@ enum { /* TPR and CR2 are always written before VMRUN */ #define VMCB_ALWAYS_DIRTY_MASK ((1U << VMCB_INTR) | (1U << VMCB_CR2)) +#define VMCB_AVIC_APIC_BAR_MASK 0xFFFFFFFFFF000ULL + +static inline void avic_update_vapic_bar(struct vcpu_svm *svm, u64 data) +{ + svm->vmcb->control.avic_vapic_bar = data & VMCB_AVIC_APIC_BAR_MASK; +} + static inline void mark_all_dirty(struct vmcb *vmcb) { vmcb->control.clean = 0; @@ -923,6 +973,13 @@ static __init int svm_hardware_setup(void) } else kvm_disable_tdp(); + if (avic && (!npt_enabled || !boot_cpu_has(X86_FEATURE_AVIC))) + avic = false; + + if (avic) { + printk(KERN_INFO "kvm: AVIC enabled\n"); + } + return 0; err: @@ -1000,6 +1057,41 @@ static void svm_adjust_tsc_offset_guest(struct kvm_vcpu *vcpu, s64 adjustment) mark_dirty(svm->vmcb, VMCB_INTERCEPTS); } +static void avic_init_vmcb(struct vcpu_svm *svm) +{ + void *vapic_bkpg; + struct vmcb *vmcb = svm->vmcb; + struct svm_vm_data *vm_data = svm->vcpu.kvm->arch.arch_data; + phys_addr_t bpa = PFN_PHYS(page_to_pfn(svm->avic_bk_page)); + phys_addr_t lpa = PFN_PHYS(page_to_pfn(vm_data->avic_log_ait_page)); + phys_addr_t ppa = PFN_PHYS(page_to_pfn(vm_data->avic_phy_ait_page)); + + if (!vmcb) + return; + + pr_debug("%s: bpa=%#llx, lpa=%#llx, ppa=%#llx\n", + __func__, (unsigned long long)bpa, + (unsigned long long)lpa, (unsigned long long)ppa); + + + /* + * Here we copy the value from the emulated LAPIC register + * to the vAPIC backign page, and then flip regs frame. + */ + if (!svm->in_kernel_lapic_regs) + svm->in_kernel_lapic_regs = svm->vcpu.arch.apic->regs; + + vapic_bkpg = pfn_to_kaddr(page_to_pfn(svm->avic_bk_page)); + memcpy(vapic_bkpg, svm->in_kernel_lapic_regs, PAGE_SIZE); + svm->vcpu.arch.apic->regs = vapic_bkpg; + + vmcb->control.avic_bk_page = bpa & AVIC_HPA_MASK; + vmcb->control.avic_log_apic_id = lpa & AVIC_HPA_MASK; + vmcb->control.avic_phy_apic_id = ppa & AVIC_HPA_MASK; + vmcb->control.avic_phy_apic_id |= AVIC_PHY_APIC_ID_MAX; + vmcb->control.avic_enable = 1; +} + static void init_vmcb(struct vcpu_svm *svm) { struct vmcb_control_area *control = &svm->vmcb->control; @@ -1113,6 +1205,293 @@ static void init_vmcb(struct vcpu_svm *svm) mark_all_dirty(svm->vmcb); enable_gif(svm); + + if (avic) + avic_init_vmcb(svm); +} + +static struct svm_avic_phy_ait_entry * +avic_get_phy_ait_entry(struct kvm_vcpu *vcpu, int index) +{ + struct svm_avic_phy_ait_entry *avic_phy_ait; + struct svm_vm_data *vm_data = vcpu->kvm->arch.arch_data; + + if (!vm_data) + return NULL; + + /* Note: APIC ID = 0xff is used for broadcast. + * APIC ID > 0xff is reserved. + */ + if (index >= 0xff) + return NULL; + + avic_phy_ait = page_address(vm_data->avic_phy_ait_page); + + return &avic_phy_ait[index]; +} + +struct svm_avic_log_ait_entry * +avic_get_log_ait_entry(struct kvm_vcpu *vcpu, u8 mda, bool is_flat) +{ + struct svm_vm_data *vm_data = vcpu->kvm->arch.arch_data; + int index; + struct svm_avic_log_ait_entry *avic_log_ait; + + if (!vm_data) + return NULL; + + if (is_flat) { /* flat */ + if (mda > 7) + return NULL; + index = mda; + } else { /* cluster */ + int apic_id = mda & 0xf; + int cluster_id = (mda & 0xf0) >> 8; + + if (apic_id > 4 || cluster_id >= 0xf) + return NULL; + index = (cluster_id << 2) + apic_id; + } + avic_log_ait = (struct svm_avic_log_ait_entry *) + page_address(vm_data->avic_log_ait_page); + + return &avic_log_ait[index]; +} + +static inline void +avic_set_bk_page_entry(struct vcpu_svm *svm, int reg_off, u32 val) +{ + void *avic_bk = page_address(svm->avic_bk_page); + + *((u32 *) (avic_bk + reg_off)) = val; +} + +static inline u32 *avic_get_bk_page_entry(struct vcpu_svm *svm, u32 offset) +{ + char *tmp = (char *)page_address(svm->avic_bk_page); + + return (u32 *)(tmp+offset); +} + +static int avic_init_log_apic_entry(struct kvm_vcpu *vcpu, u8 g_phy_apic_id, + u8 log_apic_id) +{ + u32 mod; + struct svm_avic_log_ait_entry *entry; + struct vcpu_svm *svm = to_svm(vcpu); + + if (!svm) + return -EINVAL; + + mod = (*avic_get_bk_page_entry(svm, APIC_DFR) >> 28) & 0xf; + entry = avic_get_log_ait_entry(vcpu, log_apic_id, (mod == 0xf)); + if (!entry) + return -EINVAL; + entry->guest_phy_apic_id = g_phy_apic_id; + entry->valid = 1; + + return 0; +} + +static int avic_init_bk_page(struct kvm_vcpu *vcpu) +{ + u64 addr; + struct page *page; + int id = vcpu->vcpu_id; + struct kvm *kvm = vcpu->kvm; + struct vcpu_svm *svm = to_svm(vcpu); + + addr = APIC_DEFAULT_PHYS_BASE + (id * PAGE_SIZE); + page = gfn_to_page(kvm, addr >> PAGE_SHIFT); + if (is_error_page(page)) + return -EFAULT; + + /* + * Do not pin the page in memory, so that memory hot-unplug + * is able to migrate it. + */ + put_page(page); + + /* Setting up AVIC Backing Page */ + svm->avic_bk_page = page; + + clear_page(kmap(page)); + pr_debug("%s: vAPIC bk page: cpu=%u, addr=%#llx, pa=%#llx\n", + __func__, id, addr, + (unsigned long long) PFN_PHYS(page_to_pfn(page))); + + avic_init_vmcb(svm); + + return 0; +} + +static inline void avic_unalloc_bk_page(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + + if (svm->avic_bk_page) + kunmap(svm->avic_bk_page); +} + +static int avic_alloc_bk_page(struct vcpu_svm *svm, int id) +{ + int ret = 0, i; + bool realloc = false; + struct kvm_vcpu *vcpu; + struct kvm *kvm = svm->vcpu.kvm; + struct svm_vm_data *vm_data = kvm->arch.arch_data; + + mutex_lock(&kvm->slots_lock); + + /* Check if we have already allocated vAPIC backing + * page for this vCPU. If not, we need to realloc + * a new one and re-assign all other vCPU. + */ + if (kvm->arch.apic_access_page_done && + (id > vm_data->avic_max_vcpu_id)) { + kvm_for_each_vcpu(i, vcpu, kvm) + avic_unalloc_bk_page(vcpu); + + __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, + 0, 0); + realloc = true; + vm_data->avic_max_vcpu_id = 0; + } + + /* + * We are allocating vAPIC backing page + * upto the max vCPU ID + */ + if (id >= vm_data->avic_max_vcpu_id) { + ret = __x86_set_memory_region(kvm, + APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, + APIC_DEFAULT_PHYS_BASE, + PAGE_SIZE * (id + 1)); + if (ret) + goto out; + + vm_data->avic_max_vcpu_id = id; + } + + /* Reinit vAPIC backing page for exisinting vcpus */ + if (realloc) + kvm_for_each_vcpu(i, vcpu, kvm) + avic_init_bk_page(vcpu); + + avic_init_bk_page(&svm->vcpu); + + kvm->arch.apic_access_page_done = true; + +out: + mutex_unlock(&kvm->slots_lock); + return ret; +} + +static void avic_vm_uninit(struct kvm *kvm) +{ + struct svm_vm_data *vm_data = kvm->arch.arch_data; + + if (!vm_data) + return; + + if (vm_data->avic_log_ait_page) + __free_page(vm_data->avic_log_ait_page); + if (vm_data->avic_phy_ait_page) + __free_page(vm_data->avic_phy_ait_page); + kfree(vm_data); + kvm->arch.arch_data = NULL; +} + +static void avic_vcpu_uninit(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + struct svm_vm_data *vm_data = vcpu->kvm->arch.arch_data; + + if (!avic) + return; + + avic_unalloc_bk_page(vcpu); + svm->vcpu.arch.apic->regs = svm->in_kernel_lapic_regs; + svm->in_kernel_lapic_regs = NULL; + + if (vm_data && + (atomic_read(&vm_data->count) == 0 || + atomic_dec_and_test(&vm_data->count))) + avic_vm_uninit(vcpu->kvm); +} + +static atomic_t avic_tag_gen = ATOMIC_INIT(1); + +static inline u32 avic_get_next_tag(void) +{ + u32 tag = atomic_read(&avic_tag_gen); + + atomic_inc(&avic_tag_gen); + return tag; +} + +static int avic_vm_init(struct kvm *kvm) +{ + int err = -ENOMEM; + struct svm_vm_data *vm_data; + struct page *avic_phy_ait_page; + struct page *avic_log_ait_page; + + vm_data = kzalloc(sizeof(struct svm_vm_data), + GFP_KERNEL); + if (!vm_data) + return err; + + kvm->arch.arch_data = vm_data; + atomic_set(&vm_data->count, 0); + + /* Allocating physical APIC ID table (4KB) */ + avic_phy_ait_page = alloc_page(GFP_KERNEL); + if (!avic_phy_ait_page) + goto free_avic; + + vm_data->avic_phy_ait_page = avic_phy_ait_page; + clear_page(page_address(avic_phy_ait_page)); + + /* Allocating logical APIC ID table (4KB) */ + avic_log_ait_page = alloc_page(GFP_KERNEL); + if (!avic_log_ait_page) + goto free_avic; + + vm_data->avic_log_ait_page = avic_log_ait_page; + clear_page(page_address(avic_log_ait_page)); + + vm_data->avic_tag = avic_get_next_tag(); + + return 0; + +free_avic: + avic_vm_uninit(kvm); + return err; +} + +static int avic_vcpu_init(struct kvm *kvm, struct vcpu_svm *svm, int id) +{ + int err; + struct svm_vm_data *vm_data = NULL; + + /* Note: svm_vm_data is per VM */ + if (!kvm->arch.arch_data) { + err = avic_vm_init(kvm); + if (err) + return err; + } + + err = avic_alloc_bk_page(svm, id); + if (err) { + avic_vcpu_uninit(&svm->vcpu); + return err; + } + + vm_data = kvm->arch.arch_data; + atomic_inc(&vm_data->count); + + return 0; } static void svm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) @@ -1131,6 +1510,9 @@ static void svm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_cpuid(vcpu, &eax, &dummy, &dummy, &dummy); kvm_register_write(vcpu, VCPU_REGS_RDX, eax); + + if (avic && !init_event) + avic_update_vapic_bar(svm, APIC_DEFAULT_PHYS_BASE); } static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) @@ -1169,6 +1551,12 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) if (!hsave_page) goto free_page3; + if (avic) { + err = avic_vcpu_init(kvm, svm, id); + if (err) + goto free_page4; + } + svm->nested.hsave = page_address(hsave_page); svm->msrpm = page_address(msrpm_pages); @@ -1187,6 +1575,8 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) return &svm->vcpu; +free_page4: + __free_page(hsave_page); free_page3: __free_pages(nested_msrpm_pages, MSRPM_ALLOC_ORDER); free_page2: @@ -1209,6 +1599,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu) __free_pages(virt_to_page(svm->msrpm), MSRPM_ALLOC_ORDER); __free_page(virt_to_page(svm->nested.hsave)); __free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER); + avic_vcpu_uninit(vcpu); kvm_vcpu_uninit(vcpu); kmem_cache_free(kvm_vcpu_cache, svm); } @@ -3382,6 +3773,7 @@ static void dump_vmcb(struct kvm_vcpu *vcpu) pr_err("%-20s%08x\n", "exit_int_info_err:", control->exit_int_info_err); pr_err("%-20s%lld\n", "nested_ctl:", control->nested_ctl); pr_err("%-20s%016llx\n", "nested_cr3:", control->nested_cr3); + pr_err("%-20s%016llx\n", "avic_vapic_bar:", control->avic_vapic_bar); pr_err("%-20s%08x\n", "event_inj:", control->event_inj); pr_err("%-20s%08x\n", "event_inj_err:", control->event_inj_err); pr_err("%-20s%lld\n", "lbr_ctl:", control->lbr_ctl); @@ -3613,11 +4005,31 @@ static void svm_set_virtual_x2apic_mode(struct kvm_vcpu *vcpu, bool set) static bool svm_get_enable_apicv(void) { - return false; + return avic; +} + +static void svm_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) +{ } +static void svm_hwapic_isr_update(struct kvm *kvm, int isr) +{ +} + +/* Note: Currently only used by Hyper-V. */ static void svm_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) { + struct vcpu_svm *svm = to_svm(vcpu); + struct vmcb *vmcb = svm->vmcb; + + if (!avic) + return; + + if (!svm->in_kernel_lapic_regs) + return; + + svm->vcpu.arch.apic->regs = svm->in_kernel_lapic_regs; + vmcb->control.avic_enable = 0; } static void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) @@ -4387,6 +4799,8 @@ static struct kvm_x86_ops svm_x86_ops = { .refresh_apicv_exec_ctrl = svm_refresh_apicv_exec_ctrl, .load_eoi_exitmap = svm_load_eoi_exitmap, .sync_pir_to_irr = svm_sync_pir_to_irr, + .hwapic_irr_update = svm_hwapic_irr_update, + .hwapic_isr_update = svm_hwapic_isr_update, .set_tss_addr = svm_set_tss_addr, .get_tdp_level = get_npt_level, -- 1.9.1