From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-25.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, HTML_MESSAGE,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 910CFC64E7B for ; Mon, 30 Nov 2020 20:17:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E4562073C for ; Mon, 30 Nov 2020 20:17:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="R146yY5J" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E4562073C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kjpc0-000437-4n for qemu-devel@archiver.kernel.org; Mon, 30 Nov 2020 15:17:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjpai-00039o-BL for qemu-devel@nongnu.org; Mon, 30 Nov 2020 15:16:08 -0500 Received: from mail-yb1-xb43.google.com ([2607:f8b0:4864:20::b43]:32980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kjpaZ-0007zW-W7 for qemu-devel@nongnu.org; Mon, 30 Nov 2020 15:16:08 -0500 Received: by mail-yb1-xb43.google.com with SMTP id t33so12616345ybd.0 for ; Mon, 30 Nov 2020 12:15:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=owAcvAuL9GWxeS3f+NlVToii8udN2DLC3qCmfyMYQcU=; b=R146yY5J/EhBf1Mv9Xaxe32j1w4Qn7M7eK6B2h5PimkszZ76L8ODrqHjDZilYlFS5e lLjGirwebPqxQat7Gsic3tNNNbU/T5016l9WwVY6b52v756vM60lhc2yZh2OTvB2B971 e4qfB3Jd5WhJfdAwicMvdYYMLoGnqTF9WXicniHkt4yBHEEnoSrM9rkSlBJ45Iyscc0F UUsgUJhZ+sERZgC3iPvTfiBsamViA71jzWQgry77vhZT/QOPgbMqIkXsfOSNk8ktwIRP pENFnhhfwKD8yHnLJ1A35kavDaKlfShG+DIasuspfEkEJdO1V7HrKRQ6qbvpYPwFnl7q uzJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=owAcvAuL9GWxeS3f+NlVToii8udN2DLC3qCmfyMYQcU=; b=EsFzS3/7Lh3H1jZx6nMdl2eVn0fWl08ifC06XKUACAV6kkUWJn26bHBQCvjKbTO4J7 8i3IJEiaLF+UkmJ2ULWdPeAghoAUJmC6wfuK46ZKS3IYlZlgoPPgfSoxzQ/Zf4aKSD9E UcTH8OK4Y+Nf7/BCPZ/ACafjyeOCymqjsYhg5xXytAsvw23XTbxk1n1PnVmJciWiyzks 7RucvjratNFoqH2u4EZhrN4neQC2VtlDfXCOZ/gnA1ZT/RX+E+wxE66dPiF9HdLxZ9S9 w6eouzH5DGNttCJjNrlu1HGo7eId5SFXeH18yPpWsXADGYSiY8+qy/2wCbWh2rVhSmSF sdIA== X-Gm-Message-State: AOAM5322bT4B0kDDsRuvGkHGz50jUfANXpzCPGD+BbZbvQM8Aq1n3/um wntr6VkWvQITPuV/BjQ7MPE5mDmckQ5fqAzSSgoZGw== X-Google-Smtp-Source: ABdhPJw5DzTgpBQt/Urr2tmKqKoOt9+yQatPvUsnIb+86vcN8a9x0fcmzssUyaqlBDUs1u4Sht7gVtGLHFW31HySsAg= X-Received: by 2002:a25:7711:: with SMTP id s17mr30276447ybc.240.1606767357213; Mon, 30 Nov 2020 12:15:57 -0800 (PST) MIME-Version: 1.0 References: <20201126215017.41156-1-agraf@csgraf.de> <20201126215017.41156-3-agraf@csgraf.de> <20201127200054.GC56950@SPB-NB-133.local> <392c2465-157e-e15a-0a2c-2e3faa166d22@csgraf.de> In-Reply-To: From: Frank Yang Date: Mon, 30 Nov 2020 12:15:45 -0800 Message-ID: Subject: Re: [PATCH 2/8] hvf: Move common code out To: Alexander Graf Cc: Roman Bolshakov , Peter Maydell , Eduardo Habkost , Richard Henderson , qemu-devel , Cameron Esfahani , qemu-arm@nongnu.org, Claudio Fontana , Paolo Bonzini , Peter Collingbourne Content-Type: multipart/alternative; boundary="00000000000090ab2205b558ad7a" Received-SPF: pass client-ip=2607:f8b0:4864:20::b43; envelope-from=lfy@google.com; helo=mail-yb1-xb43.google.com X-Spam_score_int: -175 X-Spam_score: -17.6 X-Spam_bar: ----------------- X-Spam_report: (-17.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --00000000000090ab2205b558ad7a Content-Type: text/plain; charset="UTF-8" Update: We're not quite sure how to compare the CNTV_CVAL and CNTVCT. But the high CPU usage seems to be mitigated by having a poll interval (like KVM does) in handling WFI: https://android-review.googlesource.com/c/platform/external/qemu/+/1512501 This is loosely inspired by https://elixir.bootlin.com/linux/v5.10-rc6/source/virt/kvm/kvm_main.c#L2766 which does seem to specify a poll interval. It would be cool if we could have a lightweight way to enter sleep and restart the vcpus precisely when CVAL passes, though. Frank On Fri, Nov 27, 2020 at 3:30 PM Frank Yang wrote: > Hi all, > > +Peter Collingbourne > > I'm a developer on the Android Emulator, which is in a fork of QEMU. > > Peter and I have been working on an HVF Apple Silicon backend with an eye > toward Android guests. > > We have gotten things to basically switch to Android userspace already > (logcat/shell and graphics available at least) > > Our strategy so far has been to import logic from the KVM implementation > and hook into QEMU's software devices that previously assumed to only work > with TCG, or have KVM-specific paths. > > Thanks to Alexander for the tip on the 36-bit address space limitation > btw; our way of addressing this is to still allow highmem but not put pci > high mmio so high. > > Also, note we have a sleep/signal based mechanism to deal with WFx, which > might be worth looking into in Alexander's implementation as well: > > https://android-review.googlesource.com/c/platform/external/qemu/+/1512551 > > Patches so far, FYI: > > > https://android-review.googlesource.com/c/platform/external/qemu/+/1513429/1 > > https://android-review.googlesource.com/c/platform/external/qemu/+/1512554/3 > > https://android-review.googlesource.com/c/platform/external/qemu/+/1512553/3 > > https://android-review.googlesource.com/c/platform/external/qemu/+/1512552/3 > > https://android-review.googlesource.com/c/platform/external/qemu/+/1512551/3 > > > https://android.googlesource.com/platform/external/qemu/+/c17eb6a3ffd50047e9646aff6640b710cb8ff48a > > https://android.googlesource.com/platform/external/qemu/+/74bed16de1afb41b7a7ab8da1d1861226c9db63b > > https://android.googlesource.com/platform/external/qemu/+/eccd9e47ab2ccb9003455e3bb721f57f9ebc3c01 > > https://android.googlesource.com/platform/external/qemu/+/54fe3d67ed4698e85826537a4f49b2b9074b2228 > > https://android.googlesource.com/platform/external/qemu/+/82ef91a6fede1d1000f36be037ad4d58fbe0d102 > > https://android.googlesource.com/platform/external/qemu/+/c28147aa7c74d98b858e99623d2fe46e74a379f6 > > Peter's also noticed that there are extra steps needed for M1's to allow > TCG to work, as it involves JIT: > > > https://android.googlesource.com/platform/external/qemu/+/740e3fe47f88926c6bda9abb22ee6eae1bc254a9 > > We'd appreciate any feedback/comments :) > > Best, > > Frank > > On Fri, Nov 27, 2020 at 1:57 PM Alexander Graf wrote: > >> >> On 27.11.20 21:00, Roman Bolshakov wrote: >> > On Thu, Nov 26, 2020 at 10:50:11PM +0100, Alexander Graf wrote: >> >> Until now, Hypervisor.framework has only been available on x86_64 >> systems. >> >> With Apple Silicon shipping now, it extends its reach to aarch64. To >> >> prepare for support for multiple architectures, let's move common code >> out >> >> into its own accel directory. >> >> >> >> Signed-off-by: Alexander Graf >> >> --- >> >> MAINTAINERS | 9 +- >> >> accel/hvf/hvf-all.c | 56 +++++ >> >> accel/hvf/hvf-cpus.c | 468 >> ++++++++++++++++++++++++++++++++++++ >> >> accel/hvf/meson.build | 7 + >> >> accel/meson.build | 1 + >> >> include/sysemu/hvf_int.h | 69 ++++++ >> >> target/i386/hvf/hvf-cpus.c | 131 ---------- >> >> target/i386/hvf/hvf-cpus.h | 25 -- >> >> target/i386/hvf/hvf-i386.h | 48 +--- >> >> target/i386/hvf/hvf.c | 360 +-------------------------- >> >> target/i386/hvf/meson.build | 1 - >> >> target/i386/hvf/x86hvf.c | 11 +- >> >> target/i386/hvf/x86hvf.h | 2 - >> >> 13 files changed, 619 insertions(+), 569 deletions(-) >> >> create mode 100644 accel/hvf/hvf-all.c >> >> create mode 100644 accel/hvf/hvf-cpus.c >> >> create mode 100644 accel/hvf/meson.build >> >> create mode 100644 include/sysemu/hvf_int.h >> >> delete mode 100644 target/i386/hvf/hvf-cpus.c >> >> delete mode 100644 target/i386/hvf/hvf-cpus.h >> >> >> >> diff --git a/MAINTAINERS b/MAINTAINERS >> >> index 68bc160f41..ca4b6d9279 100644 >> >> --- a/MAINTAINERS >> >> +++ b/MAINTAINERS >> >> @@ -444,9 +444,16 @@ M: Cameron Esfahani >> >> M: Roman Bolshakov >> >> W: https://wiki.qemu.org/Features/HVF >> >> S: Maintained >> >> -F: accel/stubs/hvf-stub.c >> > There was a patch for that in the RFC series from Claudio. >> >> >> Yeah, I'm not worried about this hunk :). >> >> >> > >> >> F: target/i386/hvf/ >> >> + >> >> +HVF >> >> +M: Cameron Esfahani >> >> +M: Roman Bolshakov >> >> +W: https://wiki.qemu.org/Features/HVF >> >> +S: Maintained >> >> +F: accel/hvf/ >> >> F: include/sysemu/hvf.h >> >> +F: include/sysemu/hvf_int.h >> >> >> >> WHPX CPUs >> >> M: Sunil Muthuswamy >> >> diff --git a/accel/hvf/hvf-all.c b/accel/hvf/hvf-all.c >> >> new file mode 100644 >> >> index 0000000000..47d77a472a >> >> --- /dev/null >> >> +++ b/accel/hvf/hvf-all.c >> >> @@ -0,0 +1,56 @@ >> >> +/* >> >> + * QEMU Hypervisor.framework support >> >> + * >> >> + * This work is licensed under the terms of the GNU GPL, version 2. >> See >> >> + * the COPYING file in the top-level directory. >> >> + * >> >> + * Contributions after 2012-01-13 are licensed under the terms of the >> >> + * GNU GPL, version 2 or (at your option) any later version. >> >> + */ >> >> + >> >> +#include "qemu/osdep.h" >> >> +#include "qemu-common.h" >> >> +#include "qemu/error-report.h" >> >> +#include "sysemu/hvf.h" >> >> +#include "sysemu/hvf_int.h" >> >> +#include "sysemu/runstate.h" >> >> + >> >> +#include "qemu/main-loop.h" >> >> +#include "sysemu/accel.h" >> >> + >> >> +#include >> >> + >> >> +bool hvf_allowed; >> >> +HVFState *hvf_state; >> >> + >> >> +void assert_hvf_ok(hv_return_t ret) >> >> +{ >> >> + if (ret == HV_SUCCESS) { >> >> + return; >> >> + } >> >> + >> >> + switch (ret) { >> >> + case HV_ERROR: >> >> + error_report("Error: HV_ERROR"); >> >> + break; >> >> + case HV_BUSY: >> >> + error_report("Error: HV_BUSY"); >> >> + break; >> >> + case HV_BAD_ARGUMENT: >> >> + error_report("Error: HV_BAD_ARGUMENT"); >> >> + break; >> >> + case HV_NO_RESOURCES: >> >> + error_report("Error: HV_NO_RESOURCES"); >> >> + break; >> >> + case HV_NO_DEVICE: >> >> + error_report("Error: HV_NO_DEVICE"); >> >> + break; >> >> + case HV_UNSUPPORTED: >> >> + error_report("Error: HV_UNSUPPORTED"); >> >> + break; >> >> + default: >> >> + error_report("Unknown Error"); >> >> + } >> >> + >> >> + abort(); >> >> +} >> >> diff --git a/accel/hvf/hvf-cpus.c b/accel/hvf/hvf-cpus.c >> >> new file mode 100644 >> >> index 0000000000..f9bb5502b7 >> >> --- /dev/null >> >> +++ b/accel/hvf/hvf-cpus.c >> >> @@ -0,0 +1,468 @@ >> >> +/* >> >> + * Copyright 2008 IBM Corporation >> >> + * 2008 Red Hat, Inc. >> >> + * Copyright 2011 Intel Corporation >> >> + * Copyright 2016 Veertu, Inc. >> >> + * Copyright 2017 The Android Open Source Project >> >> + * >> >> + * QEMU Hypervisor.framework support >> >> + * >> >> + * This program is free software; you can redistribute it and/or >> >> + * modify it under the terms of version 2 of the GNU General Public >> >> + * License as published by the Free Software Foundation. >> >> + * >> >> + * This program is distributed in the hope that it will be useful, >> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >> >> + * General Public License for more details. >> >> + * >> >> + * You should have received a copy of the GNU General Public License >> >> + * along with this program; if not, see > >. >> >> + * >> >> + * This file contain code under public domain from the hvdos project: >> >> + * https://github.com/mist64/hvdos >> >> + * >> >> + * Parts Copyright (c) 2011 NetApp, Inc. >> >> + * All rights reserved. >> >> + * >> >> + * Redistribution and use in source and binary forms, with or without >> >> + * modification, are permitted provided that the following conditions >> >> + * are met: >> >> + * 1. Redistributions of source code must retain the above copyright >> >> + * notice, this list of conditions and the following disclaimer. >> >> + * 2. Redistributions in binary form must reproduce the above >> copyright >> >> + * notice, this list of conditions and the following disclaimer in >> the >> >> + * documentation and/or other materials provided with the >> distribution. >> >> + * >> >> + * THIS SOFTWARE IS PROVIDED BY NETAPP, INC ``AS IS'' AND >> >> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >> THE >> >> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >> PURPOSE >> >> + * ARE DISCLAIMED. IN NO EVENT SHALL NETAPP, INC OR CONTRIBUTORS BE >> LIABLE >> >> + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >> CONSEQUENTIAL >> >> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE >> GOODS >> >> + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >> INTERRUPTION) >> >> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >> CONTRACT, STRICT >> >> + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN >> ANY WAY >> >> + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE >> POSSIBILITY OF >> >> + * SUCH DAMAGE. >> >> + */ >> >> + >> >> +#include "qemu/osdep.h" >> >> +#include "qemu/error-report.h" >> >> +#include "qemu/main-loop.h" >> >> +#include "exec/address-spaces.h" >> >> +#include "exec/exec-all.h" >> >> +#include "sysemu/cpus.h" >> >> +#include "sysemu/hvf.h" >> >> +#include "sysemu/hvf_int.h" >> >> +#include "sysemu/runstate.h" >> >> +#include "qemu/guest-random.h" >> >> + >> >> +#include >> >> + >> >> +/* Memory slots */ >> >> + >> >> +struct mac_slot { >> >> + int present; >> >> + uint64_t size; >> >> + uint64_t gpa_start; >> >> + uint64_t gva; >> >> +}; >> >> + >> >> +hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size) >> >> +{ >> >> + hvf_slot *slot; >> >> + int x; >> >> + for (x = 0; x < hvf_state->num_slots; ++x) { >> >> + slot = &hvf_state->slots[x]; >> >> + if (slot->size && start < (slot->start + slot->size) && >> >> + (start + size) > slot->start) { >> >> + return slot; >> >> + } >> >> + } >> >> + return NULL; >> >> +} >> >> + >> >> +struct mac_slot mac_slots[32]; >> >> + >> >> +static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t flags) >> >> +{ >> >> + struct mac_slot *macslot; >> >> + hv_return_t ret; >> >> + >> >> + macslot = &mac_slots[slot->slot_id]; >> >> + >> >> + if (macslot->present) { >> >> + if (macslot->size != slot->size) { >> >> + macslot->present = 0; >> >> + ret = hv_vm_unmap(macslot->gpa_start, macslot->size); >> >> + assert_hvf_ok(ret); >> >> + } >> >> + } >> >> + >> >> + if (!slot->size) { >> >> + return 0; >> >> + } >> >> + >> >> + macslot->present = 1; >> >> + macslot->gpa_start = slot->start; >> >> + macslot->size = slot->size; >> >> + ret = hv_vm_map(slot->mem, slot->start, slot->size, flags); >> >> + assert_hvf_ok(ret); >> >> + return 0; >> >> +} >> >> + >> >> +static void hvf_set_phys_mem(MemoryRegionSection *section, bool add) >> >> +{ >> >> + hvf_slot *mem; >> >> + MemoryRegion *area = section->mr; >> >> + bool writeable = !area->readonly && !area->rom_device; >> >> + hv_memory_flags_t flags; >> >> + >> >> + if (!memory_region_is_ram(area)) { >> >> + if (writeable) { >> >> + return; >> >> + } else if (!memory_region_is_romd(area)) { >> >> + /* >> >> + * If the memory device is not in romd_mode, then we >> actually want >> >> + * to remove the hvf memory slot so all accesses will >> trap. >> >> + */ >> >> + add = false; >> >> + } >> >> + } >> >> + >> >> + mem = hvf_find_overlap_slot( >> >> + section->offset_within_address_space, >> >> + int128_get64(section->size)); >> >> + >> >> + if (mem && add) { >> >> + if (mem->size == int128_get64(section->size) && >> >> + mem->start == section->offset_within_address_space && >> >> + mem->mem == (memory_region_get_ram_ptr(area) + >> >> + section->offset_within_region)) { >> >> + return; /* Same region was attempted to register, go >> away. */ >> >> + } >> >> + } >> >> + >> >> + /* Region needs to be reset. set the size to 0 and remap it. */ >> >> + if (mem) { >> >> + mem->size = 0; >> >> + if (do_hvf_set_memory(mem, 0)) { >> >> + error_report("Failed to reset overlapping slot"); >> >> + abort(); >> >> + } >> >> + } >> >> + >> >> + if (!add) { >> >> + return; >> >> + } >> >> + >> >> + if (area->readonly || >> >> + (!memory_region_is_ram(area) && memory_region_is_romd(area))) >> { >> >> + flags = HV_MEMORY_READ | HV_MEMORY_EXEC; >> >> + } else { >> >> + flags = HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC; >> >> + } >> >> + >> >> + /* Now make a new slot. */ >> >> + int x; >> >> + >> >> + for (x = 0; x < hvf_state->num_slots; ++x) { >> >> + mem = &hvf_state->slots[x]; >> >> + if (!mem->size) { >> >> + break; >> >> + } >> >> + } >> >> + >> >> + if (x == hvf_state->num_slots) { >> >> + error_report("No free slots"); >> >> + abort(); >> >> + } >> >> + >> >> + mem->size = int128_get64(section->size); >> >> + mem->mem = memory_region_get_ram_ptr(area) + >> section->offset_within_region; >> >> + mem->start = section->offset_within_address_space; >> >> + mem->region = area; >> >> + >> >> + if (do_hvf_set_memory(mem, flags)) { >> >> + error_report("Error registering new memory slot"); >> >> + abort(); >> >> + } >> >> +} >> >> + >> >> +static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool >> on) >> >> +{ >> >> + hvf_slot *slot; >> >> + >> >> + slot = hvf_find_overlap_slot( >> >> + section->offset_within_address_space, >> >> + int128_get64(section->size)); >> >> + >> >> + /* protect region against writes; begin tracking it */ >> >> + if (on) { >> >> + slot->flags |= HVF_SLOT_LOG; >> >> + hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size, >> >> + HV_MEMORY_READ); >> >> + /* stop tracking region*/ >> >> + } else { >> >> + slot->flags &= ~HVF_SLOT_LOG; >> >> + hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size, >> >> + HV_MEMORY_READ | HV_MEMORY_WRITE); >> >> + } >> >> +} >> >> + >> >> +static void hvf_log_start(MemoryListener *listener, >> >> + MemoryRegionSection *section, int old, int >> new) >> >> +{ >> >> + if (old != 0) { >> >> + return; >> >> + } >> >> + >> >> + hvf_set_dirty_tracking(section, 1); >> >> +} >> >> + >> >> +static void hvf_log_stop(MemoryListener *listener, >> >> + MemoryRegionSection *section, int old, int >> new) >> >> +{ >> >> + if (new != 0) { >> >> + return; >> >> + } >> >> + >> >> + hvf_set_dirty_tracking(section, 0); >> >> +} >> >> + >> >> +static void hvf_log_sync(MemoryListener *listener, >> >> + MemoryRegionSection *section) >> >> +{ >> >> + /* >> >> + * sync of dirty pages is handled elsewhere; just make sure we >> keep >> >> + * tracking the region. >> >> + */ >> >> + hvf_set_dirty_tracking(section, 1); >> >> +} >> >> + >> >> +static void hvf_region_add(MemoryListener *listener, >> >> + MemoryRegionSection *section) >> >> +{ >> >> + hvf_set_phys_mem(section, true); >> >> +} >> >> + >> >> +static void hvf_region_del(MemoryListener *listener, >> >> + MemoryRegionSection *section) >> >> +{ >> >> + hvf_set_phys_mem(section, false); >> >> +} >> >> + >> >> +static MemoryListener hvf_memory_listener = { >> >> + .priority = 10, >> >> + .region_add = hvf_region_add, >> >> + .region_del = hvf_region_del, >> >> + .log_start = hvf_log_start, >> >> + .log_stop = hvf_log_stop, >> >> + .log_sync = hvf_log_sync, >> >> +}; >> >> + >> >> +static void do_hvf_cpu_synchronize_state(CPUState *cpu, >> run_on_cpu_data arg) >> >> +{ >> >> + if (!cpu->vcpu_dirty) { >> >> + hvf_get_registers(cpu); >> >> + cpu->vcpu_dirty = true; >> >> + } >> >> +} >> >> + >> >> +static void hvf_cpu_synchronize_state(CPUState *cpu) >> >> +{ >> >> + if (!cpu->vcpu_dirty) { >> >> + run_on_cpu(cpu, do_hvf_cpu_synchronize_state, >> RUN_ON_CPU_NULL); >> >> + } >> >> +} >> >> + >> >> +static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu, >> >> + run_on_cpu_data arg) >> >> +{ >> >> + hvf_put_registers(cpu); >> >> + cpu->vcpu_dirty = false; >> >> +} >> >> + >> >> +static void hvf_cpu_synchronize_post_reset(CPUState *cpu) >> >> +{ >> >> + run_on_cpu(cpu, do_hvf_cpu_synchronize_post_reset, >> RUN_ON_CPU_NULL); >> >> +} >> >> + >> >> +static void do_hvf_cpu_synchronize_post_init(CPUState *cpu, >> >> + run_on_cpu_data arg) >> >> +{ >> >> + hvf_put_registers(cpu); >> >> + cpu->vcpu_dirty = false; >> >> +} >> >> + >> >> +static void hvf_cpu_synchronize_post_init(CPUState *cpu) >> >> +{ >> >> + run_on_cpu(cpu, do_hvf_cpu_synchronize_post_init, >> RUN_ON_CPU_NULL); >> >> +} >> >> + >> >> +static void do_hvf_cpu_synchronize_pre_loadvm(CPUState *cpu, >> >> + run_on_cpu_data arg) >> >> +{ >> >> + cpu->vcpu_dirty = true; >> >> +} >> >> + >> >> +static void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu) >> >> +{ >> >> + run_on_cpu(cpu, do_hvf_cpu_synchronize_pre_loadvm, >> RUN_ON_CPU_NULL); >> >> +} >> >> + >> >> +static void hvf_vcpu_destroy(CPUState *cpu) >> >> +{ >> >> + hv_return_t ret = hv_vcpu_destroy(cpu->hvf_fd); >> >> + assert_hvf_ok(ret); >> >> + >> >> + hvf_arch_vcpu_destroy(cpu); >> >> +} >> >> + >> >> +static void dummy_signal(int sig) >> >> +{ >> >> +} >> >> + >> >> +static int hvf_init_vcpu(CPUState *cpu) >> >> +{ >> >> + int r; >> >> + >> >> + /* init cpu signals */ >> >> + sigset_t set; >> >> + struct sigaction sigact; >> >> + >> >> + memset(&sigact, 0, sizeof(sigact)); >> >> + sigact.sa_handler = dummy_signal; >> >> + sigaction(SIG_IPI, &sigact, NULL); >> >> + >> >> + pthread_sigmask(SIG_BLOCK, NULL, &set); >> >> + sigdelset(&set, SIG_IPI); >> >> + >> >> +#ifdef __aarch64__ >> >> + r = hv_vcpu_create(&cpu->hvf_fd, (hv_vcpu_exit_t >> **)&cpu->hvf_exit, NULL); >> >> +#else >> >> + r = hv_vcpu_create((hv_vcpuid_t *)&cpu->hvf_fd, HV_VCPU_DEFAULT); >> >> +#endif >> > I think the first __aarch64__ bit fits better to arm part of the series. >> >> >> Oops. Thanks for catching it! Yes, absolutely. It should be part of the >> ARM enablement. >> >> >> > >> >> + cpu->vcpu_dirty = 1; >> >> + assert_hvf_ok(r); >> >> + >> >> + return hvf_arch_init_vcpu(cpu); >> >> +} >> >> + >> >> +/* >> >> + * The HVF-specific vCPU thread function. This one should only run >> when the host >> >> + * CPU supports the VMX "unrestricted guest" feature. >> >> + */ >> >> +static void *hvf_cpu_thread_fn(void *arg) >> >> +{ >> >> + CPUState *cpu = arg; >> >> + >> >> + int r; >> >> + >> >> + assert(hvf_enabled()); >> >> + >> >> + rcu_register_thread(); >> >> + >> >> + qemu_mutex_lock_iothread(); >> >> + qemu_thread_get_self(cpu->thread); >> >> + >> >> + cpu->thread_id = qemu_get_thread_id(); >> >> + cpu->can_do_io = 1; >> >> + current_cpu = cpu; >> >> + >> >> + hvf_init_vcpu(cpu); >> >> + >> >> + /* signal CPU creation */ >> >> + cpu_thread_signal_created(cpu); >> >> + qemu_guest_random_seed_thread_part2(cpu->random_seed); >> >> + >> >> + do { >> >> + if (cpu_can_run(cpu)) { >> >> + r = hvf_vcpu_exec(cpu); >> >> + if (r == EXCP_DEBUG) { >> >> + cpu_handle_guest_debug(cpu); >> >> + } >> >> + } >> >> + qemu_wait_io_event(cpu); >> >> + } while (!cpu->unplug || cpu_can_run(cpu)); >> >> + >> >> + hvf_vcpu_destroy(cpu); >> >> + cpu_thread_signal_destroyed(cpu); >> >> + qemu_mutex_unlock_iothread(); >> >> + rcu_unregister_thread(); >> >> + return NULL; >> >> +} >> >> + >> >> +static void hvf_start_vcpu_thread(CPUState *cpu) >> >> +{ >> >> + char thread_name[VCPU_THREAD_NAME_SIZE]; >> >> + >> >> + /* >> >> + * HVF currently does not support TCG, and only runs in >> >> + * unrestricted-guest mode. >> >> + */ >> >> + assert(hvf_enabled()); >> >> + >> >> + cpu->thread = g_malloc0(sizeof(QemuThread)); >> >> + cpu->halt_cond = g_malloc0(sizeof(QemuCond)); >> >> + qemu_cond_init(cpu->halt_cond); >> >> + >> >> + snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/HVF", >> >> + cpu->cpu_index); >> >> + qemu_thread_create(cpu->thread, thread_name, hvf_cpu_thread_fn, >> >> + cpu, QEMU_THREAD_JOINABLE); >> >> +} >> >> + >> >> +static const CpusAccel hvf_cpus = { >> >> + .create_vcpu_thread = hvf_start_vcpu_thread, >> >> + >> >> + .synchronize_post_reset = hvf_cpu_synchronize_post_reset, >> >> + .synchronize_post_init = hvf_cpu_synchronize_post_init, >> >> + .synchronize_state = hvf_cpu_synchronize_state, >> >> + .synchronize_pre_loadvm = hvf_cpu_synchronize_pre_loadvm, >> >> +}; >> >> + >> >> +static int hvf_accel_init(MachineState *ms) >> >> +{ >> >> + int x; >> >> + hv_return_t ret; >> >> + HVFState *s; >> >> + >> >> + ret = hv_vm_create(HV_VM_DEFAULT); >> >> + assert_hvf_ok(ret); >> >> + >> >> + s = g_new0(HVFState, 1); >> >> + >> >> + s->num_slots = 32; >> >> + for (x = 0; x < s->num_slots; ++x) { >> >> + s->slots[x].size = 0; >> >> + s->slots[x].slot_id = x; >> >> + } >> >> + >> >> + hvf_state = s; >> >> + memory_listener_register(&hvf_memory_listener, >> &address_space_memory); >> >> + cpus_register_accel(&hvf_cpus); >> >> + return 0; >> >> +} >> >> + >> >> +static void hvf_accel_class_init(ObjectClass *oc, void *data) >> >> +{ >> >> + AccelClass *ac = ACCEL_CLASS(oc); >> >> + ac->name = "HVF"; >> >> + ac->init_machine = hvf_accel_init; >> >> + ac->allowed = &hvf_allowed; >> >> +} >> >> + >> >> +static const TypeInfo hvf_accel_type = { >> >> + .name = TYPE_HVF_ACCEL, >> >> + .parent = TYPE_ACCEL, >> >> + .class_init = hvf_accel_class_init, >> >> +}; >> >> + >> >> +static void hvf_type_init(void) >> >> +{ >> >> + type_register_static(&hvf_accel_type); >> >> +} >> >> + >> >> +type_init(hvf_type_init); >> >> diff --git a/accel/hvf/meson.build b/accel/hvf/meson.build >> >> new file mode 100644 >> >> index 0000000000..dfd6b68dc7 >> >> --- /dev/null >> >> +++ b/accel/hvf/meson.build >> >> @@ -0,0 +1,7 @@ >> >> +hvf_ss = ss.source_set() >> >> +hvf_ss.add(files( >> >> + 'hvf-all.c', >> >> + 'hvf-cpus.c', >> >> +)) >> >> + >> >> +specific_ss.add_all(when: 'CONFIG_HVF', if_true: hvf_ss) >> >> diff --git a/accel/meson.build b/accel/meson.build >> >> index b26cca227a..6de12ce5d5 100644 >> >> --- a/accel/meson.build >> >> +++ b/accel/meson.build >> >> @@ -1,5 +1,6 @@ >> >> softmmu_ss.add(files('accel.c')) >> >> >> >> +subdir('hvf') >> >> subdir('qtest') >> >> subdir('kvm') >> >> subdir('tcg') >> >> diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h >> >> new file mode 100644 >> >> index 0000000000..de9bad23a8 >> >> --- /dev/null >> >> +++ b/include/sysemu/hvf_int.h >> >> @@ -0,0 +1,69 @@ >> >> +/* >> >> + * QEMU Hypervisor.framework (HVF) support >> >> + * >> >> + * This work is licensed under the terms of the GNU GPL, version 2 or >> later. >> >> + * See the COPYING file in the top-level directory. >> >> + * >> >> + */ >> >> + >> >> +/* header to be included in HVF-specific code */ >> >> + >> >> +#ifndef HVF_INT_H >> >> +#define HVF_INT_H >> >> + >> >> +#include >> >> + >> >> +#define HVF_MAX_VCPU 0x10 >> >> + >> >> +extern struct hvf_state hvf_global; >> >> + >> >> +struct hvf_vm { >> >> + int id; >> >> + struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU]; >> >> +}; >> >> + >> >> +struct hvf_state { >> >> + uint32_t version; >> >> + struct hvf_vm *vm; >> >> + uint64_t mem_quota; >> >> +}; >> >> + >> >> +/* hvf_slot flags */ >> >> +#define HVF_SLOT_LOG (1 << 0) >> >> + >> >> +typedef struct hvf_slot { >> >> + uint64_t start; >> >> + uint64_t size; >> >> + uint8_t *mem; >> >> + int slot_id; >> >> + uint32_t flags; >> >> + MemoryRegion *region; >> >> +} hvf_slot; >> >> + >> >> +typedef struct hvf_vcpu_caps { >> >> + uint64_t vmx_cap_pinbased; >> >> + uint64_t vmx_cap_procbased; >> >> + uint64_t vmx_cap_procbased2; >> >> + uint64_t vmx_cap_entry; >> >> + uint64_t vmx_cap_exit; >> >> + uint64_t vmx_cap_preemption_timer; >> >> +} hvf_vcpu_caps; >> >> + >> >> +struct HVFState { >> >> + AccelState parent; >> >> + hvf_slot slots[32]; >> >> + int num_slots; >> >> + >> >> + hvf_vcpu_caps *hvf_caps; >> >> +}; >> >> +extern HVFState *hvf_state; >> >> + >> >> +void assert_hvf_ok(hv_return_t ret); >> >> +int hvf_get_registers(CPUState *cpu); >> >> +int hvf_put_registers(CPUState *cpu); >> >> +int hvf_arch_init_vcpu(CPUState *cpu); >> >> +void hvf_arch_vcpu_destroy(CPUState *cpu); >> >> +int hvf_vcpu_exec(CPUState *cpu); >> >> +hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t); >> >> + >> >> +#endif >> >> diff --git a/target/i386/hvf/hvf-cpus.c b/target/i386/hvf/hvf-cpus.c >> >> deleted file mode 100644 >> >> index 817b3d7452..0000000000 >> >> --- a/target/i386/hvf/hvf-cpus.c >> >> +++ /dev/null >> >> @@ -1,131 +0,0 @@ >> >> -/* >> >> - * Copyright 2008 IBM Corporation >> >> - * 2008 Red Hat, Inc. >> >> - * Copyright 2011 Intel Corporation >> >> - * Copyright 2016 Veertu, Inc. >> >> - * Copyright 2017 The Android Open Source Project >> >> - * >> >> - * QEMU Hypervisor.framework support >> >> - * >> >> - * This program is free software; you can redistribute it and/or >> >> - * modify it under the terms of version 2 of the GNU General Public >> >> - * License as published by the Free Software Foundation. >> >> - * >> >> - * This program is distributed in the hope that it will be useful, >> >> - * but WITHOUT ANY WARRANTY; without even the implied warranty of >> >> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >> >> - * General Public License for more details. >> >> - * >> >> - * You should have received a copy of the GNU General Public License >> >> - * along with this program; if not, see > >. >> >> - * >> >> - * This file contain code under public domain from the hvdos project: >> >> - * https://github.com/mist64/hvdos >> >> - * >> >> - * Parts Copyright (c) 2011 NetApp, Inc. >> >> - * All rights reserved. >> >> - * >> >> - * Redistribution and use in source and binary forms, with or without >> >> - * modification, are permitted provided that the following conditions >> >> - * are met: >> >> - * 1. Redistributions of source code must retain the above copyright >> >> - * notice, this list of conditions and the following disclaimer. >> >> - * 2. Redistributions in binary form must reproduce the above >> copyright >> >> - * notice, this list of conditions and the following disclaimer in >> the >> >> - * documentation and/or other materials provided with the >> distribution. >> >> - * >> >> - * THIS SOFTWARE IS PROVIDED BY NETAPP, INC ``AS IS'' AND >> >> - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >> THE >> >> - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >> PURPOSE >> >> - * ARE DISCLAIMED. IN NO EVENT SHALL NETAPP, INC OR CONTRIBUTORS BE >> LIABLE >> >> - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >> CONSEQUENTIAL >> >> - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE >> GOODS >> >> - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >> INTERRUPTION) >> >> - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >> CONTRACT, STRICT >> >> - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN >> ANY WAY >> >> - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE >> POSSIBILITY OF >> >> - * SUCH DAMAGE. >> >> - */ >> >> - >> >> -#include "qemu/osdep.h" >> >> -#include "qemu/error-report.h" >> >> -#include "qemu/main-loop.h" >> >> -#include "sysemu/hvf.h" >> >> -#include "sysemu/runstate.h" >> >> -#include "target/i386/cpu.h" >> >> -#include "qemu/guest-random.h" >> >> - >> >> -#include "hvf-cpus.h" >> >> - >> >> -/* >> >> - * The HVF-specific vCPU thread function. This one should only run >> when the host >> >> - * CPU supports the VMX "unrestricted guest" feature. >> >> - */ >> >> -static void *hvf_cpu_thread_fn(void *arg) >> >> -{ >> >> - CPUState *cpu = arg; >> >> - >> >> - int r; >> >> - >> >> - assert(hvf_enabled()); >> >> - >> >> - rcu_register_thread(); >> >> - >> >> - qemu_mutex_lock_iothread(); >> >> - qemu_thread_get_self(cpu->thread); >> >> - >> >> - cpu->thread_id = qemu_get_thread_id(); >> >> - cpu->can_do_io = 1; >> >> - current_cpu = cpu; >> >> - >> >> - hvf_init_vcpu(cpu); >> >> - >> >> - /* signal CPU creation */ >> >> - cpu_thread_signal_created(cpu); >> >> - qemu_guest_random_seed_thread_part2(cpu->random_seed); >> >> - >> >> - do { >> >> - if (cpu_can_run(cpu)) { >> >> - r = hvf_vcpu_exec(cpu); >> >> - if (r == EXCP_DEBUG) { >> >> - cpu_handle_guest_debug(cpu); >> >> - } >> >> - } >> >> - qemu_wait_io_event(cpu); >> >> - } while (!cpu->unplug || cpu_can_run(cpu)); >> >> - >> >> - hvf_vcpu_destroy(cpu); >> >> - cpu_thread_signal_destroyed(cpu); >> >> - qemu_mutex_unlock_iothread(); >> >> - rcu_unregister_thread(); >> >> - return NULL; >> >> -} >> >> - >> >> -static void hvf_start_vcpu_thread(CPUState *cpu) >> >> -{ >> >> - char thread_name[VCPU_THREAD_NAME_SIZE]; >> >> - >> >> - /* >> >> - * HVF currently does not support TCG, and only runs in >> >> - * unrestricted-guest mode. >> >> - */ >> >> - assert(hvf_enabled()); >> >> - >> >> - cpu->thread = g_malloc0(sizeof(QemuThread)); >> >> - cpu->halt_cond = g_malloc0(sizeof(QemuCond)); >> >> - qemu_cond_init(cpu->halt_cond); >> >> - >> >> - snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/HVF", >> >> - cpu->cpu_index); >> >> - qemu_thread_create(cpu->thread, thread_name, hvf_cpu_thread_fn, >> >> - cpu, QEMU_THREAD_JOINABLE); >> >> -} >> >> - >> >> -const CpusAccel hvf_cpus = { >> >> - .create_vcpu_thread = hvf_start_vcpu_thread, >> >> - >> >> - .synchronize_post_reset = hvf_cpu_synchronize_post_reset, >> >> - .synchronize_post_init = hvf_cpu_synchronize_post_init, >> >> - .synchronize_state = hvf_cpu_synchronize_state, >> >> - .synchronize_pre_loadvm = hvf_cpu_synchronize_pre_loadvm, >> >> -}; >> >> diff --git a/target/i386/hvf/hvf-cpus.h b/target/i386/hvf/hvf-cpus.h >> >> deleted file mode 100644 >> >> index ced31b82c0..0000000000 >> >> --- a/target/i386/hvf/hvf-cpus.h >> >> +++ /dev/null >> >> @@ -1,25 +0,0 @@ >> >> -/* >> >> - * Accelerator CPUS Interface >> >> - * >> >> - * Copyright 2020 SUSE LLC >> >> - * >> >> - * This work is licensed under the terms of the GNU GPL, version 2 or >> later. >> >> - * See the COPYING file in the top-level directory. >> >> - */ >> >> - >> >> -#ifndef HVF_CPUS_H >> >> -#define HVF_CPUS_H >> >> - >> >> -#include "sysemu/cpus.h" >> >> - >> >> -extern const CpusAccel hvf_cpus; >> >> - >> >> -int hvf_init_vcpu(CPUState *); >> >> -int hvf_vcpu_exec(CPUState *); >> >> -void hvf_cpu_synchronize_state(CPUState *); >> >> -void hvf_cpu_synchronize_post_reset(CPUState *); >> >> -void hvf_cpu_synchronize_post_init(CPUState *); >> >> -void hvf_cpu_synchronize_pre_loadvm(CPUState *); >> >> -void hvf_vcpu_destroy(CPUState *); >> >> - >> >> -#endif /* HVF_CPUS_H */ >> >> diff --git a/target/i386/hvf/hvf-i386.h b/target/i386/hvf/hvf-i386.h >> >> index e0edffd077..6d56f8f6bb 100644 >> >> --- a/target/i386/hvf/hvf-i386.h >> >> +++ b/target/i386/hvf/hvf-i386.h >> >> @@ -18,57 +18,11 @@ >> >> >> >> #include "sysemu/accel.h" >> >> #include "sysemu/hvf.h" >> >> +#include "sysemu/hvf_int.h" >> >> #include "cpu.h" >> >> #include "x86.h" >> >> >> >> -#define HVF_MAX_VCPU 0x10 >> >> - >> >> -extern struct hvf_state hvf_global; >> >> - >> >> -struct hvf_vm { >> >> - int id; >> >> - struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU]; >> >> -}; >> >> - >> >> -struct hvf_state { >> >> - uint32_t version; >> >> - struct hvf_vm *vm; >> >> - uint64_t mem_quota; >> >> -}; >> >> - >> >> -/* hvf_slot flags */ >> >> -#define HVF_SLOT_LOG (1 << 0) >> >> - >> >> -typedef struct hvf_slot { >> >> - uint64_t start; >> >> - uint64_t size; >> >> - uint8_t *mem; >> >> - int slot_id; >> >> - uint32_t flags; >> >> - MemoryRegion *region; >> >> -} hvf_slot; >> >> - >> >> -typedef struct hvf_vcpu_caps { >> >> - uint64_t vmx_cap_pinbased; >> >> - uint64_t vmx_cap_procbased; >> >> - uint64_t vmx_cap_procbased2; >> >> - uint64_t vmx_cap_entry; >> >> - uint64_t vmx_cap_exit; >> >> - uint64_t vmx_cap_preemption_timer; >> >> -} hvf_vcpu_caps; >> >> - >> >> -struct HVFState { >> >> - AccelState parent; >> >> - hvf_slot slots[32]; >> >> - int num_slots; >> >> - >> >> - hvf_vcpu_caps *hvf_caps; >> >> -}; >> >> -extern HVFState *hvf_state; >> >> - >> >> -void hvf_set_phys_mem(MemoryRegionSection *, bool); >> >> void hvf_handle_io(CPUArchState *, uint16_t, void *, int, int, int); >> >> -hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t); >> >> >> >> #ifdef NEED_CPU_H >> >> /* Functions exported to host specific mode */ >> >> diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c >> >> index ed9356565c..8b96ecd619 100644 >> >> --- a/target/i386/hvf/hvf.c >> >> +++ b/target/i386/hvf/hvf.c >> >> @@ -51,6 +51,7 @@ >> >> #include "qemu/error-report.h" >> >> >> >> #include "sysemu/hvf.h" >> >> +#include "sysemu/hvf_int.h" >> >> #include "sysemu/runstate.h" >> >> #include "hvf-i386.h" >> >> #include "vmcs.h" >> >> @@ -72,171 +73,6 @@ >> >> #include "sysemu/accel.h" >> >> #include "target/i386/cpu.h" >> >> >> >> -#include "hvf-cpus.h" >> >> - >> >> -HVFState *hvf_state; >> >> - >> >> -static void assert_hvf_ok(hv_return_t ret) >> >> -{ >> >> - if (ret == HV_SUCCESS) { >> >> - return; >> >> - } >> >> - >> >> - switch (ret) { >> >> - case HV_ERROR: >> >> - error_report("Error: HV_ERROR"); >> >> - break; >> >> - case HV_BUSY: >> >> - error_report("Error: HV_BUSY"); >> >> - break; >> >> - case HV_BAD_ARGUMENT: >> >> - error_report("Error: HV_BAD_ARGUMENT"); >> >> - break; >> >> - case HV_NO_RESOURCES: >> >> - error_report("Error: HV_NO_RESOURCES"); >> >> - break; >> >> - case HV_NO_DEVICE: >> >> - error_report("Error: HV_NO_DEVICE"); >> >> - break; >> >> - case HV_UNSUPPORTED: >> >> - error_report("Error: HV_UNSUPPORTED"); >> >> - break; >> >> - default: >> >> - error_report("Unknown Error"); >> >> - } >> >> - >> >> - abort(); >> >> -} >> >> - >> >> -/* Memory slots */ >> >> -hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size) >> >> -{ >> >> - hvf_slot *slot; >> >> - int x; >> >> - for (x = 0; x < hvf_state->num_slots; ++x) { >> >> - slot = &hvf_state->slots[x]; >> >> - if (slot->size && start < (slot->start + slot->size) && >> >> - (start + size) > slot->start) { >> >> - return slot; >> >> - } >> >> - } >> >> - return NULL; >> >> -} >> >> - >> >> -struct mac_slot { >> >> - int present; >> >> - uint64_t size; >> >> - uint64_t gpa_start; >> >> - uint64_t gva; >> >> -}; >> >> - >> >> -struct mac_slot mac_slots[32]; >> >> - >> >> -static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t flags) >> >> -{ >> >> - struct mac_slot *macslot; >> >> - hv_return_t ret; >> >> - >> >> - macslot = &mac_slots[slot->slot_id]; >> >> - >> >> - if (macslot->present) { >> >> - if (macslot->size != slot->size) { >> >> - macslot->present = 0; >> >> - ret = hv_vm_unmap(macslot->gpa_start, macslot->size); >> >> - assert_hvf_ok(ret); >> >> - } >> >> - } >> >> - >> >> - if (!slot->size) { >> >> - return 0; >> >> - } >> >> - >> >> - macslot->present = 1; >> >> - macslot->gpa_start = slot->start; >> >> - macslot->size = slot->size; >> >> - ret = hv_vm_map((hv_uvaddr_t)slot->mem, slot->start, slot->size, >> flags); >> >> - assert_hvf_ok(ret); >> >> - return 0; >> >> -} >> >> - >> >> -void hvf_set_phys_mem(MemoryRegionSection *section, bool add) >> >> -{ >> >> - hvf_slot *mem; >> >> - MemoryRegion *area = section->mr; >> >> - bool writeable = !area->readonly && !area->rom_device; >> >> - hv_memory_flags_t flags; >> >> - >> >> - if (!memory_region_is_ram(area)) { >> >> - if (writeable) { >> >> - return; >> >> - } else if (!memory_region_is_romd(area)) { >> >> - /* >> >> - * If the memory device is not in romd_mode, then we >> actually want >> >> - * to remove the hvf memory slot so all accesses will >> trap. >> >> - */ >> >> - add = false; >> >> - } >> >> - } >> >> - >> >> - mem = hvf_find_overlap_slot( >> >> - section->offset_within_address_space, >> >> - int128_get64(section->size)); >> >> - >> >> - if (mem && add) { >> >> - if (mem->size == int128_get64(section->size) && >> >> - mem->start == section->offset_within_address_space && >> >> - mem->mem == (memory_region_get_ram_ptr(area) + >> >> - section->offset_within_region)) { >> >> - return; /* Same region was attempted to register, go >> away. */ >> >> - } >> >> - } >> >> - >> >> - /* Region needs to be reset. set the size to 0 and remap it. */ >> >> - if (mem) { >> >> - mem->size = 0; >> >> - if (do_hvf_set_memory(mem, 0)) { >> >> - error_report("Failed to reset overlapping slot"); >> >> - abort(); >> >> - } >> >> - } >> >> - >> >> - if (!add) { >> >> - return; >> >> - } >> >> - >> >> - if (area->readonly || >> >> - (!memory_region_is_ram(area) && memory_region_is_romd(area))) >> { >> >> - flags = HV_MEMORY_READ | HV_MEMORY_EXEC; >> >> - } else { >> >> - flags = HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC; >> >> - } >> >> - >> >> - /* Now make a new slot. */ >> >> - int x; >> >> - >> >> - for (x = 0; x < hvf_state->num_slots; ++x) { >> >> - mem = &hvf_state->slots[x]; >> >> - if (!mem->size) { >> >> - break; >> >> - } >> >> - } >> >> - >> >> - if (x == hvf_state->num_slots) { >> >> - error_report("No free slots"); >> >> - abort(); >> >> - } >> >> - >> >> - mem->size = int128_get64(section->size); >> >> - mem->mem = memory_region_get_ram_ptr(area) + >> section->offset_within_region; >> >> - mem->start = section->offset_within_address_space; >> >> - mem->region = area; >> >> - >> >> - if (do_hvf_set_memory(mem, flags)) { >> >> - error_report("Error registering new memory slot"); >> >> - abort(); >> >> - } >> >> -} >> >> - >> >> void vmx_update_tpr(CPUState *cpu) >> >> { >> >> /* TODO: need integrate APIC handling */ >> >> @@ -276,56 +112,6 @@ void hvf_handle_io(CPUArchState *env, uint16_t >> port, void *buffer, >> >> } >> >> } >> >> >> >> -static void do_hvf_cpu_synchronize_state(CPUState *cpu, >> run_on_cpu_data arg) >> >> -{ >> >> - if (!cpu->vcpu_dirty) { >> >> - hvf_get_registers(cpu); >> >> - cpu->vcpu_dirty = true; >> >> - } >> >> -} >> >> - >> >> -void hvf_cpu_synchronize_state(CPUState *cpu) >> >> -{ >> >> - if (!cpu->vcpu_dirty) { >> >> - run_on_cpu(cpu, do_hvf_cpu_synchronize_state, >> RUN_ON_CPU_NULL); >> >> - } >> >> -} >> >> - >> >> -static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu, >> >> - run_on_cpu_data arg) >> >> -{ >> >> - hvf_put_registers(cpu); >> >> - cpu->vcpu_dirty = false; >> >> -} >> >> - >> >> -void hvf_cpu_synchronize_post_reset(CPUState *cpu) >> >> -{ >> >> - run_on_cpu(cpu, do_hvf_cpu_synchronize_post_reset, >> RUN_ON_CPU_NULL); >> >> -} >> >> - >> >> -static void do_hvf_cpu_synchronize_post_init(CPUState *cpu, >> >> - run_on_cpu_data arg) >> >> -{ >> >> - hvf_put_registers(cpu); >> >> - cpu->vcpu_dirty = false; >> >> -} >> >> - >> >> -void hvf_cpu_synchronize_post_init(CPUState *cpu) >> >> -{ >> >> - run_on_cpu(cpu, do_hvf_cpu_synchronize_post_init, >> RUN_ON_CPU_NULL); >> >> -} >> >> - >> >> -static void do_hvf_cpu_synchronize_pre_loadvm(CPUState *cpu, >> >> - run_on_cpu_data arg) >> >> -{ >> >> - cpu->vcpu_dirty = true; >> >> -} >> >> - >> >> -void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu) >> >> -{ >> >> - run_on_cpu(cpu, do_hvf_cpu_synchronize_pre_loadvm, >> RUN_ON_CPU_NULL); >> >> -} >> >> - >> >> static bool ept_emulation_fault(hvf_slot *slot, uint64_t gpa, >> uint64_t ept_qual) >> >> { >> >> int read, write; >> >> @@ -370,109 +156,19 @@ static bool ept_emulation_fault(hvf_slot *slot, >> uint64_t gpa, uint64_t ept_qual) >> >> return false; >> >> } >> >> >> >> -static void hvf_set_dirty_tracking(MemoryRegionSection *section, bool >> on) >> >> -{ >> >> - hvf_slot *slot; >> >> - >> >> - slot = hvf_find_overlap_slot( >> >> - section->offset_within_address_space, >> >> - int128_get64(section->size)); >> >> - >> >> - /* protect region against writes; begin tracking it */ >> >> - if (on) { >> >> - slot->flags |= HVF_SLOT_LOG; >> >> - hv_vm_protect((hv_gpaddr_t)slot->start, (size_t)slot->size, >> >> - HV_MEMORY_READ); >> >> - /* stop tracking region*/ >> >> - } else { >> >> - slot->flags &= ~HVF_SLOT_LOG; >> >> - hv_vm_protect((hv_gpaddr_t)slot->start, (size_t)slot->size, >> >> - HV_MEMORY_READ | HV_MEMORY_WRITE); >> >> - } >> >> -} >> >> - >> >> -static void hvf_log_start(MemoryListener *listener, >> >> - MemoryRegionSection *section, int old, int >> new) >> >> -{ >> >> - if (old != 0) { >> >> - return; >> >> - } >> >> - >> >> - hvf_set_dirty_tracking(section, 1); >> >> -} >> >> - >> >> -static void hvf_log_stop(MemoryListener *listener, >> >> - MemoryRegionSection *section, int old, int >> new) >> >> -{ >> >> - if (new != 0) { >> >> - return; >> >> - } >> >> - >> >> - hvf_set_dirty_tracking(section, 0); >> >> -} >> >> - >> >> -static void hvf_log_sync(MemoryListener *listener, >> >> - MemoryRegionSection *section) >> >> -{ >> >> - /* >> >> - * sync of dirty pages is handled elsewhere; just make sure we >> keep >> >> - * tracking the region. >> >> - */ >> >> - hvf_set_dirty_tracking(section, 1); >> >> -} >> >> - >> >> -static void hvf_region_add(MemoryListener *listener, >> >> - MemoryRegionSection *section) >> >> -{ >> >> - hvf_set_phys_mem(section, true); >> >> -} >> >> - >> >> -static void hvf_region_del(MemoryListener *listener, >> >> - MemoryRegionSection *section) >> >> -{ >> >> - hvf_set_phys_mem(section, false); >> >> -} >> >> - >> >> -static MemoryListener hvf_memory_listener = { >> >> - .priority = 10, >> >> - .region_add = hvf_region_add, >> >> - .region_del = hvf_region_del, >> >> - .log_start = hvf_log_start, >> >> - .log_stop = hvf_log_stop, >> >> - .log_sync = hvf_log_sync, >> >> -}; >> >> - >> >> -void hvf_vcpu_destroy(CPUState *cpu) >> >> +void hvf_arch_vcpu_destroy(CPUState *cpu) >> >> { >> >> X86CPU *x86_cpu = X86_CPU(cpu); >> >> CPUX86State *env = &x86_cpu->env; >> >> >> >> - hv_return_t ret = hv_vcpu_destroy((hv_vcpuid_t)cpu->hvf_fd); >> >> g_free(env->hvf_mmio_buf); >> >> - assert_hvf_ok(ret); >> >> -} >> >> - >> >> -static void dummy_signal(int sig) >> >> -{ >> >> } >> >> >> >> -int hvf_init_vcpu(CPUState *cpu) >> >> +int hvf_arch_init_vcpu(CPUState *cpu) >> >> { >> >> >> >> X86CPU *x86cpu = X86_CPU(cpu); >> >> CPUX86State *env = &x86cpu->env; >> >> - int r; >> >> - >> >> - /* init cpu signals */ >> >> - sigset_t set; >> >> - struct sigaction sigact; >> >> - >> >> - memset(&sigact, 0, sizeof(sigact)); >> >> - sigact.sa_handler = dummy_signal; >> >> - sigaction(SIG_IPI, &sigact, NULL); >> >> - >> >> - pthread_sigmask(SIG_BLOCK, NULL, &set); >> >> - sigdelset(&set, SIG_IPI); >> >> >> >> init_emu(); >> >> init_decoder(); >> >> @@ -480,10 +176,6 @@ int hvf_init_vcpu(CPUState *cpu) >> >> hvf_state->hvf_caps = g_new0(struct hvf_vcpu_caps, 1); >> >> env->hvf_mmio_buf = g_new(char, 4096); >> >> >> >> - r = hv_vcpu_create((hv_vcpuid_t *)&cpu->hvf_fd, HV_VCPU_DEFAULT); >> >> - cpu->vcpu_dirty = 1; >> >> - assert_hvf_ok(r); >> >> - >> >> if (hv_vmx_read_capability(HV_VMX_CAP_PINBASED, >> >> &hvf_state->hvf_caps->vmx_cap_pinbased)) { >> >> abort(); >> >> @@ -865,49 +557,3 @@ int hvf_vcpu_exec(CPUState *cpu) >> >> >> >> return ret; >> >> } >> >> - >> >> -bool hvf_allowed; >> >> - >> >> -static int hvf_accel_init(MachineState *ms) >> >> -{ >> >> - int x; >> >> - hv_return_t ret; >> >> - HVFState *s; >> >> - >> >> - ret = hv_vm_create(HV_VM_DEFAULT); >> >> - assert_hvf_ok(ret); >> >> - >> >> - s = g_new0(HVFState, 1); >> >> - >> >> - s->num_slots = 32; >> >> - for (x = 0; x < s->num_slots; ++x) { >> >> - s->slots[x].size = 0; >> >> - s->slots[x].slot_id = x; >> >> - } >> >> - >> >> - hvf_state = s; >> >> - memory_listener_register(&hvf_memory_listener, >> &address_space_memory); >> >> - cpus_register_accel(&hvf_cpus); >> >> - return 0; >> >> -} >> >> - >> >> -static void hvf_accel_class_init(ObjectClass *oc, void *data) >> >> -{ >> >> - AccelClass *ac = ACCEL_CLASS(oc); >> >> - ac->name = "HVF"; >> >> - ac->init_machine = hvf_accel_init; >> >> - ac->allowed = &hvf_allowed; >> >> -} >> >> - >> >> -static const TypeInfo hvf_accel_type = { >> >> - .name = TYPE_HVF_ACCEL, >> >> - .parent = TYPE_ACCEL, >> >> - .class_init = hvf_accel_class_init, >> >> -}; >> >> - >> >> -static void hvf_type_init(void) >> >> -{ >> >> - type_register_static(&hvf_accel_type); >> >> -} >> >> - >> >> -type_init(hvf_type_init); >> >> diff --git a/target/i386/hvf/meson.build b/target/i386/hvf/meson.build >> >> index 409c9a3f14..c8a43717ee 100644 >> >> --- a/target/i386/hvf/meson.build >> >> +++ b/target/i386/hvf/meson.build >> >> @@ -1,6 +1,5 @@ >> >> i386_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files( >> >> 'hvf.c', >> >> - 'hvf-cpus.c', >> >> 'x86.c', >> >> 'x86_cpuid.c', >> >> 'x86_decode.c', >> >> diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c >> >> index bbec412b6c..89b8e9d87a 100644 >> >> --- a/target/i386/hvf/x86hvf.c >> >> +++ b/target/i386/hvf/x86hvf.c >> >> @@ -20,6 +20,9 @@ >> >> #include "qemu/osdep.h" >> >> >> >> #include "qemu-common.h" >> >> +#include "sysemu/hvf.h" >> >> +#include "sysemu/hvf_int.h" >> >> +#include "sysemu/hw_accel.h" >> >> #include "x86hvf.h" >> >> #include "vmx.h" >> >> #include "vmcs.h" >> >> @@ -32,8 +35,6 @@ >> >> #include >> >> #include >> >> >> >> -#include "hvf-cpus.h" >> >> - >> >> void hvf_set_segment(struct CPUState *cpu, struct vmx_segment >> *vmx_seg, >> >> SegmentCache *qseg, bool is_tr) >> >> { >> >> @@ -437,7 +438,7 @@ int hvf_process_events(CPUState *cpu_state) >> >> env->eflags = rreg(cpu_state->hvf_fd, HV_X86_RFLAGS); >> >> >> >> if (cpu_state->interrupt_request & CPU_INTERRUPT_INIT) { >> >> - hvf_cpu_synchronize_state(cpu_state); >> >> + cpu_synchronize_state(cpu_state); >> >> do_cpu_init(cpu); >> >> } >> >> >> >> @@ -451,12 +452,12 @@ int hvf_process_events(CPUState *cpu_state) >> >> cpu_state->halted = 0; >> >> } >> >> if (cpu_state->interrupt_request & CPU_INTERRUPT_SIPI) { >> >> - hvf_cpu_synchronize_state(cpu_state); >> >> + cpu_synchronize_state(cpu_state); >> >> do_cpu_sipi(cpu); >> >> } >> >> if (cpu_state->interrupt_request & CPU_INTERRUPT_TPR) { >> >> cpu_state->interrupt_request &= ~CPU_INTERRUPT_TPR; >> >> - hvf_cpu_synchronize_state(cpu_state); >> >> + cpu_synchronize_state(cpu_state); >> > The changes from hvf_cpu_*() to cpu_*() are cleanup and perhaps should >> > be a separate patch. It follows cpu/accel cleanups Claudio was doing the >> > summer. >> >> >> The only reason they're in here is because we no longer have access to >> the hvf_ functions from the file. I am perfectly happy to rebase the >> patch on top of Claudio's if his goes in first. I'm sure it'll be >> trivial for him to rebase on top of this too if my series goes in first. >> >> >> > >> > Phillipe raised the idea that the patch might go ahead of ARM-specific >> > part (which might involve some discussions) and I agree with that. >> > >> > Some sync between Claudio series (CC'd him) and the patch might be need. >> >> >> I would prefer not to hold back because of the sync. Claudio's cleanup >> is trivial enough to adjust for if it gets merged ahead of this. >> >> >> Alex >> >> >> >> --00000000000090ab2205b558ad7a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Update: We're not quite sure how to compare the CNTV_C= VAL and CNTVCT. But the high CPU usage seems to be mitigated by having a po= ll interval (like KVM does) in handling WFI:


This is loosely inspired by=C2=A0https://elixir.bootlin.com/linux/v5.10-rc6/source/virt/kvm/kvm_main.c= #L2766 which does seem to specify a poll interval.

=
It would be cool if we could have a lightweight way to enter sleep and= restart the vcpus precisely when CVAL passes, though.

=
Frank


On Fri, Nov 27, 2020 at 3:30 PM Frank Yang &= lt;lfy@google.com> wrote:
Hi all,=


I'm a develo= per on the Android Emulator, which is in a fork of QEMU.

Peter and I have been working on an HVF Apple Silicon backend with a= n eye toward Android guests.

We have gotten things= to basically switch to Android userspace already (logcat/shell and graphic= s available at least)

Our strategy so far has been= to import logic from the KVM implementation and hook into QEMU's softw= are devices that=C2=A0previously assumed to only=C2=A0work with TCG, or hav= e KVM-specific paths.

Thanks to Alexander for the = tip on the 36-bit address space limitation btw; our way of addressing this = is to still allow highmem but not put pci high mmio so high.

=
Also, note we have a sleep/signal based mechanism to deal with W= Fx, which might be worth looking into in Alexander's implementation as = well:

=
Patches so far, FYI:


<= div>https://android.g= ooglesource.com/platform/external/qemu/+/eccd9e47ab2ccb9003455e3bb721f57f9e= bc3c01

Peter's also noticed that there are extra = steps needed for M1's to allow TCG to work, as it involves JIT:


We'd appreciate a= ny feedback/comments :)

Best,

=
Frank

On Fri, Nov 27, 2020 at 1:57 PM Alexander Graf <agraf@csgraf.de> wrote= :

On 27.11.20 21:00, Roman Bolshakov wrote:
> On Thu, Nov 26, 2020 at 10:50:11PM +0100, Alexander Graf wrote:
>> Until now, Hypervisor.framework has only been available on x86_64 = systems.
>> With Apple Silicon shipping now, it extends its reach to aarch64. = To
>> prepare for support for multiple architectures, let's move com= mon code out
>> into its own accel directory.
>>
>> Signed-off-by: Alexander Graf <agraf@csgraf.de>
>> ---
>>=C2=A0 =C2=A0MAINTAINERS=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A09 +-
>>=C2=A0 =C2=A0accel/hvf/hvf-all.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|= =C2=A0 56 +++++
>>=C2=A0 =C2=A0accel/hvf/hvf-cpus.c=C2=A0 =C2=A0 =C2=A0 =C2=A0 | 468 = ++++++++++++++++++++++++++++++++++++
>>=C2=A0 =C2=A0accel/hvf/meson.build=C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2= =A0 =C2=A07 +
>>=C2=A0 =C2=A0accel/meson.build=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|=C2=A0 =C2=A01 +
>>=C2=A0 =C2=A0include/sysemu/hvf_int.h=C2=A0 =C2=A0 |=C2=A0 69 +++++= +
>>=C2=A0 =C2=A0target/i386/hvf/hvf-cpus.c=C2=A0 | 131 ----------
>>=C2=A0 =C2=A0target/i386/hvf/hvf-cpus.h=C2=A0 |=C2=A0 25 --
>>=C2=A0 =C2=A0target/i386/hvf/hvf-i386.h=C2=A0 |=C2=A0 48 +---
>>=C2=A0 =C2=A0target/i386/hvf/hvf.c=C2=A0 =C2=A0 =C2=A0 =C2=A0| 360 = +--------------------------
>>=C2=A0 =C2=A0target/i386/hvf/meson.build |=C2=A0 =C2=A01 -
>>=C2=A0 =C2=A0target/i386/hvf/x86hvf.c=C2=A0 =C2=A0 |=C2=A0 11 +- >>=C2=A0 =C2=A0target/i386/hvf/x86hvf.h=C2=A0 =C2=A0 |=C2=A0 =C2=A02 = -
>>=C2=A0 =C2=A013 files changed, 619 insertions(+), 569 deletions(-)<= br> >>=C2=A0 =C2=A0create mode 100644 accel/hvf/hvf-all.c
>>=C2=A0 =C2=A0create mode 100644 accel/hvf/hvf-cpus.c
>>=C2=A0 =C2=A0create mode 100644 accel/hvf/meson.build
>>=C2=A0 =C2=A0create mode 100644 include/sysemu/hvf_int.h
>>=C2=A0 =C2=A0delete mode 100644 target/i386/hvf/hvf-cpus.c
>>=C2=A0 =C2=A0delete mode 100644 target/i386/hvf/hvf-cpus.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 68bc160f41..ca4b6d9279 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -444,9 +444,16 @@ M: Cameron Esfahani <dirty@apple.com>
>>=C2=A0 =C2=A0M: Roman Bolshakov <r.bolshakov@yadro.com>
>>=C2=A0 =C2=A0W: https://wiki.qemu.org/Features/HVF >>=C2=A0 =C2=A0S: Maintained
>> -F: accel/stubs/hvf-stub.c
> There was a patch for that in the RFC series from Claudio.


Yeah, I'm not worried about this hunk :).


>
>>=C2=A0 =C2=A0F: target/i386/hvf/
>> +
>> +HVF
>> +M: Cameron Esfahani <dirty@apple.com>
>> +M: Roman Bolshakov <r.bolshakov@yadro.com>
>> +W: https://wiki.qemu.org/Features/HVF
>> +S: Maintained
>> +F: accel/hvf/
>>=C2=A0 =C2=A0F: include/sysemu/hvf.h
>> +F: include/sysemu/hvf_int.h
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0WHPX CPUs
>>=C2=A0 =C2=A0M: Sunil Muthuswamy <sunilmut@microsoft.com>
>> diff --git a/accel/hvf/hvf-all.c b/accel/hvf/hvf-all.c
>> new file mode 100644
>> index 0000000000..47d77a472a
>> --- /dev/null
>> +++ b/accel/hvf/hvf-all.c
>> @@ -0,0 +1,56 @@
>> +/*
>> + * QEMU Hypervisor.framework support
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version = 2.=C2=A0 See
>> + * the COPYING file in the top-level directory.
>> + *
>> + * Contributions after 2012-01-13 are licensed under the terms of= the
>> + * GNU GPL, version 2 or (at your option) any later version.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu-common.h"
>> +#include "qemu/error-report.h"
>> +#include "sysemu/hvf.h"
>> +#include "sysemu/hvf_int.h"
>> +#include "sysemu/runstate.h"
>> +
>> +#include "qemu/main-loop.h"
>> +#include "sysemu/accel.h"
>> +
>> +#include <Hypervisor/Hypervisor.h>
>> +
>> +bool hvf_allowed;
>> +HVFState *hvf_state;
>> +
>> +void assert_hvf_ok(hv_return_t ret)
>> +{
>> +=C2=A0 =C2=A0 if (ret =3D=3D HV_SUCCESS) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 switch (ret) {
>> +=C2=A0 =C2=A0 case HV_ERROR:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_ERROR&qu= ot;);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 case HV_BUSY:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_BUSY&quo= t;);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 case HV_BAD_ARGUMENT:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_BAD_ARGU= MENT");
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 case HV_NO_RESOURCES:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_NO_RESOU= RCES");
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 case HV_NO_DEVICE:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_NO_DEVIC= E");
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 case HV_UNSUPPORTED:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_UNSUPPOR= TED");
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 default:
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Unknown Error"= ;);
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 abort();
>> +}
>> diff --git a/accel/hvf/hvf-cpus.c b/accel/hvf/hvf-cpus.c
>> new file mode 100644
>> index 0000000000..f9bb5502b7
>> --- /dev/null
>> +++ b/accel/hvf/hvf-cpus.c
>> @@ -0,0 +1,468 @@
>> +/*
>> + * Copyright 2008 IBM Corporation
>> + *=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02008 Red Hat, Inc.
>> + * Copyright 2011 Intel Corporation
>> + * Copyright 2016 Veertu, Inc.
>> + * Copyright 2017 The Android Open Source Project
>> + *
>> + * QEMU Hypervisor.framework support
>> + *
>> + * This program is free software; you can redistribute it and/or<= br> >> + * modify it under the terms of version 2 of the GNU General Publ= ic
>> + * License as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful= ,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of=
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.=C2=A0 See= the GNU
>> + * General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public Lice= nse
>> + * along with this program; if not, see <http://www.gnu.org= /licenses/>.
>> + *
>> + * This file contain code under public domain from the hvdos proj= ect:
>> + * https://github.com/mist64/hvdos
>> + *
>> + * Parts Copyright (c) 2011 NetApp, Inc.
>> + * All rights reserved.
>> + *
>> + * Redistribution and use in source and binary forms, with or wit= hout
>> + * modification, are permitted provided that the following condit= ions
>> + * are met:
>> + * 1. Redistributions of source code must retain the above copyri= ght
>> + *=C2=A0 =C2=A0 notice, this list of conditions and the following= disclaimer.
>> + * 2. Redistributions in binary form must reproduce the above cop= yright
>> + *=C2=A0 =C2=A0 notice, this list of conditions and the following= disclaimer in the
>> + *=C2=A0 =C2=A0 documentation and/or other materials provided wit= h the distribution.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY NETAPP, INC ``AS IS'' AND=
>> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED = TO, THE
>> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTIC= ULAR PURPOSE
>> + * ARE DISCLAIMED.=C2=A0 IN NO EVENT SHALL NETAPP, INC OR CONTRIB= UTORS BE LIABLE
>> + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR C= ONSEQUENTIAL
>> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTIT= UTE GOODS
>> + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERR= UPTION)
>> + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONT= RACT, STRICT
>> + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING= IN ANY WAY
>> + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIB= ILITY OF
>> + * SUCH DAMAGE.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/error-report.h"
>> +#include "qemu/main-loop.h"
>> +#include "exec/address-spaces.h"
>> +#include "exec/exec-all.h"
>> +#include "sysemu/cpus.h"
>> +#include "sysemu/hvf.h"
>> +#include "sysemu/hvf_int.h"
>> +#include "sysemu/runstate.h"
>> +#include "qemu/guest-random.h"
>> +
>> +#include <Hypervisor/Hypervisor.h>
>> +
>> +/* Memory slots */
>> +
>> +struct mac_slot {
>> +=C2=A0 =C2=A0 int present;
>> +=C2=A0 =C2=A0 uint64_t size;
>> +=C2=A0 =C2=A0 uint64_t gpa_start;
>> +=C2=A0 =C2=A0 uint64_t gva;
>> +};
>> +
>> +hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size) >> +{
>> +=C2=A0 =C2=A0 hvf_slot *slot;
>> +=C2=A0 =C2=A0 int x;
>> +=C2=A0 =C2=A0 for (x =3D 0; x < hvf_state->num_slots; ++x) = {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 slot =3D &hvf_state->slots[x];=
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (slot->size && start &l= t; (slot->start + slot->size) &&
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (start + size) > slo= t->start) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return slot;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 return NULL;
>> +}
>> +
>> +struct mac_slot mac_slots[32];
>> +
>> +static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t fl= ags)
>> +{
>> +=C2=A0 =C2=A0 struct mac_slot *macslot;
>> +=C2=A0 =C2=A0 hv_return_t ret;
>> +
>> +=C2=A0 =C2=A0 macslot =3D &mac_slots[slot->slot_id];
>> +
>> +=C2=A0 =C2=A0 if (macslot->present) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (macslot->size !=3D slot->si= ze) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 macslot->present =3D= 0;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ret =3D hv_vm_unmap(mac= slot->gpa_start, macslot->size);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 assert_hvf_ok(ret);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 if (!slot->size) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 macslot->present =3D 1;
>> +=C2=A0 =C2=A0 macslot->gpa_start =3D slot->start;
>> +=C2=A0 =C2=A0 macslot->size =3D slot->size;
>> +=C2=A0 =C2=A0 ret =3D hv_vm_map(slot->mem, slot->start, slo= t->size, flags);
>> +=C2=A0 =C2=A0 assert_hvf_ok(ret);
>> +=C2=A0 =C2=A0 return 0;
>> +}
>> +
>> +static void hvf_set_phys_mem(MemoryRegionSection *section, bool a= dd)
>> +{
>> +=C2=A0 =C2=A0 hvf_slot *mem;
>> +=C2=A0 =C2=A0 MemoryRegion *area =3D section->mr;
>> +=C2=A0 =C2=A0 bool writeable =3D !area->readonly && !a= rea->rom_device;
>> +=C2=A0 =C2=A0 hv_memory_flags_t flags;
>> +
>> +=C2=A0 =C2=A0 if (!memory_region_is_ram(area)) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (writeable) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 } else if (!memory_region_is_romd(are= a)) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /*
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* If the memory d= evice is not in romd_mode, then we actually want
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* to remove the h= vf memory slot so all accesses will trap.
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0add =3D false; >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 mem =3D hvf_find_overlap_slot(
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 section->offset_with= in_address_space,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int128_get64(section-&g= t;size));
>> +
>> +=C2=A0 =C2=A0 if (mem && add) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (mem->size =3D=3D int128_get64(= section->size) &&
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mem->start =3D=3D se= ction->offset_within_address_space &&
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mem->mem =3D=3D (mem= ory_region_get_ram_ptr(area) +
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 section->offset_with= in_region)) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return; /* Same region = was attempted to register, go away. */
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 /* Region needs to be reset. set the size to 0 and = remap it. */
>> +=C2=A0 =C2=A0 if (mem) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 mem->size =3D 0;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (do_hvf_set_memory(mem, 0)) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Fail= ed to reset overlapping slot");
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 if (!add) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 if (area->readonly ||
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 (!memory_region_is_ram(area) &&am= p; memory_region_is_romd(area))) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 flags =3D HV_MEMORY_READ | HV_MEMORY_= EXEC;
>> +=C2=A0 =C2=A0 } else {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 flags =3D HV_MEMORY_READ | HV_MEMORY_= WRITE | HV_MEMORY_EXEC;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 /* Now make a new slot. */
>> +=C2=A0 =C2=A0 int x;
>> +
>> +=C2=A0 =C2=A0 for (x =3D 0; x < hvf_state->num_slots; ++x) = {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 mem =3D &hvf_state->slots[x];<= br> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!mem->size) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 if (x =3D=3D hvf_state->num_slots) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("No free slots"= ;);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 mem->size =3D int128_get64(section->size); >> +=C2=A0 =C2=A0 mem->mem =3D memory_region_get_ram_ptr(area) + s= ection->offset_within_region;
>> +=C2=A0 =C2=A0 mem->start =3D section->offset_within_address= _space;
>> +=C2=A0 =C2=A0 mem->region =3D area;
>> +
>> +=C2=A0 =C2=A0 if (do_hvf_set_memory(mem, flags)) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error registering = new memory slot");
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();
>> +=C2=A0 =C2=A0 }
>> +}
>> +
>> +static void hvf_set_dirty_tracking(MemoryRegionSection *section, = bool on)
>> +{
>> +=C2=A0 =C2=A0 hvf_slot *slot;
>> +
>> +=C2=A0 =C2=A0 slot =3D hvf_find_overlap_slot(
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 section->offset_with= in_address_space,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int128_get64(section-&g= t;size));
>> +
>> +=C2=A0 =C2=A0 /* protect region against writes; begin tracking it= */
>> +=C2=A0 =C2=A0 if (on) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 slot->flags |=3D HVF_SLOT_LOG;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 hv_vm_protect((uintptr_t)slot->sta= rt, (size_t)slot->size,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 HV_MEMORY_READ);
>> +=C2=A0 =C2=A0 /* stop tracking region*/
>> +=C2=A0 =C2=A0 } else {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 slot->flags &=3D ~HVF_SLOT_LOG= ;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 hv_vm_protect((uintptr_t)slot->sta= rt, (size_t)slot->size,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 HV_MEMORY_READ | HV_MEMORY_WRITE);
>> +=C2=A0 =C2=A0 }
>> +}
>> +
>> +static void hvf_log_start(MemoryListener *listener,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 MemoryRegionSection *section, int old, int new)=
>> +{
>> +=C2=A0 =C2=A0 if (old !=3D 0) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 hvf_set_dirty_tracking(section, 1);
>> +}
>> +
>> +static void hvf_log_stop(MemoryListener *listener,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section, int old, int new)<= br> >> +{
>> +=C2=A0 =C2=A0 if (new !=3D 0) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 hvf_set_dirty_tracking(section, 0);
>> +}
>> +
>> +static void hvf_log_sync(MemoryListener *listener,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section)
>> +{
>> +=C2=A0 =C2=A0 /*
>> +=C2=A0 =C2=A0 =C2=A0* sync of dirty pages is handled elsewhere; j= ust make sure we keep
>> +=C2=A0 =C2=A0 =C2=A0* tracking the region.
>> +=C2=A0 =C2=A0 =C2=A0*/
>> +=C2=A0 =C2=A0 hvf_set_dirty_tracking(section, 1);
>> +}
>> +
>> +static void hvf_region_add(MemoryListener *listener,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section)
>> +{
>> +=C2=A0 =C2=A0 hvf_set_phys_mem(section, true);
>> +}
>> +
>> +static void hvf_region_del(MemoryListener *listener,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section)
>> +{
>> +=C2=A0 =C2=A0 hvf_set_phys_mem(section, false);
>> +}
>> +
>> +static MemoryListener hvf_memory_listener =3D {
>> +=C2=A0 =C2=A0 .priority =3D 10,
>> +=C2=A0 =C2=A0 .region_add =3D hvf_region_add,
>> +=C2=A0 =C2=A0 .region_del =3D hvf_region_del,
>> +=C2=A0 =C2=A0 .log_start =3D hvf_log_start,
>> +=C2=A0 =C2=A0 .log_stop =3D hvf_log_stop,
>> +=C2=A0 =C2=A0 .log_sync =3D hvf_log_sync,
>> +};
>> +
>> +static void do_hvf_cpu_synchronize_state(CPUState *cpu, run_on_cp= u_data arg)
>> +{
>> +=C2=A0 =C2=A0 if (!cpu->vcpu_dirty) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 hvf_get_registers(cpu);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu->vcpu_dirty =3D true;
>> +=C2=A0 =C2=A0 }
>> +}
>> +
>> +static void hvf_cpu_synchronize_state(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 if (!cpu->vcpu_dirty) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchroniz= e_state, RUN_ON_CPU_NULL);
>> +=C2=A0 =C2=A0 }
>> +}
>> +
>> +static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 run_on_cpu_data arg)
>> +{
>> +=C2=A0 =C2=A0 hvf_put_registers(cpu);
>> +=C2=A0 =C2=A0 cpu->vcpu_dirty =3D false;
>> +}
>> +
>> +static void hvf_cpu_synchronize_post_reset(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchronize_post_reset, = RUN_ON_CPU_NULL);
>> +}
>> +
>> +static void do_hvf_cpu_synchronize_post_init(CPUState *cpu,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0run_on_cpu_data arg)
>> +{
>> +=C2=A0 =C2=A0 hvf_put_registers(cpu);
>> +=C2=A0 =C2=A0 cpu->vcpu_dirty =3D false;
>> +}
>> +
>> +static void hvf_cpu_synchronize_post_init(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchronize_post_init, R= UN_ON_CPU_NULL);
>> +}
>> +
>> +static void do_hvf_cpu_synchronize_pre_loadvm(CPUState *cpu,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 run_on_cpu_data arg)
>> +{
>> +=C2=A0 =C2=A0 cpu->vcpu_dirty =3D true;
>> +}
>> +
>> +static void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchronize_pre_loadvm, = RUN_ON_CPU_NULL);
>> +}
>> +
>> +static void hvf_vcpu_destroy(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 hv_return_t ret =3D hv_vcpu_destroy(cpu->hvf_fd)= ;
>> +=C2=A0 =C2=A0 assert_hvf_ok(ret);
>> +
>> +=C2=A0 =C2=A0 hvf_arch_vcpu_destroy(cpu);
>> +}
>> +
>> +static void dummy_signal(int sig)
>> +{
>> +}
>> +
>> +static int hvf_init_vcpu(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 int r;
>> +
>> +=C2=A0 =C2=A0 /* init cpu signals */
>> +=C2=A0 =C2=A0 sigset_t set;
>> +=C2=A0 =C2=A0 struct sigaction sigact;
>> +
>> +=C2=A0 =C2=A0 memset(&sigact, 0, sizeof(sigact));
>> +=C2=A0 =C2=A0 sigact.sa_handler =3D dummy_signal;
>> +=C2=A0 =C2=A0 sigaction(SIG_IPI, &sigact, NULL);
>> +
>> +=C2=A0 =C2=A0 pthread_sigmask(SIG_BLOCK, NULL, &set);
>> +=C2=A0 =C2=A0 sigdelset(&set, SIG_IPI);
>> +
>> +#ifdef __aarch64__
>> +=C2=A0 =C2=A0 r =3D hv_vcpu_create(&cpu->hvf_fd, (hv_vcpu_= exit_t **)&cpu->hvf_exit, NULL);
>> +#else
>> +=C2=A0 =C2=A0 r =3D hv_vcpu_create((hv_vcpuid_t *)&cpu->hv= f_fd, HV_VCPU_DEFAULT);
>> +#endif
> I think the first __aarch64__ bit fits better to arm part of the serie= s.


Oops. Thanks for catching it! Yes, absolutely. It should be part of the ARM enablement.


>
>> +=C2=A0 =C2=A0 cpu->vcpu_dirty =3D 1;
>> +=C2=A0 =C2=A0 assert_hvf_ok(r);
>> +
>> +=C2=A0 =C2=A0 return hvf_arch_init_vcpu(cpu);
>> +}
>> +
>> +/*
>> + * The HVF-specific vCPU thread function. This one should only ru= n when the host
>> + * CPU supports the VMX "unrestricted guest" feature. >> + */
>> +static void *hvf_cpu_thread_fn(void *arg)
>> +{
>> +=C2=A0 =C2=A0 CPUState *cpu =3D arg;
>> +
>> +=C2=A0 =C2=A0 int r;
>> +
>> +=C2=A0 =C2=A0 assert(hvf_enabled());
>> +
>> +=C2=A0 =C2=A0 rcu_register_thread();
>> +
>> +=C2=A0 =C2=A0 qemu_mutex_lock_iothread();
>> +=C2=A0 =C2=A0 qemu_thread_get_self(cpu->thread);
>> +
>> +=C2=A0 =C2=A0 cpu->thread_id =3D qemu_get_thread_id();
>> +=C2=A0 =C2=A0 cpu->can_do_io =3D 1;
>> +=C2=A0 =C2=A0 current_cpu =3D cpu;
>> +
>> +=C2=A0 =C2=A0 hvf_init_vcpu(cpu);
>> +
>> +=C2=A0 =C2=A0 /* signal CPU creation */
>> +=C2=A0 =C2=A0 cpu_thread_signal_created(cpu);
>> +=C2=A0 =C2=A0 qemu_guest_random_seed_thread_part2(cpu->random_= seed);
>> +
>> +=C2=A0 =C2=A0 do {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (cpu_can_run(cpu)) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r =3D hvf_vcpu_exec(cpu= );
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (r =3D=3D EXCP_DEBUG= ) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu_handl= e_guest_debug(cpu);
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 qemu_wait_io_event(cpu);
>> +=C2=A0 =C2=A0 } while (!cpu->unplug || cpu_can_run(cpu));
>> +
>> +=C2=A0 =C2=A0 hvf_vcpu_destroy(cpu);
>> +=C2=A0 =C2=A0 cpu_thread_signal_destroyed(cpu);
>> +=C2=A0 =C2=A0 qemu_mutex_unlock_iothread();
>> +=C2=A0 =C2=A0 rcu_unregister_thread();
>> +=C2=A0 =C2=A0 return NULL;
>> +}
>> +
>> +static void hvf_start_vcpu_thread(CPUState *cpu)
>> +{
>> +=C2=A0 =C2=A0 char thread_name[VCPU_THREAD_NAME_SIZE];
>> +
>> +=C2=A0 =C2=A0 /*
>> +=C2=A0 =C2=A0 =C2=A0* HVF currently does not support TCG, and onl= y runs in
>> +=C2=A0 =C2=A0 =C2=A0* unrestricted-guest mode.
>> +=C2=A0 =C2=A0 =C2=A0*/
>> +=C2=A0 =C2=A0 assert(hvf_enabled());
>> +
>> +=C2=A0 =C2=A0 cpu->thread =3D g_malloc0(sizeof(QemuThread)); >> +=C2=A0 =C2=A0 cpu->halt_cond =3D g_malloc0(sizeof(QemuCond));<= br> >> +=C2=A0 =C2=A0 qemu_cond_init(cpu->halt_cond);
>> +
>> +=C2=A0 =C2=A0 snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "= CPU %d/HVF",
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu->cpu_index= );
>> +=C2=A0 =C2=A0 qemu_thread_create(cpu->thread, thread_name, hvf= _cpu_thread_fn,
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0cpu, QEMU_THREAD_JOINABLE);
>> +}
>> +
>> +static const CpusAccel hvf_cpus =3D {
>> +=C2=A0 =C2=A0 .create_vcpu_thread =3D hvf_start_vcpu_thread,
>> +
>> +=C2=A0 =C2=A0 .synchronize_post_reset =3D hvf_cpu_synchronize_pos= t_reset,
>> +=C2=A0 =C2=A0 .synchronize_post_init =3D hvf_cpu_synchronize_post= _init,
>> +=C2=A0 =C2=A0 .synchronize_state =3D hvf_cpu_synchronize_state, >> +=C2=A0 =C2=A0 .synchronize_pre_loadvm =3D hvf_cpu_synchronize_pre= _loadvm,
>> +};
>> +
>> +static int hvf_accel_init(MachineState *ms)
>> +{
>> +=C2=A0 =C2=A0 int x;
>> +=C2=A0 =C2=A0 hv_return_t ret;
>> +=C2=A0 =C2=A0 HVFState *s;
>> +
>> +=C2=A0 =C2=A0 ret =3D hv_vm_create(HV_VM_DEFAULT);
>> +=C2=A0 =C2=A0 assert_hvf_ok(ret);
>> +
>> +=C2=A0 =C2=A0 s =3D g_new0(HVFState, 1);
>> +
>> +=C2=A0 =C2=A0 s->num_slots =3D 32;
>> +=C2=A0 =C2=A0 for (x =3D 0; x < s->num_slots; ++x) {
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 s->slots[x].size =3D 0;
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 s->slots[x].slot_id =3D x;
>> +=C2=A0 =C2=A0 }
>> +
>> +=C2=A0 =C2=A0 hvf_state =3D s;
>> +=C2=A0 =C2=A0 memory_listener_register(&hvf_memory_listener, = &address_space_memory);
>> +=C2=A0 =C2=A0 cpus_register_accel(&hvf_cpus);
>> +=C2=A0 =C2=A0 return 0;
>> +}
>> +
>> +static void hvf_accel_class_init(ObjectClass *oc, void *data)
>> +{
>> +=C2=A0 =C2=A0 AccelClass *ac =3D ACCEL_CLASS(oc);
>> +=C2=A0 =C2=A0 ac->name =3D "HVF";
>> +=C2=A0 =C2=A0 ac->init_machine =3D hvf_accel_init;
>> +=C2=A0 =C2=A0 ac->allowed =3D &hvf_allowed;
>> +}
>> +
>> +static const TypeInfo hvf_accel_type =3D {
>> +=C2=A0 =C2=A0 .name =3D TYPE_HVF_ACCEL,
>> +=C2=A0 =C2=A0 .parent =3D TYPE_ACCEL,
>> +=C2=A0 =C2=A0 .class_init =3D hvf_accel_class_init,
>> +};
>> +
>> +static void hvf_type_init(void)
>> +{
>> +=C2=A0 =C2=A0 type_register_static(&hvf_accel_type);
>> +}
>> +
>> +type_init(hvf_type_init);
>> diff --git a/accel/hvf/meson.build b/accel/hvf/meson.build
>> new file mode 100644
>> index 0000000000..dfd6b68dc7
>> --- /dev/null
>> +++ b/accel/hvf/meson.build
>> @@ -0,0 +1,7 @@
>> +hvf_ss =3D ss.source_set()
>> +hvf_ss.add(files(
>> +=C2=A0 'hvf-all.c',
>> +=C2=A0 'hvf-cpus.c',
>> +))
>> +
>> +specific_ss.add_all(when: 'CONFIG_HVF', if_true: hvf_ss)<= br> >> diff --git a/accel/meson.build b/accel/meson.build
>> index b26cca227a..6de12ce5d5 100644
>> --- a/accel/meson.build
>> +++ b/accel/meson.build
>> @@ -1,5 +1,6 @@
>>=C2=A0 =C2=A0softmmu_ss.add(files('accel.c'))
>>=C2=A0 =C2=A0
>> +subdir('hvf')
>>=C2=A0 =C2=A0subdir('qtest')
>>=C2=A0 =C2=A0subdir('kvm')
>>=C2=A0 =C2=A0subdir('tcg')
>> diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h >> new file mode 100644
>> index 0000000000..de9bad23a8
>> --- /dev/null
>> +++ b/include/sysemu/hvf_int.h
>> @@ -0,0 +1,69 @@
>> +/*
>> + * QEMU Hypervisor.framework (HVF) support
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version = 2 or later.
>> + * See the COPYING file in the top-level directory.
>> + *
>> + */
>> +
>> +/* header to be included in HVF-specific code */
>> +
>> +#ifndef HVF_INT_H
>> +#define HVF_INT_H
>> +
>> +#include <Hypervisor/Hypervisor.h>
>> +
>> +#define HVF_MAX_VCPU 0x10
>> +
>> +extern struct hvf_state hvf_global;
>> +
>> +struct hvf_vm {
>> +=C2=A0 =C2=A0 int id;
>> +=C2=A0 =C2=A0 struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU];
>> +};
>> +
>> +struct hvf_state {
>> +=C2=A0 =C2=A0 uint32_t version;
>> +=C2=A0 =C2=A0 struct hvf_vm *vm;
>> +=C2=A0 =C2=A0 uint64_t mem_quota;
>> +};
>> +
>> +/* hvf_slot flags */
>> +#define HVF_SLOT_LOG (1 << 0)
>> +
>> +typedef struct hvf_slot {
>> +=C2=A0 =C2=A0 uint64_t start;
>> +=C2=A0 =C2=A0 uint64_t size;
>> +=C2=A0 =C2=A0 uint8_t *mem;
>> +=C2=A0 =C2=A0 int slot_id;
>> +=C2=A0 =C2=A0 uint32_t flags;
>> +=C2=A0 =C2=A0 MemoryRegion *region;
>> +} hvf_slot;
>> +
>> +typedef struct hvf_vcpu_caps {
>> +=C2=A0 =C2=A0 uint64_t vmx_cap_pinbased;
>> +=C2=A0 =C2=A0 uint64_t vmx_cap_procbased;
>> +=C2=A0 =C2=A0 uint64_t vmx_cap_procbased2;
>> +=C2=A0 =C2=A0 uint64_t vmx_cap_entry;
>> +=C2=A0 =C2=A0 uint64_t vmx_cap_exit;
>> +=C2=A0 =C2=A0 uint64_t vmx_cap_preemption_timer;
>> +} hvf_vcpu_caps;
>> +
>> +struct HVFState {
>> +=C2=A0 =C2=A0 AccelState parent;
>> +=C2=A0 =C2=A0 hvf_slot slots[32];
>> +=C2=A0 =C2=A0 int num_slots;
>> +
>> +=C2=A0 =C2=A0 hvf_vcpu_caps *hvf_caps;
>> +};
>> +extern HVFState *hvf_state;
>> +
>> +void assert_hvf_ok(hv_return_t ret);
>> +int hvf_get_registers(CPUState *cpu);
>> +int hvf_put_registers(CPUState *cpu);
>> +int hvf_arch_init_vcpu(CPUState *cpu);
>> +void hvf_arch_vcpu_destroy(CPUState *cpu);
>> +int hvf_vcpu_exec(CPUState *cpu);
>> +hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t);
>> +
>> +#endif
>> diff --git a/target/i386/hvf/hvf-cpus.c b/target/i386/hvf/hvf-cpus= .c
>> deleted file mode 100644
>> index 817b3d7452..0000000000
>> --- a/target/i386/hvf/hvf-cpus.c
>> +++ /dev/null
>> @@ -1,131 +0,0 @@
>> -/*
>> - * Copyright 2008 IBM Corporation
>> - *=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02008 Red Hat, Inc.
>> - * Copyright 2011 Intel Corporation
>> - * Copyright 2016 Veertu, Inc.
>> - * Copyright 2017 The Android Open Source Project
>> - *
>> - * QEMU Hypervisor.framework support
>> - *
>> - * This program is free software; you can redistribute it and/or<= br> >> - * modify it under the terms of version 2 of the GNU General Publ= ic
>> - * License as published by the Free Software Foundation.
>> - *
>> - * This program is distributed in the hope that it will be useful= ,
>> - * but WITHOUT ANY WARRANTY; without even the implied warranty of=
>> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.=C2=A0 See= the GNU
>> - * General Public License for more details.
>> - *
>> - * You should have received a copy of the GNU General Public Lice= nse
>> - * along with this program; if not, see <http://www.gnu.org= /licenses/>.
>> - *
>> - * This file contain code under public domain from the hvdos proj= ect:
>> - * https://github.com/mist64/hvdos
>> - *
>> - * Parts Copyright (c) 2011 NetApp, Inc.
>> - * All rights reserved.
>> - *
>> - * Redistribution and use in source and binary forms, with or wit= hout
>> - * modification, are permitted provided that the following condit= ions
>> - * are met:
>> - * 1. Redistributions of source code must retain the above copyri= ght
>> - *=C2=A0 =C2=A0 notice, this list of conditions and the following= disclaimer.
>> - * 2. Redistributions in binary form must reproduce the above cop= yright
>> - *=C2=A0 =C2=A0 notice, this list of conditions and the following= disclaimer in the
>> - *=C2=A0 =C2=A0 documentation and/or other materials provided wit= h the distribution.
>> - *
>> - * THIS SOFTWARE IS PROVIDED BY NETAPP, INC ``AS IS'' AND=
>> - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED = TO, THE
>> - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTIC= ULAR PURPOSE
>> - * ARE DISCLAIMED.=C2=A0 IN NO EVENT SHALL NETAPP, INC OR CONTRIB= UTORS BE LIABLE
>> - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR C= ONSEQUENTIAL
>> - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTIT= UTE GOODS
>> - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERR= UPTION)
>> - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONT= RACT, STRICT
>> - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING= IN ANY WAY
>> - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIB= ILITY OF
>> - * SUCH DAMAGE.
>> - */
>> -
>> -#include "qemu/osdep.h"
>> -#include "qemu/error-report.h"
>> -#include "qemu/main-loop.h"
>> -#include "sysemu/hvf.h"
>> -#include "sysemu/runstate.h"
>> -#include "target/i386/cpu.h"
>> -#include "qemu/guest-random.h"
>> -
>> -#include "hvf-cpus.h"
>> -
>> -/*
>> - * The HVF-specific vCPU thread function. This one should only ru= n when the host
>> - * CPU supports the VMX "unrestricted guest" feature. >> - */
>> -static void *hvf_cpu_thread_fn(void *arg)
>> -{
>> -=C2=A0 =C2=A0 CPUState *cpu =3D arg;
>> -
>> -=C2=A0 =C2=A0 int r;
>> -
>> -=C2=A0 =C2=A0 assert(hvf_enabled());
>> -
>> -=C2=A0 =C2=A0 rcu_register_thread();
>> -
>> -=C2=A0 =C2=A0 qemu_mutex_lock_iothread();
>> -=C2=A0 =C2=A0 qemu_thread_get_self(cpu->thread);
>> -
>> -=C2=A0 =C2=A0 cpu->thread_id =3D qemu_get_thread_id();
>> -=C2=A0 =C2=A0 cpu->can_do_io =3D 1;
>> -=C2=A0 =C2=A0 current_cpu =3D cpu;
>> -
>> -=C2=A0 =C2=A0 hvf_init_vcpu(cpu);
>> -
>> -=C2=A0 =C2=A0 /* signal CPU creation */
>> -=C2=A0 =C2=A0 cpu_thread_signal_created(cpu);
>> -=C2=A0 =C2=A0 qemu_guest_random_seed_thread_part2(cpu->random_= seed);
>> -
>> -=C2=A0 =C2=A0 do {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (cpu_can_run(cpu)) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r =3D hvf_vcpu_exec(cpu= );
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (r =3D=3D EXCP_DEBUG= ) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu_handl= e_guest_debug(cpu);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 qemu_wait_io_event(cpu);
>> -=C2=A0 =C2=A0 } while (!cpu->unplug || cpu_can_run(cpu));
>> -
>> -=C2=A0 =C2=A0 hvf_vcpu_destroy(cpu);
>> -=C2=A0 =C2=A0 cpu_thread_signal_destroyed(cpu);
>> -=C2=A0 =C2=A0 qemu_mutex_unlock_iothread();
>> -=C2=A0 =C2=A0 rcu_unregister_thread();
>> -=C2=A0 =C2=A0 return NULL;
>> -}
>> -
>> -static void hvf_start_vcpu_thread(CPUState *cpu)
>> -{
>> -=C2=A0 =C2=A0 char thread_name[VCPU_THREAD_NAME_SIZE];
>> -
>> -=C2=A0 =C2=A0 /*
>> -=C2=A0 =C2=A0 =C2=A0* HVF currently does not support TCG, and onl= y runs in
>> -=C2=A0 =C2=A0 =C2=A0* unrestricted-guest mode.
>> -=C2=A0 =C2=A0 =C2=A0*/
>> -=C2=A0 =C2=A0 assert(hvf_enabled());
>> -
>> -=C2=A0 =C2=A0 cpu->thread =3D g_malloc0(sizeof(QemuThread)); >> -=C2=A0 =C2=A0 cpu->halt_cond =3D g_malloc0(sizeof(QemuCond));<= br> >> -=C2=A0 =C2=A0 qemu_cond_init(cpu->halt_cond);
>> -
>> -=C2=A0 =C2=A0 snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "= CPU %d/HVF",
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu->cpu_index= );
>> -=C2=A0 =C2=A0 qemu_thread_create(cpu->thread, thread_name, hvf= _cpu_thread_fn,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0cpu, QEMU_THREAD_JOINABLE);
>> -}
>> -
>> -const CpusAccel hvf_cpus =3D {
>> -=C2=A0 =C2=A0 .create_vcpu_thread =3D hvf_start_vcpu_thread,
>> -
>> -=C2=A0 =C2=A0 .synchronize_post_reset =3D hvf_cpu_synchronize_pos= t_reset,
>> -=C2=A0 =C2=A0 .synchronize_post_init =3D hvf_cpu_synchronize_post= _init,
>> -=C2=A0 =C2=A0 .synchronize_state =3D hvf_cpu_synchronize_state, >> -=C2=A0 =C2=A0 .synchronize_pre_loadvm =3D hvf_cpu_synchronize_pre= _loadvm,
>> -};
>> diff --git a/target/i386/hvf/hvf-cpus.h b/target/i386/hvf/hvf-cpus= .h
>> deleted file mode 100644
>> index ced31b82c0..0000000000
>> --- a/target/i386/hvf/hvf-cpus.h
>> +++ /dev/null
>> @@ -1,25 +0,0 @@
>> -/*
>> - * Accelerator CPUS Interface
>> - *
>> - * Copyright 2020 SUSE LLC
>> - *
>> - * This work is licensed under the terms of the GNU GPL, version = 2 or later.
>> - * See the COPYING file in the top-level directory.
>> - */
>> -
>> -#ifndef HVF_CPUS_H
>> -#define HVF_CPUS_H
>> -
>> -#include "sysemu/cpus.h"
>> -
>> -extern const CpusAccel hvf_cpus;
>> -
>> -int hvf_init_vcpu(CPUState *);
>> -int hvf_vcpu_exec(CPUState *);
>> -void hvf_cpu_synchronize_state(CPUState *);
>> -void hvf_cpu_synchronize_post_reset(CPUState *);
>> -void hvf_cpu_synchronize_post_init(CPUState *);
>> -void hvf_cpu_synchronize_pre_loadvm(CPUState *);
>> -void hvf_vcpu_destroy(CPUState *);
>> -
>> -#endif /* HVF_CPUS_H */
>> diff --git a/target/i386/hvf/hvf-i386.h b/target/i386/hvf/hvf-i386= .h
>> index e0edffd077..6d56f8f6bb 100644
>> --- a/target/i386/hvf/hvf-i386.h
>> +++ b/target/i386/hvf/hvf-i386.h
>> @@ -18,57 +18,11 @@
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0#include "sysemu/accel.h"
>>=C2=A0 =C2=A0#include "sysemu/hvf.h"
>> +#include "sysemu/hvf_int.h"
>>=C2=A0 =C2=A0#include "cpu.h"
>>=C2=A0 =C2=A0#include "x86.h"
>>=C2=A0 =C2=A0
>> -#define HVF_MAX_VCPU 0x10
>> -
>> -extern struct hvf_state hvf_global;
>> -
>> -struct hvf_vm {
>> -=C2=A0 =C2=A0 int id;
>> -=C2=A0 =C2=A0 struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU];
>> -};
>> -
>> -struct hvf_state {
>> -=C2=A0 =C2=A0 uint32_t version;
>> -=C2=A0 =C2=A0 struct hvf_vm *vm;
>> -=C2=A0 =C2=A0 uint64_t mem_quota;
>> -};
>> -
>> -/* hvf_slot flags */
>> -#define HVF_SLOT_LOG (1 << 0)
>> -
>> -typedef struct hvf_slot {
>> -=C2=A0 =C2=A0 uint64_t start;
>> -=C2=A0 =C2=A0 uint64_t size;
>> -=C2=A0 =C2=A0 uint8_t *mem;
>> -=C2=A0 =C2=A0 int slot_id;
>> -=C2=A0 =C2=A0 uint32_t flags;
>> -=C2=A0 =C2=A0 MemoryRegion *region;
>> -} hvf_slot;
>> -
>> -typedef struct hvf_vcpu_caps {
>> -=C2=A0 =C2=A0 uint64_t vmx_cap_pinbased;
>> -=C2=A0 =C2=A0 uint64_t vmx_cap_procbased;
>> -=C2=A0 =C2=A0 uint64_t vmx_cap_procbased2;
>> -=C2=A0 =C2=A0 uint64_t vmx_cap_entry;
>> -=C2=A0 =C2=A0 uint64_t vmx_cap_exit;
>> -=C2=A0 =C2=A0 uint64_t vmx_cap_preemption_timer;
>> -} hvf_vcpu_caps;
>> -
>> -struct HVFState {
>> -=C2=A0 =C2=A0 AccelState parent;
>> -=C2=A0 =C2=A0 hvf_slot slots[32];
>> -=C2=A0 =C2=A0 int num_slots;
>> -
>> -=C2=A0 =C2=A0 hvf_vcpu_caps *hvf_caps;
>> -};
>> -extern HVFState *hvf_state;
>> -
>> -void hvf_set_phys_mem(MemoryRegionSection *, bool);
>>=C2=A0 =C2=A0void hvf_handle_io(CPUArchState *, uint16_t, void *, i= nt, int, int);
>> -hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t);
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0#ifdef NEED_CPU_H
>>=C2=A0 =C2=A0/* Functions exported to host specific mode */
>> diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
>> index ed9356565c..8b96ecd619 100644
>> --- a/target/i386/hvf/hvf.c
>> +++ b/target/i386/hvf/hvf.c
>> @@ -51,6 +51,7 @@
>>=C2=A0 =C2=A0#include "qemu/error-report.h"
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0#include "sysemu/hvf.h"
>> +#include "sysemu/hvf_int.h"
>>=C2=A0 =C2=A0#include "sysemu/runstate.h"
>>=C2=A0 =C2=A0#include "hvf-i386.h"
>>=C2=A0 =C2=A0#include "vmcs.h"
>> @@ -72,171 +73,6 @@
>>=C2=A0 =C2=A0#include "sysemu/accel.h"
>>=C2=A0 =C2=A0#include "target/i386/cpu.h"
>>=C2=A0 =C2=A0
>> -#include "hvf-cpus.h"
>> -
>> -HVFState *hvf_state;
>> -
>> -static void assert_hvf_ok(hv_return_t ret)
>> -{
>> -=C2=A0 =C2=A0 if (ret =3D=3D HV_SUCCESS) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 switch (ret) {
>> -=C2=A0 =C2=A0 case HV_ERROR:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_ERROR&qu= ot;);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 case HV_BUSY:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_BUSY&quo= t;);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 case HV_BAD_ARGUMENT:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_BAD_ARGU= MENT");
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 case HV_NO_RESOURCES:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_NO_RESOU= RCES");
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 case HV_NO_DEVICE:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_NO_DEVIC= E");
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 case HV_UNSUPPORTED:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error: HV_UNSUPPOR= TED");
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 default:
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Unknown Error"= ;);
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 abort();
>> -}
>> -
>> -/* Memory slots */
>> -hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size) >> -{
>> -=C2=A0 =C2=A0 hvf_slot *slot;
>> -=C2=A0 =C2=A0 int x;
>> -=C2=A0 =C2=A0 for (x =3D 0; x < hvf_state->num_slots; ++x) = {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 slot =3D &hvf_state->slots[x];=
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (slot->size && start &l= t; (slot->start + slot->size) &&
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (start + size) > slo= t->start) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return slot;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 return NULL;
>> -}
>> -
>> -struct mac_slot {
>> -=C2=A0 =C2=A0 int present;
>> -=C2=A0 =C2=A0 uint64_t size;
>> -=C2=A0 =C2=A0 uint64_t gpa_start;
>> -=C2=A0 =C2=A0 uint64_t gva;
>> -};
>> -
>> -struct mac_slot mac_slots[32];
>> -
>> -static int do_hvf_set_memory(hvf_slot *slot, hv_memory_flags_t fl= ags)
>> -{
>> -=C2=A0 =C2=A0 struct mac_slot *macslot;
>> -=C2=A0 =C2=A0 hv_return_t ret;
>> -
>> -=C2=A0 =C2=A0 macslot =3D &mac_slots[slot->slot_id];
>> -
>> -=C2=A0 =C2=A0 if (macslot->present) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (macslot->size !=3D slot->si= ze) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 macslot->present =3D= 0;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ret =3D hv_vm_unmap(mac= slot->gpa_start, macslot->size);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 assert_hvf_ok(ret);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 if (!slot->size) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 macslot->present =3D 1;
>> -=C2=A0 =C2=A0 macslot->gpa_start =3D slot->start;
>> -=C2=A0 =C2=A0 macslot->size =3D slot->size;
>> -=C2=A0 =C2=A0 ret =3D hv_vm_map((hv_uvaddr_t)slot->mem, slot-&= gt;start, slot->size, flags);
>> -=C2=A0 =C2=A0 assert_hvf_ok(ret);
>> -=C2=A0 =C2=A0 return 0;
>> -}
>> -
>> -void hvf_set_phys_mem(MemoryRegionSection *section, bool add)
>> -{
>> -=C2=A0 =C2=A0 hvf_slot *mem;
>> -=C2=A0 =C2=A0 MemoryRegion *area =3D section->mr;
>> -=C2=A0 =C2=A0 bool writeable =3D !area->readonly && !a= rea->rom_device;
>> -=C2=A0 =C2=A0 hv_memory_flags_t flags;
>> -
>> -=C2=A0 =C2=A0 if (!memory_region_is_ram(area)) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (writeable) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 } else if (!memory_region_is_romd(are= a)) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /*
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* If the memory d= evice is not in romd_mode, then we actually want
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* to remove the h= vf memory slot so all accesses will trap.
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0add =3D false; >> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 mem =3D hvf_find_overlap_slot(
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 section->offset_with= in_address_space,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int128_get64(section-&g= t;size));
>> -
>> -=C2=A0 =C2=A0 if (mem && add) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (mem->size =3D=3D int128_get64(= section->size) &&
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mem->start =3D=3D se= ction->offset_within_address_space &&
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mem->mem =3D=3D (mem= ory_region_get_ram_ptr(area) +
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 section->offset_with= in_region)) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return; /* Same region = was attempted to register, go away. */
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 /* Region needs to be reset. set the size to 0 and = remap it. */
>> -=C2=A0 =C2=A0 if (mem) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 mem->size =3D 0;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (do_hvf_set_memory(mem, 0)) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Fail= ed to reset overlapping slot");
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 if (!add) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 if (area->readonly ||
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 (!memory_region_is_ram(area) &&am= p; memory_region_is_romd(area))) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 flags =3D HV_MEMORY_READ | HV_MEMORY_= EXEC;
>> -=C2=A0 =C2=A0 } else {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 flags =3D HV_MEMORY_READ | HV_MEMORY_= WRITE | HV_MEMORY_EXEC;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 /* Now make a new slot. */
>> -=C2=A0 =C2=A0 int x;
>> -
>> -=C2=A0 =C2=A0 for (x =3D 0; x < hvf_state->num_slots; ++x) = {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 mem =3D &hvf_state->slots[x];<= br> >> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!mem->size) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 break;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 if (x =3D=3D hvf_state->num_slots) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("No free slots"= ;);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 mem->size =3D int128_get64(section->size); >> -=C2=A0 =C2=A0 mem->mem =3D memory_region_get_ram_ptr(area) + s= ection->offset_within_region;
>> -=C2=A0 =C2=A0 mem->start =3D section->offset_within_address= _space;
>> -=C2=A0 =C2=A0 mem->region =3D area;
>> -
>> -=C2=A0 =C2=A0 if (do_hvf_set_memory(mem, flags)) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 error_report("Error registering = new memory slot");
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 abort();
>> -=C2=A0 =C2=A0 }
>> -}
>> -
>>=C2=A0 =C2=A0void vmx_update_tpr(CPUState *cpu)
>>=C2=A0 =C2=A0{
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0/* TODO: need integrate APIC handling */=
>> @@ -276,56 +112,6 @@ void hvf_handle_io(CPUArchState *env, uint16_= t port, void *buffer,
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0}
>>=C2=A0 =C2=A0}
>>=C2=A0 =C2=A0
>> -static void do_hvf_cpu_synchronize_state(CPUState *cpu, run_on_cp= u_data arg)
>> -{
>> -=C2=A0 =C2=A0 if (!cpu->vcpu_dirty) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 hvf_get_registers(cpu);
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu->vcpu_dirty =3D true;
>> -=C2=A0 =C2=A0 }
>> -}
>> -
>> -void hvf_cpu_synchronize_state(CPUState *cpu)
>> -{
>> -=C2=A0 =C2=A0 if (!cpu->vcpu_dirty) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchroniz= e_state, RUN_ON_CPU_NULL);
>> -=C2=A0 =C2=A0 }
>> -}
>> -
>> -static void do_hvf_cpu_synchronize_post_reset(CPUState *cpu,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 run_on_cpu_data arg)
>> -{
>> -=C2=A0 =C2=A0 hvf_put_registers(cpu);
>> -=C2=A0 =C2=A0 cpu->vcpu_dirty =3D false;
>> -}
>> -
>> -void hvf_cpu_synchronize_post_reset(CPUState *cpu)
>> -{
>> -=C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchronize_post_reset, = RUN_ON_CPU_NULL);
>> -}
>> -
>> -static void do_hvf_cpu_synchronize_post_init(CPUState *cpu,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0run_on_cpu_data arg)
>> -{
>> -=C2=A0 =C2=A0 hvf_put_registers(cpu);
>> -=C2=A0 =C2=A0 cpu->vcpu_dirty =3D false;
>> -}
>> -
>> -void hvf_cpu_synchronize_post_init(CPUState *cpu)
>> -{
>> -=C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchronize_post_init, R= UN_ON_CPU_NULL);
>> -}
>> -
>> -static void do_hvf_cpu_synchronize_pre_loadvm(CPUState *cpu,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 run_on_cpu_data arg)
>> -{
>> -=C2=A0 =C2=A0 cpu->vcpu_dirty =3D true;
>> -}
>> -
>> -void hvf_cpu_synchronize_pre_loadvm(CPUState *cpu)
>> -{
>> -=C2=A0 =C2=A0 run_on_cpu(cpu, do_hvf_cpu_synchronize_pre_loadvm, = RUN_ON_CPU_NULL);
>> -}
>> -
>>=C2=A0 =C2=A0static bool ept_emulation_fault(hvf_slot *slot, uint64= _t gpa, uint64_t ept_qual)
>>=C2=A0 =C2=A0{
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0int read, write;
>> @@ -370,109 +156,19 @@ static bool ept_emulation_fault(hvf_slot *s= lot, uint64_t gpa, uint64_t ept_qual)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0return false;
>>=C2=A0 =C2=A0}
>>=C2=A0 =C2=A0
>> -static void hvf_set_dirty_tracking(MemoryRegionSection *section, = bool on)
>> -{
>> -=C2=A0 =C2=A0 hvf_slot *slot;
>> -
>> -=C2=A0 =C2=A0 slot =3D hvf_find_overlap_slot(
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 section->offset_with= in_address_space,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int128_get64(section-&g= t;size));
>> -
>> -=C2=A0 =C2=A0 /* protect region against writes; begin tracking it= */
>> -=C2=A0 =C2=A0 if (on) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 slot->flags |=3D HVF_SLOT_LOG;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 hv_vm_protect((hv_gpaddr_t)slot->s= tart, (size_t)slot->size,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 HV_MEMORY_READ);
>> -=C2=A0 =C2=A0 /* stop tracking region*/
>> -=C2=A0 =C2=A0 } else {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 slot->flags &=3D ~HVF_SLOT_LOG= ;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 hv_vm_protect((hv_gpaddr_t)slot->s= tart, (size_t)slot->size,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 HV_MEMORY_READ | HV_MEMORY_WRITE);
>> -=C2=A0 =C2=A0 }
>> -}
>> -
>> -static void hvf_log_start(MemoryListener *listener,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 MemoryRegionSection *section, int old, int new)=
>> -{
>> -=C2=A0 =C2=A0 if (old !=3D 0) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 hvf_set_dirty_tracking(section, 1);
>> -}
>> -
>> -static void hvf_log_stop(MemoryListener *listener,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section, int old, int new)<= br> >> -{
>> -=C2=A0 =C2=A0 if (new !=3D 0) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 return;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 hvf_set_dirty_tracking(section, 0);
>> -}
>> -
>> -static void hvf_log_sync(MemoryListener *listener,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section)
>> -{
>> -=C2=A0 =C2=A0 /*
>> -=C2=A0 =C2=A0 =C2=A0* sync of dirty pages is handled elsewhere; j= ust make sure we keep
>> -=C2=A0 =C2=A0 =C2=A0* tracking the region.
>> -=C2=A0 =C2=A0 =C2=A0*/
>> -=C2=A0 =C2=A0 hvf_set_dirty_tracking(section, 1);
>> -}
>> -
>> -static void hvf_region_add(MemoryListener *listener,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section)
>> -{
>> -=C2=A0 =C2=A0 hvf_set_phys_mem(section, true);
>> -}
>> -
>> -static void hvf_region_del(MemoryListener *listener,
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0MemoryRegionSection *section)
>> -{
>> -=C2=A0 =C2=A0 hvf_set_phys_mem(section, false);
>> -}
>> -
>> -static MemoryListener hvf_memory_listener =3D {
>> -=C2=A0 =C2=A0 .priority =3D 10,
>> -=C2=A0 =C2=A0 .region_add =3D hvf_region_add,
>> -=C2=A0 =C2=A0 .region_del =3D hvf_region_del,
>> -=C2=A0 =C2=A0 .log_start =3D hvf_log_start,
>> -=C2=A0 =C2=A0 .log_stop =3D hvf_log_stop,
>> -=C2=A0 =C2=A0 .log_sync =3D hvf_log_sync,
>> -};
>> -
>> -void hvf_vcpu_destroy(CPUState *cpu)
>> +void hvf_arch_vcpu_destroy(CPUState *cpu)
>>=C2=A0 =C2=A0{
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0X86CPU *x86_cpu =3D X86_CPU(cpu);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0CPUX86State *env =3D &x86_cpu->en= v;
>>=C2=A0 =C2=A0
>> -=C2=A0 =C2=A0 hv_return_t ret =3D hv_vcpu_destroy((hv_vcpuid_t)cp= u->hvf_fd);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0g_free(env->hvf_mmio_buf);
>> -=C2=A0 =C2=A0 assert_hvf_ok(ret);
>> -}
>> -
>> -static void dummy_signal(int sig)
>> -{
>>=C2=A0 =C2=A0}
>>=C2=A0 =C2=A0
>> -int hvf_init_vcpu(CPUState *cpu)
>> +int hvf_arch_init_vcpu(CPUState *cpu)
>>=C2=A0 =C2=A0{
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0X86CPU *x86cpu =3D X86_CPU(cpu);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0CPUX86State *env =3D &x86cpu->env= ;
>> -=C2=A0 =C2=A0 int r;
>> -
>> -=C2=A0 =C2=A0 /* init cpu signals */
>> -=C2=A0 =C2=A0 sigset_t set;
>> -=C2=A0 =C2=A0 struct sigaction sigact;
>> -
>> -=C2=A0 =C2=A0 memset(&sigact, 0, sizeof(sigact));
>> -=C2=A0 =C2=A0 sigact.sa_handler =3D dummy_signal;
>> -=C2=A0 =C2=A0 sigaction(SIG_IPI, &sigact, NULL);
>> -
>> -=C2=A0 =C2=A0 pthread_sigmask(SIG_BLOCK, NULL, &set);
>> -=C2=A0 =C2=A0 sigdelset(&set, SIG_IPI);
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0init_emu();
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0init_decoder();
>> @@ -480,10 +176,6 @@ int hvf_init_vcpu(CPUState *cpu)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0hvf_state->hvf_caps =3D g_new0(struct= hvf_vcpu_caps, 1);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0env->hvf_mmio_buf =3D g_new(char, 409= 6);
>>=C2=A0 =C2=A0
>> -=C2=A0 =C2=A0 r =3D hv_vcpu_create((hv_vcpuid_t *)&cpu->hv= f_fd, HV_VCPU_DEFAULT);
>> -=C2=A0 =C2=A0 cpu->vcpu_dirty =3D 1;
>> -=C2=A0 =C2=A0 assert_hvf_ok(r);
>> -
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0if (hv_vmx_read_capability(HV_VMX_CAP_PI= NBASED,
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&hvf_state->hvf_cap= s->vmx_cap_pinbased)) {
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0abort();
>> @@ -865,49 +557,3 @@ int hvf_vcpu_exec(CPUState *cpu)
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0return ret;
>>=C2=A0 =C2=A0}
>> -
>> -bool hvf_allowed;
>> -
>> -static int hvf_accel_init(MachineState *ms)
>> -{
>> -=C2=A0 =C2=A0 int x;
>> -=C2=A0 =C2=A0 hv_return_t ret;
>> -=C2=A0 =C2=A0 HVFState *s;
>> -
>> -=C2=A0 =C2=A0 ret =3D hv_vm_create(HV_VM_DEFAULT);
>> -=C2=A0 =C2=A0 assert_hvf_ok(ret);
>> -
>> -=C2=A0 =C2=A0 s =3D g_new0(HVFState, 1);
>> -
>> -=C2=A0 =C2=A0 s->num_slots =3D 32;
>> -=C2=A0 =C2=A0 for (x =3D 0; x < s->num_slots; ++x) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 s->slots[x].size =3D 0;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 s->slots[x].slot_id =3D x;
>> -=C2=A0 =C2=A0 }
>> -
>> -=C2=A0 =C2=A0 hvf_state =3D s;
>> -=C2=A0 =C2=A0 memory_listener_register(&hvf_memory_listener, = &address_space_memory);
>> -=C2=A0 =C2=A0 cpus_register_accel(&hvf_cpus);
>> -=C2=A0 =C2=A0 return 0;
>> -}
>> -
>> -static void hvf_accel_class_init(ObjectClass *oc, void *data)
>> -{
>> -=C2=A0 =C2=A0 AccelClass *ac =3D ACCEL_CLASS(oc);
>> -=C2=A0 =C2=A0 ac->name =3D "HVF";
>> -=C2=A0 =C2=A0 ac->init_machine =3D hvf_accel_init;
>> -=C2=A0 =C2=A0 ac->allowed =3D &hvf_allowed;
>> -}
>> -
>> -static const TypeInfo hvf_accel_type =3D {
>> -=C2=A0 =C2=A0 .name =3D TYPE_HVF_ACCEL,
>> -=C2=A0 =C2=A0 .parent =3D TYPE_ACCEL,
>> -=C2=A0 =C2=A0 .class_init =3D hvf_accel_class_init,
>> -};
>> -
>> -static void hvf_type_init(void)
>> -{
>> -=C2=A0 =C2=A0 type_register_static(&hvf_accel_type);
>> -}
>> -
>> -type_init(hvf_type_init);
>> diff --git a/target/i386/hvf/meson.build b/target/i386/hvf/meson.b= uild
>> index 409c9a3f14..c8a43717ee 100644
>> --- a/target/i386/hvf/meson.build
>> +++ b/target/i386/hvf/meson.build
>> @@ -1,6 +1,5 @@
>>=C2=A0 =C2=A0i386_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'],= if_true: files(
>>=C2=A0 =C2=A0 =C2=A0'hvf.c',
>> -=C2=A0 'hvf-cpus.c',
>>=C2=A0 =C2=A0 =C2=A0'x86.c',
>>=C2=A0 =C2=A0 =C2=A0'x86_cpuid.c',
>>=C2=A0 =C2=A0 =C2=A0'x86_decode.c',
>> diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c >> index bbec412b6c..89b8e9d87a 100644
>> --- a/target/i386/hvf/x86hvf.c
>> +++ b/target/i386/hvf/x86hvf.c
>> @@ -20,6 +20,9 @@
>>=C2=A0 =C2=A0#include "qemu/osdep.h"
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0#include "qemu-common.h"
>> +#include "sysemu/hvf.h"
>> +#include "sysemu/hvf_int.h"
>> +#include "sysemu/hw_accel.h"
>>=C2=A0 =C2=A0#include "x86hvf.h"
>>=C2=A0 =C2=A0#include "vmx.h"
>>=C2=A0 =C2=A0#include "vmcs.h"
>> @@ -32,8 +35,6 @@
>>=C2=A0 =C2=A0#include <Hypervisor/hv.h>
>>=C2=A0 =C2=A0#include <Hypervisor/hv_vmx.h>
>>=C2=A0 =C2=A0
>> -#include "hvf-cpus.h"
>> -
>>=C2=A0 =C2=A0void hvf_set_segment(struct CPUState *cpu, struct vmx_= segment *vmx_seg,
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 SegmentCache *qseg, bool is_tr)
>>=C2=A0 =C2=A0{
>> @@ -437,7 +438,7 @@ int hvf_process_events(CPUState *cpu_state) >>=C2=A0 =C2=A0 =C2=A0 =C2=A0env->eflags =3D rreg(cpu_state->hv= f_fd, HV_X86_RFLAGS);
>>=C2=A0 =C2=A0
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0if (cpu_state->interrupt_request &= ; CPU_INTERRUPT_INIT) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 hvf_cpu_synchronize_state(cpu_state);=
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu_synchronize_state(cpu_state);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0do_cpu_init(cpu);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0}
>>=C2=A0 =C2=A0
>> @@ -451,12 +452,12 @@ int hvf_process_events(CPUState *cpu_state)<= br> >>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu_state->halted =3D 0= ;
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0}
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0if (cpu_state->interrupt_request &= ; CPU_INTERRUPT_SIPI) {
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 hvf_cpu_synchronize_state(cpu_state);=
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu_synchronize_state(cpu_state);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0do_cpu_sipi(cpu);
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0}
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0if (cpu_state->interrupt_request &= ; CPU_INTERRUPT_TPR) {
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu_state->interrupt_re= quest &=3D ~CPU_INTERRUPT_TPR;
>> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 hvf_cpu_synchronize_state(cpu_state);=
>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu_synchronize_state(cpu_state);
> The changes from hvf_cpu_*() to cpu_*() are cleanup and perhaps should=
> be a separate patch. It follows cpu/accel cleanups Claudio was doing t= he
> summer.


The only reason they're in here is because we no longer have access to =
the hvf_ functions from the file. I am perfectly happy to rebase the
patch on top of Claudio's if his goes in first. I'm sure it'll = be
trivial for him to rebase on top of this too if my series goes in first.

>
> Phillipe raised the idea that the patch might go ahead of ARM-specific=
> part (which might involve some discussions) and I agree with that.
>
> Some sync between Claudio series (CC'd him) and the patch might be= need.


I would prefer not to hold back because of the sync. Claudio's cleanup =
is trivial enough to adjust for if it gets merged ahead of this.


Alex



--00000000000090ab2205b558ad7a--