From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B356C433E7 for ; Thu, 15 Oct 2020 13:27:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6881B22210 for ; Thu, 15 Oct 2020 13:27:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DIeXKXbj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6881B22210 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40384 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kT3Hw-0001B5-D8 for qemu-devel@archiver.kernel.org; Thu, 15 Oct 2020 09:27:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kT3HI-0000Yv-3B for qemu-devel@nongnu.org; Thu, 15 Oct 2020 09:26:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47557) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kT3HE-00078j-Qt for qemu-devel@nongnu.org; Thu, 15 Oct 2020 09:26:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602768399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9+++Y27GlKLWEVEq2S9REbyh2iJra991JQ1AQaWmzEM=; b=DIeXKXbjDAbvPOq5zxf45Z50x4Hg5BuQZASzchn4HvDDJLJlwJaTIiLZtARzgSG30RFHAx cxLrqeFLjQMAbkfBvBO5USml9RUcorMN4KBJQgJCJ4DA6lh1Nx5yUONGnX4QE3HSszf640 GarXQ83X7Z7Pxvtz2/y/eYY+HxQVnKk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-339-55PDe9xzPA6jCYTWpz_o8A-1; Thu, 15 Oct 2020 09:26:37 -0400 X-MC-Unique: 55PDe9xzPA6jCYTWpz_o8A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 815F710AC943; Thu, 15 Oct 2020 13:26:35 +0000 (UTC) Received: from kamzik.brq.redhat.com (unknown [10.40.194.132]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9642B76674; Thu, 15 Oct 2020 13:26:32 +0000 (UTC) Date: Thu, 15 Oct 2020 15:26:29 +0200 From: Andrew Jones To: =?utf-8?B?5byg5Lic5pet?= Subject: Re: Kernel patch cases qemu live migration failed. Message-ID: <20201015132629.n4k5k7wdruclcrev@kamzik.brq.redhat.com> References: <20201015040639.70C6D414009F@webmail.sinamail.sina.com.cn> MIME-Version: 1.0 In-Reply-To: <20201015040639.70C6D414009F@webmail.sinamail.sina.com.cn> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=drjones@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=216.205.24.124; envelope-from=drjones@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/15 02:38:26 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "marc.zyngier" , qemu-devel , "Dave.Martin" , kvmarm Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Thu, Oct 15, 2020 at 12:06:39PM +0800, 张东旭 wrote: > I'm so sorry for disturbing you.When I apply this kernel patch：KVM: arm64/sve: System register context switch and access supporthttps://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/arm64/kvm/sys_regs.c?id=73433762fcaeb9d59e84d299021c6b15466c96ddqemu live migration will failed with messages： ​qemu-kvm: Invalid value 233 expecting positive value <= 232 qemu-kvm: Failed to load cpu:cpreg_vmstate_array_lenNew version kernel exclude ID_AA64ZFR0_EL1 register when host not support SVE feature, so qemu ioctl kvm(KVM_GET_REG_LIST) will not contain ID_AA64ZFR0_EL1 register. > I'm using CentOS Linux kernel, old kernel version：4.18.0-80.11.el8 (migration source) new kernel version：4.18.0-147.5.el8 (migration destination) > CentOS linux kernel version 4.18.0-111.el8 applied this patch. So 4.18.0-147.5.el8 also applied this patch. > Migration source and destination hosts have the same hardware, just kernel version is different, and the hardware on either side of the migration not support SVE. > Is there some good suggestions，which can make sure old version kernel live migration to new version kernel with qemu?Thanks a lot. ARM KVM guests must be run with CPU host passthrough. This means the host hardware and host kernel versions must be identical in order to guarantee successful migrations. That said, upgrading the host kernel without shutting down the guests is a reasonable thing to try. Unfortunately, in this case, the only way to do it would be to hack QEMU on the destination to allow this extra register in KVM_GET_REG_LIST. It should be harmless, as it's not used. Allowing it is similar to allowing the destination to have a larger number of registers than the source in KVM_GET_REG_LIST (which is why the failing test is <=, not ==). I wouldn't post a real patch to resolve this issue, though, as everything is working as expected. The failing test is failing because it detected a risky migration. And, KVM's filtering of registers from KVM_GET_REG_LIST is also correct, even if previous KVM versions didn't do that. With CPU models you could request the reg list for a particular model and expect it to be the same across all kernel versions that support it, but we don't have those. Thanks, drew From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5EF9C433DF for ; Thu, 15 Oct 2020 13:26:47 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id D2C5B2222B for ; Thu, 15 Oct 2020 13:26:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MWa9nRsS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D2C5B2222B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id CD7D64B665; Thu, 15 Oct 2020 09:26:45 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@redhat.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GTjxH9mGHeEn; Thu, 15 Oct 2020 09:26:44 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id B16554B6ED; Thu, 15 Oct 2020 09:26:44 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C1DE24B665 for ; Thu, 15 Oct 2020 09:26:43 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 05LtOPs1xbEP for ; Thu, 15 Oct 2020 09:26:41 -0400 (EDT) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mm01.cs.columbia.edu (Postfix) with ESMTP id CC4164B134 for ; Thu, 15 Oct 2020 09:26:41 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602768401; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9+++Y27GlKLWEVEq2S9REbyh2iJra991JQ1AQaWmzEM=; b=MWa9nRsS+XsgH0zmAOLGutvEn2hYoCZa9lbCv5y3G4BCRVhUUScGikWNMFyOo2w1nValFG Hyyr5/BPkcGX2oBzDLGuB6X8vOwmOGQ2O8+A6dl+2yreHme2j8dAvW8q4k0c0O6bdEYkXL t2YhpB4GbT5DDp/izM+3SYPw6yZrBhE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-339-55PDe9xzPA6jCYTWpz_o8A-1; Thu, 15 Oct 2020 09:26:37 -0400 X-MC-Unique: 55PDe9xzPA6jCYTWpz_o8A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 815F710AC943; Thu, 15 Oct 2020 13:26:35 +0000 (UTC) Received: from kamzik.brq.redhat.com (unknown [10.40.194.132]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9642B76674; Thu, 15 Oct 2020 13:26:32 +0000 (UTC) Date: Thu, 15 Oct 2020 15:26:29 +0200 From: Andrew Jones To: =?utf-8?B?5byg5Lic5pet?= Subject: Re: Kernel patch cases qemu live migration failed. Message-ID: <20201015132629.n4k5k7wdruclcrev@kamzik.brq.redhat.com> References: <20201015040639.70C6D414009F@webmail.sinamail.sina.com.cn> MIME-Version: 1.0 In-Reply-To: <20201015040639.70C6D414009F@webmail.sinamail.sina.com.cn> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=drjones@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: "marc.zyngier" , qemu-devel , "Dave.Martin" , kvmarm X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu T24gVGh1LCBPY3QgMTUsIDIwMjAgYXQgMTI6MDY6MzlQTSArMDgwMCwg5byg5Lic5petIHdyb3Rl Ogo+IEknbSBzbyBzb3JyeSBmb3IgZGlzdHVyYmluZyB5b3UuV2hlbiBJIGFwcGx5IHRoaXMga2Vy bmVsIHBhdGNo77yaS1ZNOiBhcm02NC9zdmU6IFN5c3RlbSByZWdpc3RlciBjb250ZXh0IHN3aXRj aCBhbmQgYWNjZXNzIHN1cHBvcnRodHRwczovL2dpdC5rZXJuZWwub3JnL3B1Yi9zY20vbGludXgv a2VybmVsL2dpdC90b3J2YWxkcy9saW51eC5naXQvY29tbWl0L2FyY2gvYXJtNjQva3ZtL3N5c19y ZWdzLmM/aWQ9NzM0MzM3NjJmY2FlYjlkNTllODRkMjk5MDIxYzZiMTU0NjZjOTZkZHFlbXUgbGl2 ZSBtaWdyYXRpb24gd2lsbCBmYWlsZWQgd2l0aCBtZXNzYWdlc++8miAgICAgICAgICAgICAg4oCL cWVtdS1rdm06IEludmFsaWQgdmFsdWUgMjMzIGV4cGVjdGluZyBwb3NpdGl2ZSB2YWx1ZSA8PSAy MzIgICAgICAgICAgICAgIHFlbXUta3ZtOiBGYWlsZWQgdG8gbG9hZCBjcHU6Y3ByZWdfdm1zdGF0 ZV9hcnJheV9sZW5OZXcgdmVyc2lvbiBrZXJuZWwgIGV4Y2x1ZGUgSURfQUE2NFpGUjBfRUwxIHJl Z2lzdGVyIHdoZW4gaG9zdCBub3Qgc3VwcG9ydCBTVkUgZmVhdHVyZSwgc28gcWVtdSBpb2N0bCBr dm0oS1ZNX0dFVF9SRUdfTElTVCkgd2lsbCBub3QgY29udGFpbiBJRF9BQTY0WkZSMF9FTDEgcmVn aXN0ZXIuCj4gSSdtIHVzaW5nIENlbnRPUyBMaW51eCBrZXJuZWwsIG9sZCBrZXJuZWwgdmVyc2lv bu+8mjQuMTguMC04MC4xMS5lbDggKG1pZ3JhdGlvbiBzb3VyY2UpCQkJCQkgICAgIG5ldyBrZXJu ZWwgdmVyc2lvbu+8mjQuMTguMC0xNDcuNS5lbDggKG1pZ3JhdGlvbiBkZXN0aW5hdGlvbikKPiBD ZW50T1MgbGludXgga2VybmVsIHZlcnNpb24gNC4xOC4wLTExMS5lbDggYXBwbGllZCB0aGlzIHBh dGNoLiBTbyA0LjE4LjAtMTQ3LjUuZWw4IGFsc28gYXBwbGllZCB0aGlzIHBhdGNoLgo+IE1pZ3Jh dGlvbiBzb3VyY2UgYW5kIGRlc3RpbmF0aW9uIGhvc3RzIGhhdmUgdGhlIHNhbWUgaGFyZHdhcmUs IGp1c3Qga2VybmVsIHZlcnNpb24gaXMgZGlmZmVyZW50LCBhbmQgdGhlIGhhcmR3YXJlIG9uIGVp dGhlciBzaWRlIG9mIHRoZSBtaWdyYXRpb24gbm90IHN1cHBvcnQgU1ZFLgo+ICBJcyB0aGVyZSBz b21lIGdvb2Qgc3VnZ2VzdGlvbnPvvIx3aGljaCBjYW4gbWFrZSBzdXJlIG9sZCB2ZXJzaW9uIGtl cm5lbCBsaXZlIG1pZ3JhdGlvbiB0byBuZXcgdmVyc2lvbiBrZXJuZWwgd2l0aCBxZW11P1RoYW5r cyBhIGxvdC4KCkFSTSBLVk0gZ3Vlc3RzIG11c3QgYmUgcnVuIHdpdGggQ1BVIGhvc3QgcGFzc3Ro cm91Z2guIFRoaXMgbWVhbnMgdGhlCmhvc3QgaGFyZHdhcmUgYW5kIGhvc3Qga2VybmVsIHZlcnNp b25zIG11c3QgYmUgaWRlbnRpY2FsIGluIG9yZGVyCnRvIGd1YXJhbnRlZSBzdWNjZXNzZnVsIG1p Z3JhdGlvbnMuCgpUaGF0IHNhaWQsIHVwZ3JhZGluZyB0aGUgaG9zdCBrZXJuZWwgd2l0aG91dCBz aHV0dGluZyBkb3duIHRoZSBndWVzdHMKaXMgYSByZWFzb25hYmxlIHRoaW5nIHRvIHRyeS4gVW5m b3J0dW5hdGVseSwgaW4gdGhpcyBjYXNlLCB0aGUgb25seQp3YXkgdG8gZG8gaXQgd291bGQgYmUg dG8gaGFjayBRRU1VIG9uIHRoZSBkZXN0aW5hdGlvbiB0byBhbGxvdyB0aGlzCmV4dHJhIHJlZ2lz dGVyIGluIEtWTV9HRVRfUkVHX0xJU1QuIEl0IHNob3VsZCBiZSBoYXJtbGVzcywgYXMgaXQncwpu b3QgdXNlZC4gQWxsb3dpbmcgaXQgaXMgc2ltaWxhciB0byBhbGxvd2luZyB0aGUgZGVzdGluYXRp b24gdG8gaGF2ZQphIGxhcmdlciBudW1iZXIgb2YgcmVnaXN0ZXJzIHRoYW4gdGhlIHNvdXJjZSBp biBLVk1fR0VUX1JFR19MSVNUICh3aGljaAppcyB3aHkgdGhlIGZhaWxpbmcgdGVzdCBpcyA8PSwg bm90ID09KS4KCkkgd291bGRuJ3QgcG9zdCBhIHJlYWwgcGF0Y2ggdG8gcmVzb2x2ZSB0aGlzIGlz c3VlLCB0aG91Z2gsIGFzIGV2ZXJ5dGhpbmcKaXMgd29ya2luZyBhcyBleHBlY3RlZC4gVGhlIGZh aWxpbmcgdGVzdCBpcyBmYWlsaW5nIGJlY2F1c2UgaXQgZGV0ZWN0ZWQKYSByaXNreSBtaWdyYXRp b24uIEFuZCwgS1ZNJ3MgZmlsdGVyaW5nIG9mIHJlZ2lzdGVycyBmcm9tIEtWTV9HRVRfUkVHX0xJ U1QKaXMgYWxzbyBjb3JyZWN0LCBldmVuIGlmIHByZXZpb3VzIEtWTSB2ZXJzaW9ucyBkaWRuJ3Qg ZG8gdGhhdC4gV2l0aCBDUFUKbW9kZWxzIHlvdSBjb3VsZCByZXF1ZXN0IHRoZSByZWcgbGlzdCBm b3IgYSBwYXJ0aWN1bGFyIG1vZGVsIGFuZCBleHBlY3QKaXQgdG8gYmUgdGhlIHNhbWUgYWNyb3Nz IGFsbCBrZXJuZWwgdmVyc2lvbnMgdGhhdCBzdXBwb3J0IGl0LCBidXQgd2UKZG9uJ3QgaGF2ZSB0 aG9zZS4KClRoYW5rcywKZHJldwoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18Ka3ZtYXJtIG1haWxpbmcgbGlzdAprdm1hcm1AbGlzdHMuY3MuY29sdW1iaWEu ZWR1Cmh0dHBzOi8vbGlzdHMuY3MuY29sdW1iaWEuZWR1L21haWxtYW4vbGlzdGluZm8va3ZtYXJt Cg==