From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752770AbdJDNUK (ORCPT ); Wed, 4 Oct 2017 09:20:10 -0400 Received: from mail-cys01nam02on0084.outbound.protection.outlook.com ([104.47.37.84]:31550 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752375AbdJDNPW (ORCPT ); Wed, 4 Oct 2017 09:15:22 -0400 From: Brijesh Singh To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Brijesh Singh , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Joerg Roedel , Borislav Petkov , Tom Lendacky Subject: [Part2 PATCH v5 18/31] KVM: SVM: VMRUN should use assosiated ASID when SEV is enabled Date: Wed, 4 Oct 2017 08:13:59 -0500 Message-Id: <20171004131412.13038-19-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171004131412.13038-1-brijesh.singh@amd.com> References: <20171004131412.13038-1-brijesh.singh@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: MWHPR2201CA0056.namprd22.prod.outlook.com (10.172.59.30) To SN1PR12MB0160.namprd12.prod.outlook.com (10.162.3.147) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9d640366-1dd3-459e-1de5-08d50b29f229 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:SN1PR12MB0160; X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;3:KD6+vt9sxuUgqulS9qaMrvFgy4DHOL96qMP4yu8tzYtSHTkPOVBOuYVzfEleiMC/xRWhM0JfcpqPA5haJgPd2rDTCYiM4qQUZp0m3XI6AbH33QrGmpC/dv3tWnUPjeG/bGoCdIYUiWR5rX8TwVFJ9A+dFzCsa3Bk0vWjeqYUV4DyHFKuwAHMM8DEyiIQ5mjETKui6QdESGBsPdiCwKWUN3CcbmNvs8ucW1JcnTO2vQAZyQf0r15etI0xCinWLCwQ;25:YADYIJUC+JJ+O0f6jIadNKellTb8kWxCoJ0XOBChvVTQtpUUbodzFNZUInhEbsx9hxYNeUVFkywG3ZpQnABb26pI1dpQDm872YSwAwXS7Fo2RWwouNN2fsqU7iBK8ii7pUX9sZ8iPbipEcE54FFEiADW9e4yjG5DwdW2TEtOHVI9yCdafiastcLUgcx6dX36LSLy5EcUBGxIVA6V2dzKuhoHzSG1WV5TfRjbRBQcnUjlbkTi0xn1RHJiDPDAqEFTNklIM/TxPMPq6Hmz5HuHc19nIkQ21bvkPVTWTATTMxfesqmik7VfR+35LWQN2iqPImCvXYYpDbmgKZ9I8Ue3Xw==;31:gS0aWhGbutSIC3EYLlpvXipmoKUlUaoI2ZFne2YEUOqzh3Lqq2KCsw7aBssrtlYLxSPXzhtXiOlHh0Nrtlwqt5/DvGL7Joq7rLH6U3O9jYmYeCPNzp23NrzYcMKo2qa+l8M1GA+/uaXeO+OSPRSz5Ypf3El6TXdcmEG+Wr6MC1oDVAQ3+3xJfaoyF05oMlS72cVDYrE3+gVX+d6pt9dMrrViUMiHsQCPSlCstCaFVDY= X-MS-TrafficTypeDiagnostic: SN1PR12MB0160: X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;20:FimuKvsGuF0saJbuJk876CKul+495SMWbsx8D+rhEGsFmgVnn56Taybw1AYkyNPSr98Xd9EYKXZMY5DQc3OPktAYZHRGKUV48PefN3knB3326rWW7gILySd0jOrwvJ5e+KwZnjsAFIo0AcLhk0Ha29n0y88q/KIoH1DseAoJ47/LYf7R/aZ3Q/VKqWig5NZOxany7/umuORBvdsBaRwztb135tLzQssZHv1tHCFp1TXXu9i3Fn76NNxeqnBDqxudVCLIptfyCrwcPGYCfZITZsb0obzJbMLmlKd4HrNw+je5BQwb4BjKRSpytvro30S0ItVKcxa1dkcYMtOXDBtZKl1HwCztZFV6kVp4I7kLdLO0r5Ff2ws3gk/q3Ev5/79Eiz/NFmUhQrDLYodx6mM4FtAjWx40nkpnOMKZYrwSsLzQjxIG1Sk/wfAljGg5TwXaa53HFVMZX88cCDdDmpr9ZqYFFPNu+05OKXIvDPHmkxtEngnWNlyxutSgsUj4O3Id;4:GjjtBDGlP9svNoneWe+9wP9kuX2BYuLdt6ocKkduYesFkzosZoAWLG6ZkLQQsEH+DAoF8aYpnCkRArJLcqTTjkLhdHcwozSV7bweB4Ja0KhoE7T+nI9nBq8VqlIs7MD1QUCM73wh5q6jIlmY6hJ/aNaMNrXVQM8B6QVHBJOuUZh5yPTfSJFnIMCV4qSuJhmqCYPXrRiRVvyF7+dDzz41WEexOj+xnTrWWYa/g8sfPp1EaA+GznRZZTaEClvMwhEYIeo8/dxBamCU65bxQCw3A0393SGXDAaV0Fig9YVeIqBsRtlA5j+xNToNcT3qMS5R X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(3002001)(93006095)(93001095)(10201501046)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123562025)(20161123555025)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:SN1PR12MB0160;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:SN1PR12MB0160; X-Forefront-PRVS: 0450A714CB X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(39860400002)(346002)(376002)(199003)(189002)(97736004)(86362001)(23676002)(76176999)(53936002)(50986999)(1076002)(6116002)(3846002)(478600001)(36756003)(47776003)(101416001)(33646002)(6486002)(66066001)(189998001)(68736007)(105586002)(53416004)(50466002)(4326008)(54906003)(8936002)(305945005)(6666003)(2870700001)(81166006)(5660300001)(81156014)(25786009)(2906002)(8676002)(7736002)(2950100002)(7416002)(50226002)(106356001)(316002)(16526018);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR12MB0160;H:ubuntu-010236106000.amd.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtTTjFQUjEyTUIwMTYwOzIzOkdERXlVa2hjQkIzU1Q0WEExdDNGV0pPNzRw?= =?utf-8?B?T3c1V1ZlSG1UREpCMzJ3QTJDVTJMQTNXakY0WmoxTy9WYjZyM0RLODRRWXNN?= =?utf-8?B?bDBuVitrZTZPZzBzSXNtN0hzWW9zR1BZQXdWSVM5KzlONDhFN0RRUkpmSThS?= =?utf-8?B?MjRzSWROT3owTWFNWWlPcE9scEx1RUp1ZjEzOFB6Z1lGTlA2bW9ieFEwc2Ri?= =?utf-8?B?USszRzNWdnM0aVE5dFo2T05EY3hsNE1GcGVpbGl1MldLTVNRZ1J6c3oxK1pv?= =?utf-8?B?NXRKNDBaQTBPdWdVL1ZDaTAwdmI5TWRoYlRUOGcvZDkrVnhnVmFDc1Q4N2h4?= =?utf-8?B?OHc3RDhqNWNtOFhEY3pQU1krLzFKbjFrRHFhaE85ZkJOTmtRaE8xRUhFbEJB?= =?utf-8?B?eEpqd0RUQ2V5aGlnRVdySkFsZGk1ZjhacTkxM3E5OXo1ODE0NGVld2xFc3py?= =?utf-8?B?RzEzdUxlcUJpdVk4UDVhLzIva1ROVEkyU1dVcElQdnMvVmdIbXRzR2RNcVFj?= =?utf-8?B?YnJrTkVISGlNV21CNGJ4cmUzTmMvVkVsWEdPLzM3bWQrM1p6alY1dDdMQTA2?= =?utf-8?B?NXVOOHFzU1lpclREZVEvWXY1RVd4Q1dhUkI0WldvV1ZCS1M5OEdsMVJtblpY?= =?utf-8?B?Vkt3dWw0K2lYbG5LWk9EUThURFY4VDhvWmVDeGwwcThOSXM0ZVoyMW55MU5R?= =?utf-8?B?SjhIeVlPN3lsRUlOQTd3dVhaU2k2dnJ1QTkxSTFFem53RENuMGJTY0tiYXFv?= =?utf-8?B?c3RjZWt3UEhMZE5EVE1QY2xTWTlNTHhFeHVBZllxN0Nqak1kSElYUGQza2xN?= =?utf-8?B?aGNZSVpnVjdCQ1ZSV2VTRnk4Y1FnRm5KU0hOeXpTcThJT01RMXh1R1RSVFdm?= =?utf-8?B?VzRodVdjZFh1VkFINW9OekhsZ3B0N1B6cUs4RzRYUEpNM3pOT3ZreVI0Rzk4?= =?utf-8?B?RWhwcHZVS0IxL0lTcnh5TXM5SHE5QkZyT2p3dHNHYVhDREVmVlkrbEF4dGFr?= =?utf-8?B?dEdCd1NIY3hZcXlZcWJYdzlnbkVRV044MCsxcnNIR3g1UEN2TE1OOEZucFdx?= =?utf-8?B?QmY0Z2F2TVNwS2YvOWpxcHl3T2U2UFFyUWZ3VXNUaFVISktVSlpHRG96M1Vx?= =?utf-8?B?MUt3N1dEMWFEMDJGSnZ5cVpsSGQ2SGQ1UWVSNjl5MU9WOGg0U2wzeUhoeUxh?= =?utf-8?B?ck9IT2NTVThlSk51Ry9ybkNyV0NEcFRGTDdGWmpFaWNlUmg5aEVERE9xb2xQ?= =?utf-8?B?VzduTDlET05INytWN2tZM2FzdS9IMm5hZFF1SVh3MlNIRDAwTUcrSlB1TlFk?= =?utf-8?B?cnBEcEo1UnRaeHVqa3RvQS9SdWZDZmtJQTN0OTVGaWl2SWlvY0Z2ZWhzTWFr?= =?utf-8?B?TzNwYXk4QUFSQTI1dWI0QWpWL08vUVNSblJkS0ZlTVU0aFJrVFZqQmVGQ3NQ?= =?utf-8?Q?uFGfmj9AMSAOsuUR3Fui6sDJguD?= X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;6:umaEySUXCG3AeSYFfOQl+Exe+KBqrKooD9LejuouXbh7Ncs6NzuybJ7bHYSoM2LCECsuLWiRMZ6S0gSCXFKGWBeurNrg7iVQm2yKJpyVOwt8HfybGzKaj7sNM4IkpudFHcjq5U2IcXC0y14fG0pGcFnRwXpH2fT0l55RkuNHOqxKXe+pbkHnIobmzHr4jg6mm3kA2WoxEeH4ALF6NWPnM094hidKpkxAxWovs3FlBa6ZLrjF0wpRtDhKAre6p+japi/0kkrCpheXWpaiyibI/TAKEdespxgAlzleuMGTkx07AF7ks72beWsJSmvY20lV4I+E8gg4UVp00thsBhDDrA==;5:JBO3tC9xAe6PK5FNJ0LF1iJffvzIhcxlYX6JqzGNcb1aNfajX2znlk4RVEsCP+6mchk6fQRaWXw7oWDiKJ9FrHtQHIRfRC/eI3H2X6egmGhIbo5zdNZdtkMCtzfeaHcZdfetZyiyTO+AuP/5xrh6kw==;24:kAr0JabGtinNYBysQhPJm5q6JtXlrnZ45+OBP58dN21U5kA+Zoc+UVbrlpK+zQs+M6stawhYy1An2KwfKmwtGNIEf4EdHgNs3Of6qKENM/o=;7:3Byc3ikpQpUWsDK3po81ZMvoKMye60Vi+gukq+ccWxTTVf+AX5+UgIx4SSwD+ktcRn7TV6OyzemFM5+XgAKK2UP/rEAFi7a7loF5iE+vfUf9mCExz1L5qnQXkk+hNhVnfGJbWrNjGQFL0HMFc7tcT6GaOrEkFLxsvhF96YB6iDEp13XvGCg/qDq8cmh7gRM1jZP9L6al4wVeIVcX4vtRS9Tsb4GKFrBYC6FcWXwtDns= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;20:KmOsa1M4aUb2ATTCfwfCfSTj67dCeT4lH4EnDT2pl63nAzhoHtk72kdVM0fs095DpY0LDwvzVQgYwsXVrN6WRJglxWwWOT2ljug22tjt5hhHn7+qWp0KUzcES5UoueZeHp7+svngmSSmFWbuwRYPdcgBu93JW6tWZdubfG2Vk05k7rruNF93iurOj88d78PndLGBus5sNTOeaqgmNFrQ5VqrLGyQOa3+Q/Qm2gEYXWc/diCE11bI51Hfgo0bRa// X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Oct 2017 13:15:11.1357 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0160 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org SEV hardware uses ASIDs to associate memory encryption key with the guest VM. During the guest creation time, SEV VM use SEV_CMD_ACTIVATE command to bind a particular ASID to the guest. Lets make sure that the VMCB is programmed with the bound ASID before a VMRUN. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Joerg Roedel Cc: Borislav Petkov Cc: Tom Lendacky Cc: x86@kernel.org Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Brijesh Singh --- arch/x86/kvm/svm.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 7f3179555a57..a1388e74149f 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -213,6 +213,9 @@ struct vcpu_svm { */ struct list_head ir_list; spinlock_t ir_list_lock; + + /* which host CPU was used for running this vcpu */ + unsigned int last_cpu; }; /* @@ -340,6 +343,13 @@ static inline bool sev_guest(struct kvm *kvm) return sev->active; } +static inline int sev_get_asid(struct kvm *kvm) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + + return sev->asid; +} + static inline void mark_all_dirty(struct vmcb *vmcb) { vmcb->control.clean = 0; @@ -550,6 +560,9 @@ struct svm_cpu_data { struct kvm_ldttss_desc *tss_desc; struct page *save_area; + + /* index = sev_asid, value = vmcb pointer */ + struct vmcb **sev_vmcbs; }; static DEFINE_PER_CPU(struct svm_cpu_data *, svm_data); @@ -863,6 +876,7 @@ static void svm_cpu_uninit(int cpu) return; per_cpu(svm_data, raw_smp_processor_id()) = NULL; + kfree(sd->sev_vmcbs); __free_page(sd->save_area); kfree(sd); } @@ -876,11 +890,18 @@ static int svm_cpu_init(int cpu) if (!sd) return -ENOMEM; sd->cpu = cpu; - sd->save_area = alloc_page(GFP_KERNEL); r = -ENOMEM; + sd->save_area = alloc_page(GFP_KERNEL); if (!sd->save_area) goto err_1; + if (svm_sev_enabled()) { + r = -ENOMEM; + sd->sev_vmcbs = kmalloc((max_sev_asid + 1) * sizeof(void *), GFP_KERNEL); + if (!sd->sev_vmcbs) + goto err_1; + } + per_cpu(svm_data, cpu) = sd; return 0; @@ -1514,7 +1535,8 @@ static void sev_firmware_exit(void) static void sev_asid_free(struct kvm *kvm) { struct kvm_sev_info *sev = &kvm->arch.sev_info; - int pos, asid; + struct svm_cpu_data *sd; + int pos, asid, cpu; if (!svm_sev_enabled()) return; @@ -1522,6 +1544,11 @@ static void sev_asid_free(struct kvm *kvm) asid = sev->asid; pos = asid - 1; clear_bit(pos, sev_asid_bitmap); + + for_each_possible_cpu(cpu) { + sd = per_cpu(svm_data, cpu); + sd->sev_vmcbs[pos] = NULL; + } } static void sev_vm_destroy(struct kvm *kvm) @@ -4456,12 +4483,39 @@ static void reload_tss(struct kvm_vcpu *vcpu) load_TR_desc(); } +static void pre_sev_run(struct vcpu_svm *svm, int cpu) +{ + struct svm_cpu_data *sd = per_cpu(svm_data, cpu); + int asid = sev_get_asid(svm->vcpu.kvm); + + /* Assign the asid allocated with this SEV guest */ + svm->vmcb->control.asid = asid; + + /* + * Flush guest TLB: + * + * 1) when different VMCB for the same ASID is to be run on the same host CPU. + * 2) or this VMCB was executed on different host CPU in previous VMRUNs. + */ + if (sd->sev_vmcbs[asid] == svm->vmcb && + svm->last_cpu == cpu) + return; + + svm->last_cpu = cpu; + sd->sev_vmcbs[asid] = svm->vmcb; + svm->vmcb->control.tlb_ctl = TLB_CONTROL_FLUSH_ASID; + mark_dirty(svm->vmcb, VMCB_ASID); +} + static void pre_svm_run(struct vcpu_svm *svm) { int cpu = raw_smp_processor_id(); struct svm_cpu_data *sd = per_cpu(svm_data, cpu); + if (sev_guest(svm->vcpu.kvm)) + return pre_sev_run(svm, cpu); + /* FIXME: handle wraparound of asid_generation */ if (svm->asid_generation != sd->asid_generation) new_asid(svm, sd); -- 2.9.5