From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDF18C28CBC for ; Wed, 6 May 2020 06:08:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB9F2206E6 for ; Wed, 6 May 2020 06:08:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727028AbgEFGIg (ORCPT ); Wed, 6 May 2020 02:08:36 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:10520 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725873AbgEFGIf (ORCPT ); Wed, 6 May 2020 02:08:35 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0466250q018593; Wed, 6 May 2020 02:08:34 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 30s1sxqvc4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 May 2020 02:08:34 -0400 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 04662Vwp020178; Wed, 6 May 2020 02:08:33 -0400 Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com [149.81.74.107]) by mx0b-001b2d01.pphosted.com with ESMTP id 30s1sxqvb3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 May 2020 02:08:33 -0400 Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1]) by ppma03fra.de.ibm.com (8.16.0.27/8.16.0.27) with SMTP id 0465pFdA027513; Wed, 6 May 2020 06:08:32 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma03fra.de.ibm.com with ESMTP id 30s0g5ke1x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 06 May 2020 06:08:32 +0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 04668TQM8978942 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 May 2020 06:08:29 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E566952050; Wed, 6 May 2020 06:08:28 +0000 (GMT) Received: from oc7455500831.ibm.com (unknown [9.145.91.11]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 787D152059; Wed, 6 May 2020 06:08:28 +0000 (GMT) Subject: Re: [PATCH] KVM: s390: Remove false WARN_ON_ONCE for the PQAP instruction To: Tony Krowiak , Cornelia Huck Cc: Janosch Frank , KVM , David Hildenbrand , linux-s390 , Qian Cai , Pierre Morel References: <20200505073525.2287-1-borntraeger@de.ibm.com> <20200505095332.528254e5.cohuck@redhat.com> <889a7e3d-8318-4c85-67c8-a42a665b56f4@linux.ibm.com> From: Christian Borntraeger Autocrypt: addr=borntraeger@de.ibm.com; prefer-encrypt=mutual; keydata= xsFNBE6cPPgBEAC2VpALY0UJjGmgAmavkL/iAdqul2/F9ONz42K6NrwmT+SI9CylKHIX+fdf J34pLNJDmDVEdeb+brtpwC9JEZOLVE0nb+SR83CsAINJYKG3V1b3Kfs0hydseYKsBYqJTN2j CmUXDYq9J7uOyQQ7TNVoQejmpp5ifR4EzwIFfmYDekxRVZDJygD0wL/EzUr8Je3/j548NLyL 4Uhv6CIPf3TY3/aLVKXdxz/ntbLgMcfZsDoHgDk3lY3r1iwbWwEM2+eYRdSZaR4VD+JRD7p8 0FBadNwWnBce1fmQp3EklodGi5y7TNZ/CKdJ+jRPAAnw7SINhSd7PhJMruDAJaUlbYaIm23A +82g+IGe4z9tRGQ9TAflezVMhT5J3ccu6cpIjjvwDlbxucSmtVi5VtPAMTLmfjYp7VY2Tgr+ T92v7+V96jAfE3Zy2nq52e8RDdUo/F6faxcumdl+aLhhKLXgrozpoe2nL0Nyc2uqFjkjwXXI OBQiaqGeWtxeKJP+O8MIpjyGuHUGzvjNx5S/592TQO3phpT5IFWfMgbu4OreZ9yekDhf7Cvn /fkYsiLDz9W6Clihd/xlpm79+jlhm4E3xBPiQOPCZowmHjx57mXVAypOP2Eu+i2nyQrkapaY IdisDQfWPdNeHNOiPnPS3+GhVlPcqSJAIWnuO7Ofw1ZVOyg/jwARAQABzUNDaHJpc3RpYW4g Qm9ybnRyYWVnZXIgKDJuZCBJQk0gYWRkcmVzcykgPGJvcm50cmFlZ2VyQGxpbnV4LmlibS5j b20+wsF5BBMBAgAjBQJdP/hMAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AACgkQEXu8 gLWmHHy/pA/+JHjpEnd01A0CCyfVnb5fmcOlQ0LdmoKWLWPvU840q65HycCBFTt6V62cDljB kXFFxMNA4y/2wqU0H5/CiL963y3gWIiJsZa4ent+KrHl5GK1nIgbbesfJyA7JqlB0w/E/SuY NRQwIWOo/uEvOgXnk/7+rtvBzNaPGoGiiV1LZzeaxBVWrqLtmdi1iulW/0X/AlQPuF9dD1Px hx+0mPjZ8ClLpdSp5d0yfpwgHtM1B7KMuQPQZGFKMXXTUd3ceBUGGczsgIMipZWJukqMJiJj QIMH0IN7XYErEnhf0GCxJ3xAn/J7iFpPFv8sFZTvukntJXSUssONnwiKuld6ttUaFhSuSoQg OFYR5v7pOfinM0FcScPKTkrRsB5iUvpdthLq5qgwdQjmyINt3cb+5aSvBX2nNN135oGOtlb5 tf4dh00kUR8XFHRrFxXx4Dbaw4PKgV3QLIHKEENlqnthH5t0tahDygQPnSucuXbVQEcDZaL9 WgJqlRAAj0pG8M6JNU5+2ftTFXoTcoIUbb0KTOibaO9zHVeGegwAvPLLNlKHiHXcgLX1tkjC DrvE2Z0e2/4q7wgZgn1kbvz7ZHQZB76OM2mjkFu7QNHlRJ2VXJA8tMXyTgBX6kq1cYMmd/Hl OhFrAU3QO1SjCsXA2CDk9MM1471mYB3CTXQuKzXckJnxHkHOwU0ETpw8+AEQAJjyNXvMQdJN t07BIPDtbAQk15FfB0hKuyZVs+0lsjPKBZCamAAexNRk11eVGXK/YrqwjChkk60rt3q5i42u PpNMO9aS8cLPOfVft89Y654Qd3Rs1WRFIQq9xLjdLfHh0i0jMq5Ty+aiddSXpZ7oU6E+ud+X Czs3k5RAnOdW6eV3+v10sUjEGiFNZwzN9Udd6PfKET0J70qjnpY3NuWn5Sp1ZEn6lkq2Zm+G 9G3FlBRVClT30OWeiRHCYB6e6j1x1u/rSU4JiNYjPwSJA8EPKnt1s/Eeq37qXXvk+9DYiHdT PcOa3aNCSbIygD3jyjkg6EV9ZLHibE2R/PMMid9FrqhKh/cwcYn9FrT0FE48/2IBW5mfDpAd YvpawQlRz3XJr2rYZJwMUm1y+49+1ZmDclaF3s9dcz2JvuywNq78z/VsUfGz4Sbxy4ShpNpG REojRcz/xOK+FqNuBk+HoWKw6OxgRzfNleDvScVmbY6cQQZfGx/T7xlgZjl5Mu/2z+ofeoxb vWWM1YCJAT91GFvj29Wvm8OAPN/+SJj8LQazd9uGzVMTz6lFjVtH7YkeW/NZrP6znAwv5P1a DdQfiB5F63AX++NlTiyA+GD/ggfRl68LheSskOcxDwgI5TqmaKtX1/8RkrLpnzO3evzkfJb1 D5qh3wM1t7PZ+JWTluSX8W25ABEBAAHCwV8EGAECAAkFAk6cPPgCGwwACgkQEXu8gLWmHHz8 2w//VjRlX+tKF3szc0lQi4X0t+pf88uIsvR/a1GRZpppQbn1jgE44hgF559K6/yYemcvTR7r 6Xt7cjWGS4wfaR0+pkWV+2dbw8Xi4DI07/fN00NoVEpYUUnOnupBgychtVpxkGqsplJZQpng v6fauZtyEcUK3dLJH3TdVQDLbUcL4qZpzHbsuUnTWsmNmG4Vi0NsEt1xyd/Wuw+0kM/oFEH1 4BN6X9xZcG8GYUbVUd8+bmio8ao8m0tzo4pseDZFo4ncDmlFWU6hHnAVfkAs4tqA6/fl7RLN JuWBiOL/mP5B6HDQT9JsnaRdzqF73FnU2+WrZPjinHPLeE74istVgjbowvsgUqtzjPIG5pOj cAsKoR0M1womzJVRfYauWhYiW/KeECklci4TPBDNx7YhahSUlexfoftltJA8swRshNA/M90/ i9zDo9ySSZHwsGxG06ZOH5/MzG6HpLja7g8NTgA0TD5YaFm/oOnsQVsf2DeAGPS2xNirmknD jaqYefx7yQ7FJXXETd2uVURiDeNEFhVZWb5CiBJM5c6qQMhmkS4VyT7/+raaEGgkEKEgHOWf ZDP8BHfXtszHqI3Fo1F4IKFo/AP8GOFFxMRgbvlAs8z/+rEEaQYjxYJqj08raw6P4LFBqozr nS4h0HDFPrrp1C2EMVYIQrMokWvlFZbCpsdYbBI= Message-ID: <39228085-aed6-2fe8-79bc-dfd21c8cf2e9@de.ibm.com> Date: Wed, 6 May 2020 08:08:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: <889a7e3d-8318-4c85-67c8-a42a665b56f4@linux.ibm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138,18.0.676 definitions=2020-05-06_01:2020-05-04,2020-05-06 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 phishscore=0 adultscore=0 impostorscore=0 mlxlogscore=999 bulkscore=0 clxscore=1015 suspectscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2005060043 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On 06.05.20 00:34, Tony Krowiak wrote: > > > On 5/5/20 3:55 AM, Christian Borntraeger wrote: >> >> On 05.05.20 09:53, Cornelia Huck wrote: >>> On Tue,  5 May 2020 09:35:25 +0200 >>> Christian Borntraeger wrote: >>> >>>> In LPAR we will only get an intercept for FC==3 for the PQAP >>>> instruction. Running nested under z/VM can result in other intercepts as >>>> well, for example PQAP(QCI). So the WARN_ON_ONCE is not right. Let >>>> us simply remove it. >>> While I agree with removing the WARN_ON_ONCE, I'm wondering why z/VM >>> gives us intercepts for those fcs... is that just a result of nesting >>> (or the z/VM implementation), or is there anything we might want to do? >> Yes nesting. >> The ECA bit for interpretion is an effective one. So if the ECA bit is off >> in z/VM (no crypto cards) our ECA bit is basically ignored as these bits >> are ANDed. >> I asked Tony to ask the z/VM team if that is the case here. > > I apologize, but I was on vacation yesterday and did not have a > chance to look at this until today. I left a slack message for > my z/VM contact, but have not yet gotten a response. > > The only AP virtualization support we currently provide with > Linux on Z relies on AP interpretive execution. The ECA.28 > bit in the SIE state description determines whether AP > instructions executed on a guest are intercepted (0) or > interpreted (1). The problem here is that ECA.28 is an > effective control meaning that ECA.28 for the guest is > logically ANDed with the host's. If linux is running as a > guest of z/VM and z/VM is sets ECA.28 to zero, > then ECA.28 for the guest will also be zero, in which case > every AP instruction executed on the guest will be intercepted. > > The only AP instruction that has an interception handler is > PQAP with function code 0x03 (AP-queue interruption control), so > this warning is being issued for all other AP instructions being > intercepted; so, maybe this is the right thing to do? After all, > running a linux as a guest of z/VM that is setting ECA.28 to zero is not > a supported configuration. > > Having said that, the root of the problem is the fact that > a guest is allowed to start without AP interpretive execution > turned on because that is the only currently supported configuration. > If there is a way to determine the effective value of ECA.28 for a > KVM guest, when KVM could respond appropriately when QEMU > queries whether the KVM_S390_VM_CRYPTO_ENABLE_APIE attribute > is available in the CPU model. If it is not, then QEMU does not set the > S390_FEAT_AP (AP instructions installed) feature and a guest started > with cpu model feature ap='on' will fail to start. I think the problem is that there is no way to find out the effective value of ECA_APIE (and I think this could even change during the lifetime of a guest). So I see 3 options: 1. check for z/VM and do not advertise crypto (ap=on will always fail). This will disable the possibility to pass through a pass through device. (I think if the zVM guest has an adapter APDEDICATED then z/VM does set ECA_APIE) 2. reject all crypto instructions that do exit and are not fc==3 (PQAP ACIQ). This is kind of not ideal as we will pass through facility stfle.12 (PQAP QCI) but then fail to handle it. Linux does handle faults on these instructions just fine, though. 3. Implement emulation of crypto. I think this not what we want to do, especially as this only makes sense for acceleration but not for any of the secure key or protected key schemes. The WARN_ON is certainly something that must go as long as we construct cases that can trigger this. So I would suggest to go with the miminal patch (variant 2) which basically just removes the WARN_ON. We can then think about doing a nicer variant on top. e.g. implement PQAP QCI that just returns an empty config. I will look into this.