From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 536A9C433EF for ; Wed, 20 Apr 2022 15:27:20 +0000 (UTC) Received: from localhost ([::1]:52080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nhCEh-0001C5-2G for qemu-devel@archiver.kernel.org; Wed, 20 Apr 2022 11:27:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nhC0i-0007Wk-SE; Wed, 20 Apr 2022 11:12:52 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:25532 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nhC0g-0005o1-PY; Wed, 20 Apr 2022 11:12:52 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23KDS9oQ005575; Wed, 20 Apr 2022 15:12:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=2QjDcmuQezqaNkyEO6jI8dBmrXaIfCpYK/bI29yIygM=; b=iODecKy6eTuthRJLWnFBgU7QfENmRjCceN9DisDFPbfj5Rd/JqChgX/r/ES/rjJXu5Vc 9iu85HbW6I7frGI2Ojv1guhSvtYWi7n2aLfurBvTe6gBY6Qp6yd7h0sIwmWpYibLGB/U fihnWViBFEZoVPmWHZKq6LvtukC4qAiqeRev2HgfPmg1+aECUbupwDda07TiuHGCY4gJ PV8bQN1sMkPOG5oevDmtBOP0nkQJ4w7TlcsPVQ5vsZd+wlT6UDuTnLcYEG/uCYmxTidM OecN7qzdQpWyjtECbfqnFf+nhhu0mDY4nloEZ6GlThTGsMJGaxEF8dVq9RIa7A1SOFa6 pg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3fjf52037r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Apr 2022 15:12:39 +0000 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 23KF7CXZ026644; Wed, 20 Apr 2022 15:12:39 GMT Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 3fjf520372-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Apr 2022 15:12:39 +0000 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 23KF33Nb018672; Wed, 20 Apr 2022 15:12:38 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma02wdc.us.ibm.com with ESMTP id 3fg2xw1qp4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Apr 2022 15:12:38 +0000 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 23KFCbel25166218 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 Apr 2022 15:12:37 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B19F7AE060; Wed, 20 Apr 2022 15:12:37 +0000 (GMT) Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AA18FAE05C; Wed, 20 Apr 2022 15:12:35 +0000 (GMT) Received: from [9.211.82.47] (unknown [9.211.82.47]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 20 Apr 2022 15:12:35 +0000 (GMT) Message-ID: <1d9a128a-1391-712b-abdc-7d4d9c1e5cc0@linux.ibm.com> Date: Wed, 20 Apr 2022 11:12:34 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH v5 5/9] s390x/pci: enable for load/store intepretation Content-Language: en-US To: Pierre Morel , qemu-s390x@nongnu.org References: <20220404181726.60291-1-mjrosato@linux.ibm.com> <20220404181726.60291-6-mjrosato@linux.ibm.com> From: Matthew Rosato In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: VQkOyMvibtvLhK9kE5Pjk8lXQdukuxaH X-Proofpoint-ORIG-GUID: KLLJjORwFVWkq2bf7jyoGfePBSlGTOGj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-20_04,2022-04-20_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 bulkscore=0 spamscore=0 phishscore=0 adultscore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204200090 Received-SPF: pass client-ip=148.163.158.5; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: farman@linux.ibm.com, kvm@vger.kernel.org, schnelle@linux.ibm.com, cohuck@redhat.com, richard.henderson@linaro.org, thuth@redhat.com, qemu-devel@nongnu.org, pasic@linux.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, david@redhat.com, borntraeger@linux.ibm.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On 4/19/22 3:47 PM, Pierre Morel wrote: > > > On 4/4/22 20:17, Matthew Rosato wrote: >> If the appropriate CPU facilty is available as well as the necessary >> ZPCI_OP ioctl, then the underlying KVM host will enable load/store >> intepretation for any guest device without a SHM bit in the guest >> function handle.  For a device that will be using interpretation >> support, ensure the guest function handle matches the host function >> handle; this value is re-checked every time the guest issues a SET PCI FN >> to enable the guest device as it is the only opportunity to reflect >> function handle changes. >> >> By default, unless interpret=off is specified, interpretation support >> will >> always be assumed and exploited if the necessary ioctl and features are >> available on the host kernel.  When these are unavailable, we will >> silently >> revert to the interception model; this allows existing guest >> configurations >> to work unmodified on hosts with and without zPCI interpretation support, >> allowing QEMU to choose the best support model available. >> >> Signed-off-by: Matthew Rosato >> --- >>   hw/s390x/meson.build            |  1 + >>   hw/s390x/s390-pci-bus.c         | 66 ++++++++++++++++++++++++++++++++- >>   hw/s390x/s390-pci-inst.c        | 12 ++++++ >>   hw/s390x/s390-pci-kvm.c         | 21 +++++++++++ >>   include/hw/s390x/s390-pci-bus.h |  1 + >>   include/hw/s390x/s390-pci-kvm.h | 24 ++++++++++++ >>   target/s390x/kvm/kvm.c          |  7 ++++ >>   target/s390x/kvm/kvm_s390x.h    |  1 + >>   8 files changed, 132 insertions(+), 1 deletion(-) >>   create mode 100644 hw/s390x/s390-pci-kvm.c >>   create mode 100644 include/hw/s390x/s390-pci-kvm.h >> > > ...snip... > >>           if (s390_pci_msix_init(pbdev)) { >> @@ -1360,6 +1423,7 @@ static Property s390_pci_device_properties[] = { >>       DEFINE_PROP_UINT16("uid", S390PCIBusDevice, uid, UID_UNDEFINED), >>       DEFINE_PROP_S390_PCI_FID("fid", S390PCIBusDevice, fid), >>       DEFINE_PROP_STRING("target", S390PCIBusDevice, target), >> +    DEFINE_PROP_BOOL("interpret", S390PCIBusDevice, interp, true), >>       DEFINE_PROP_END_OF_LIST(), >>   }; >> diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c >> index 6d400d4147..c898c8abe9 100644 >> --- a/hw/s390x/s390-pci-inst.c >> +++ b/hw/s390x/s390-pci-inst.c >> @@ -18,6 +18,8 @@ >>   #include "sysemu/hw_accel.h" >>   #include "hw/s390x/s390-pci-inst.h" >>   #include "hw/s390x/s390-pci-bus.h" >> +#include "hw/s390x/s390-pci-kvm.h" >> +#include "hw/s390x/s390-pci-vfio.h" >>   #include "hw/s390x/tod.h" >>   #ifndef DEBUG_S390PCI_INST >> @@ -246,6 +248,16 @@ int clp_service_call(S390CPU *cpu, uint8_t r2, >> uintptr_t ra) >>                   goto out; >>               } >> +            /* >> +             * Take this opportunity to make sure we still have an >> accurate >> +             * host fh.  It's possible part of the handle changed >> while the >> +             * device was disabled to the guest (e.g. vfio hot reset for >> +             * ISM during plug) >> +             */ >> +            if (pbdev->interp) { >> +                /* Take this opportunity to make sure we are sync'd >> with host */ >> +                s390_pci_get_host_fh(pbdev, &pbdev->fh); >> +            } >>               pbdev->fh |= FH_MASK_ENABLE; > > Are we sure here that the PCI device is always enabled? > Shouldn't we check? I guess you mean the host device? Interesting thought. So, to be clear, the idea on setting FH_MASK_ENABLE here is that we are handling a guest CLP SET PCI FN enable so the guest fh should always have FH_MASK_ENABLE set if we return CLP_RC_OK to the guest. But for interpretation, if we find the host function is disabled, I suppose we could return an error on the guest CLP (not sure which error yet); otherwise, if we return the force-enabled handle and CLP_RC_OK as we do here then the guest will just get errors attempting to use it.