From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7177BC433F5 for ; Fri, 24 Sep 2021 20:38:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5233760F0F for ; Fri, 24 Sep 2021 20:38:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345886AbhIXUjg (ORCPT ); Fri, 24 Sep 2021 16:39:36 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:41098 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344835AbhIXUjf (ORCPT ); Fri, 24 Sep 2021 16:39:35 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18OIe81S020426; Fri, 24 Sep 2021 16:37:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=6/nUw5At1aN1CT7b27JpFvMKOk4+CitUZqiAFwOaYWY=; b=ETiVHkINm08KvWlpWSzR7vkWNKlGBOuTf44DqGOdzP7RNL7TgwTQ8zvWEypOcAWYQoRf XU5dsSAFhYctzBfEJ5n3gCqUW7x0201NzHpenN+0TmC/H/nnb/Sh5tpMYKq3kZh650W3 uNuF3y1XG43Kh6StejOk1/+NGsCXh/R0CfEKZk/+u3LIiLKNp5MVORCgFZW2qxhHH50X mYEG8K0kthfHYSz2RM4mIWOVb4ezgSLfM2dMDAeXo54mthJqLg6pwD+Sx4e18Z+0G+Ku 75Q3Qgeqf1OtNZXPz/bIa6uoqZ0p/03wWbMb3QX8//wZYafb7UmAG/FjUSyluq2hBhSu 5Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b991et954-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Sep 2021 16:37:52 -0400 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18OKbqxu023722; Fri, 24 Sep 2021 16:37:52 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b991et94s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Sep 2021 16:37:51 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18OKSM8A030037; Fri, 24 Sep 2021 20:37:51 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma05wdc.us.ibm.com with ESMTP id 3b93g7mm1a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Sep 2021 20:37:51 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18OKbocD27918732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Sep 2021 20:37:50 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 292AC12405A; Fri, 24 Sep 2021 20:37:50 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 46A25124054; Fri, 24 Sep 2021 20:37:44 +0000 (GMT) Received: from farman-thinkpad-t470p (unknown [9.211.34.14]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Fri, 24 Sep 2021 20:37:44 +0000 (GMT) Message-ID: <9c1f3f9321f595e6d42dab1413637ad927b6bf2d.camel@linux.ibm.com> Subject: Re: [PATCH v2 3/9] vfio/ccw: Convert to use vfio_register_group_dev() From: Eric Farman To: Jason Gunthorpe , David Airlie , Tony Krowiak , Alex Williamson , Christian Borntraeger , Cornelia Huck , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Christoph Hellwig Date: Fri, 24 Sep 2021 16:37:43 -0400 In-Reply-To: <3-v2-7d3a384024cf+2060-ccw_mdev_jgg@nvidia.com> References: <3-v2-7d3a384024cf+2060-ccw_mdev_jgg@nvidia.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 (3.28.5-16.el8) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Pe-5TWfx0OkA7Xf0fxmKLuWfEvxU88oH X-Proofpoint-GUID: XJdgXexPzpEuKd86av5RkT9tD0GdhQ34 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-09-24_05,2021-09-24_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 mlxscore=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109240124 Precedence: bulk List-ID: X-Mailing-List: linux-s390@vger.kernel.org On Thu, 2021-09-09 at 16:38 -0300, Jason Gunthorpe wrote: > This is a more complicated conversion because vfio_ccw is sharing the > vfio_device between both the mdev_device, its vfio_device and the > css_driver. > > The mdev is a singleton, and the reason for this sharing is so the > extra > css_driver function callbacks to be delivered to the vfio_device > implementation. > > This keeps things as they are, with the css_driver allocating the > singleton, not the mdev_driver. Following patches work to clean this > further. > > Embed the vfio_device in the vfio_ccw_private and instantiate it as a > vfio_device when the mdev probes. The drvdata of both the css_device > and > the mdev_device point at the private, and container_of is used to get > it > back from the vfio_device. > > Signed-off-by: Jason Gunthorpe > --- > drivers/s390/cio/vfio_ccw_drv.c | 21 ++++-- > drivers/s390/cio/vfio_ccw_ops.c | 107 +++++++++++++++++--------- > -- > drivers/s390/cio/vfio_ccw_private.h | 5 ++ > 3 files changed, 85 insertions(+), 48 deletions(-) > > diff --git a/drivers/s390/cio/vfio_ccw_drv.c > b/drivers/s390/cio/vfio_ccw_drv.c > index 1e8d3151e5480e..396e815f81f8a4 100644 > --- a/drivers/s390/cio/vfio_ccw_drv.c > +++ b/drivers/s390/cio/vfio_ccw_drv.c > @@ -469,7 +469,7 @@ static int __init vfio_ccw_sch_init(void) > vfio_ccw_work_q = create_singlethread_workqueue("vfio-ccw"); > if (!vfio_ccw_work_q) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_io_region = > kmem_cache_create_usercopy("vfio_ccw_io_region", > @@ -478,7 +478,7 @@ static int __init vfio_ccw_sch_init(void) > sizeof(struct ccw_io_region), > NULL); > if (!vfio_ccw_io_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_cmd_region = > kmem_cache_create_usercopy("vfio_ccw_cmd_region", > @@ -487,7 +487,7 @@ static int __init vfio_ccw_sch_init(void) > sizeof(struct ccw_cmd_region), > NULL); > if (!vfio_ccw_cmd_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_schib_region = > kmem_cache_create_usercopy("vfio_ccw_schib_region", > @@ -497,7 +497,7 @@ static int __init vfio_ccw_sch_init(void) > > if (!vfio_ccw_schib_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_crw_region = > kmem_cache_create_usercopy("vfio_ccw_crw_region", > @@ -507,19 +507,25 @@ static int __init vfio_ccw_sch_init(void) > > if (!vfio_ccw_crw_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > + ret = mdev_register_driver(&vfio_ccw_mdev_driver); > + if (ret) > + goto out_regions; > + > isc_register(VFIO_CCW_ISC); > ret = css_driver_register(&vfio_ccw_sch_driver); > if (ret) { > isc_unregister(VFIO_CCW_ISC); > - goto out_err; > + goto out_driver; > } > > return ret; > > -out_err: > +out_driver: > + mdev_unregister_driver(&vfio_ccw_mdev_driver); > +out_regions: > vfio_ccw_destroy_regions(); > destroy_workqueue(vfio_ccw_work_q); > vfio_ccw_debug_exit(); > @@ -528,6 +534,7 @@ static int __init vfio_ccw_sch_init(void) > > static void __exit vfio_ccw_sch_exit(void) > { > + mdev_unregister_driver(&vfio_ccw_mdev_driver); Wouldn't it be better to mirror the unwind-init case, such that the above goes... > css_driver_unregister(&vfio_ccw_sch_driver); > isc_unregister(VFIO_CCW_ISC); ...here? > vfio_ccw_destroy_regions(); > diff --git a/drivers/s390/cio/vfio_ccw_ops.c ...snip... Besides that, looks fine to me. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C2F1C433F5 for ; Fri, 24 Sep 2021 21:29:41 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3376D6105A for ; Fri, 24 Sep 2021 21:29:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3376D6105A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4C13D6EE84; Fri, 24 Sep 2021 21:29:38 +0000 (UTC) Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3E1006E23B; Fri, 24 Sep 2021 20:37:57 +0000 (UTC) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18OIe81S020426; Fri, 24 Sep 2021 16:37:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=6/nUw5At1aN1CT7b27JpFvMKOk4+CitUZqiAFwOaYWY=; b=ETiVHkINm08KvWlpWSzR7vkWNKlGBOuTf44DqGOdzP7RNL7TgwTQ8zvWEypOcAWYQoRf XU5dsSAFhYctzBfEJ5n3gCqUW7x0201NzHpenN+0TmC/H/nnb/Sh5tpMYKq3kZh650W3 uNuF3y1XG43Kh6StejOk1/+NGsCXh/R0CfEKZk/+u3LIiLKNp5MVORCgFZW2qxhHH50X mYEG8K0kthfHYSz2RM4mIWOVb4ezgSLfM2dMDAeXo54mthJqLg6pwD+Sx4e18Z+0G+Ku 75Q3Qgeqf1OtNZXPz/bIa6uoqZ0p/03wWbMb3QX8//wZYafb7UmAG/FjUSyluq2hBhSu 5Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b991et954-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Sep 2021 16:37:52 -0400 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 18OKbqxu023722; Fri, 24 Sep 2021 16:37:52 -0400 Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b991et94s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Sep 2021 16:37:51 -0400 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18OKSM8A030037; Fri, 24 Sep 2021 20:37:51 GMT Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by ppma05wdc.us.ibm.com with ESMTP id 3b93g7mm1a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Sep 2021 20:37:51 +0000 Received: from b01ledav002.gho.pok.ibm.com (b01ledav002.gho.pok.ibm.com [9.57.199.107]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18OKbocD27918732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Sep 2021 20:37:50 GMT Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 292AC12405A; Fri, 24 Sep 2021 20:37:50 +0000 (GMT) Received: from b01ledav002.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 46A25124054; Fri, 24 Sep 2021 20:37:44 +0000 (GMT) Received: from farman-thinkpad-t470p (unknown [9.211.34.14]) by b01ledav002.gho.pok.ibm.com (Postfix) with ESMTP; Fri, 24 Sep 2021 20:37:44 +0000 (GMT) Message-ID: <9c1f3f9321f595e6d42dab1413637ad927b6bf2d.camel@linux.ibm.com> From: Eric Farman To: Jason Gunthorpe , David Airlie , Tony Krowiak , Alex Williamson , Christian Borntraeger , Cornelia Huck , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Christoph Hellwig Date: Fri, 24 Sep 2021 16:37:43 -0400 In-Reply-To: <3-v2-7d3a384024cf+2060-ccw_mdev_jgg@nvidia.com> References: <3-v2-7d3a384024cf+2060-ccw_mdev_jgg@nvidia.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 (3.28.5-16.el8) Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Pe-5TWfx0OkA7Xf0fxmKLuWfEvxU88oH X-Proofpoint-GUID: XJdgXexPzpEuKd86av5RkT9tD0GdhQ34 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-09-24_05,2021-09-24_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 mlxscore=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109240124 X-Mailman-Approved-At: Fri, 24 Sep 2021 21:29:37 +0000 Subject: Re: [Intel-gfx] [PATCH v2 3/9] vfio/ccw: Convert to use vfio_register_group_dev() X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On Thu, 2021-09-09 at 16:38 -0300, Jason Gunthorpe wrote: > This is a more complicated conversion because vfio_ccw is sharing the > vfio_device between both the mdev_device, its vfio_device and the > css_driver. > > The mdev is a singleton, and the reason for this sharing is so the > extra > css_driver function callbacks to be delivered to the vfio_device > implementation. > > This keeps things as they are, with the css_driver allocating the > singleton, not the mdev_driver. Following patches work to clean this > further. > > Embed the vfio_device in the vfio_ccw_private and instantiate it as a > vfio_device when the mdev probes. The drvdata of both the css_device > and > the mdev_device point at the private, and container_of is used to get > it > back from the vfio_device. > > Signed-off-by: Jason Gunthorpe > --- > drivers/s390/cio/vfio_ccw_drv.c | 21 ++++-- > drivers/s390/cio/vfio_ccw_ops.c | 107 +++++++++++++++++--------- > -- > drivers/s390/cio/vfio_ccw_private.h | 5 ++ > 3 files changed, 85 insertions(+), 48 deletions(-) > > diff --git a/drivers/s390/cio/vfio_ccw_drv.c > b/drivers/s390/cio/vfio_ccw_drv.c > index 1e8d3151e5480e..396e815f81f8a4 100644 > --- a/drivers/s390/cio/vfio_ccw_drv.c > +++ b/drivers/s390/cio/vfio_ccw_drv.c > @@ -469,7 +469,7 @@ static int __init vfio_ccw_sch_init(void) > vfio_ccw_work_q = create_singlethread_workqueue("vfio-ccw"); > if (!vfio_ccw_work_q) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_io_region = > kmem_cache_create_usercopy("vfio_ccw_io_region", > @@ -478,7 +478,7 @@ static int __init vfio_ccw_sch_init(void) > sizeof(struct ccw_io_region), > NULL); > if (!vfio_ccw_io_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_cmd_region = > kmem_cache_create_usercopy("vfio_ccw_cmd_region", > @@ -487,7 +487,7 @@ static int __init vfio_ccw_sch_init(void) > sizeof(struct ccw_cmd_region), > NULL); > if (!vfio_ccw_cmd_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_schib_region = > kmem_cache_create_usercopy("vfio_ccw_schib_region", > @@ -497,7 +497,7 @@ static int __init vfio_ccw_sch_init(void) > > if (!vfio_ccw_schib_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > vfio_ccw_crw_region = > kmem_cache_create_usercopy("vfio_ccw_crw_region", > @@ -507,19 +507,25 @@ static int __init vfio_ccw_sch_init(void) > > if (!vfio_ccw_crw_region) { > ret = -ENOMEM; > - goto out_err; > + goto out_regions; > } > > + ret = mdev_register_driver(&vfio_ccw_mdev_driver); > + if (ret) > + goto out_regions; > + > isc_register(VFIO_CCW_ISC); > ret = css_driver_register(&vfio_ccw_sch_driver); > if (ret) { > isc_unregister(VFIO_CCW_ISC); > - goto out_err; > + goto out_driver; > } > > return ret; > > -out_err: > +out_driver: > + mdev_unregister_driver(&vfio_ccw_mdev_driver); > +out_regions: > vfio_ccw_destroy_regions(); > destroy_workqueue(vfio_ccw_work_q); > vfio_ccw_debug_exit(); > @@ -528,6 +534,7 @@ static int __init vfio_ccw_sch_init(void) > > static void __exit vfio_ccw_sch_exit(void) > { > + mdev_unregister_driver(&vfio_ccw_mdev_driver); Wouldn't it be better to mirror the unwind-init case, such that the above goes... > css_driver_unregister(&vfio_ccw_sch_driver); > isc_unregister(VFIO_CCW_ISC); ...here? > vfio_ccw_destroy_regions(); > diff --git a/drivers/s390/cio/vfio_ccw_ops.c ...snip... Besides that, looks fine to me.