From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 034C7C433E2 for ; Fri, 4 Sep 2020 08:19:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B4EE3206C9 for ; Fri, 4 Sep 2020 08:19:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gt3FzVO1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729759AbgIDITH (ORCPT ); Fri, 4 Sep 2020 04:19:07 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:60892 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726151AbgIDITG (ORCPT ); Fri, 4 Sep 2020 04:19:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599207545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=phO9cBpvfpkXU4NPdjyoUela0DTAh6976p8XX9PeDZQ=; b=gt3FzVO1r2yoGTpmyOLBtsqzEFshVTceLGSCx28+bUlFZ8tAUsf8zjaEPpnFCwULBUdPwu Gj/5k/fVykSTvKnHE+pJelJaKcBxmRg4HGml00ycCSlFeOyOioMZsK2iUvd4+R0psrk9zc uUVlJ5P7gKNJ7kBNSTGge0b8vMl9DGE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-26-wpVnDR64MJ6lLMQmyz9u9Q-1; Fri, 04 Sep 2020 04:19:03 -0400 X-MC-Unique: wpVnDR64MJ6lLMQmyz9u9Q-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 40121107464E; Fri, 4 Sep 2020 08:19:02 +0000 (UTC) Received: from [10.36.112.51] (ovpn-112-51.ams2.redhat.com [10.36.112.51]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DD3A1002D64; Fri, 4 Sep 2020 08:18:57 +0000 (UTC) Subject: Re: [PATCH v4 09/10] vfio/fsl-mc: Add read/write support for fsl-mc devices To: Diana Craciun , alex.williamson@redhat.com, kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, bharatb.linux@gmail.com, laurentiu.tudor@nxp.com, Bharat Bhushan References: <20200826093315.5279-1-diana.craciun@oss.nxp.com> <20200826093315.5279-10-diana.craciun@oss.nxp.com> From: Auger Eric Message-ID: <182a6686-a1ca-398b-2ccc-8a5638ffe7aa@redhat.com> Date: Fri, 4 Sep 2020 10:18:56 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <20200826093315.5279-10-diana.craciun@oss.nxp.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Diana, On 8/26/20 11:33 AM, Diana Craciun wrote: > The software uses a memory-mapped I/O command interface (MC portals) to > communicate with the MC hardware. This command interface is used to > discover, enumerate, configure and remove DPAA2 objects. The DPAA2 > objects use MSIs, so the command interface needs to be emulated > such that the correct MSI is configured in the hardware (the guest > has the virtual MSIs). What I don't get is all the regions are mmappable too. And this patch does not seem to introduce special handling with respect to MSIs. Please could you clarify? > > This patch is adding read/write support for fsl-mc devices. The mc > commands are emulated by the userspace. The host is just passing > the correct command to the hardware. > > Also the current patch limits userspace to write complete > 64byte command once and read 64byte response by one ioctl. > > Signed-off-by: Bharat Bhushan > Signed-off-by: Diana Craciun > --- > drivers/vfio/fsl-mc/vfio_fsl_mc.c | 115 +++++++++++++++++++++- > drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 1 + > 2 files changed, 114 insertions(+), 2 deletions(-) > > diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c > index 73834f488a94..27713aa86878 100644 > --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c > +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c > @@ -12,6 +12,7 @@ > #include > #include > #include > +#include > > #include "vfio_fsl_mc_private.h" > > @@ -106,6 +107,9 @@ static int vfio_fsl_mc_regions_init(struct vfio_fsl_mc_device *vdev) > vdev->regions[i].size = resource_size(res); > vdev->regions[i].flags = VFIO_REGION_INFO_FLAG_MMAP; > vdev->regions[i].type = mc_dev->regions[i].flags & IORESOURCE_BITS; > + vdev->regions[i].flags |= VFIO_REGION_INFO_FLAG_READ; > + if (!(mc_dev->regions[i].flags & IORESOURCE_READONLY)) > + vdev->regions[i].flags |= VFIO_REGION_INFO_FLAG_WRITE; > } > > vdev->num_regions = mc_dev->obj_desc.region_count; > @@ -114,6 +118,11 @@ static int vfio_fsl_mc_regions_init(struct vfio_fsl_mc_device *vdev) > > static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev) > { > + int i; > + > + for (i = 0; i < vdev->num_regions; i++) > + iounmap(vdev->regions[i].ioaddr); > + > vdev->num_regions = 0; > kfree(vdev->regions); > } > @@ -311,13 +320,115 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, > static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf, > size_t count, loff_t *ppos) > { > - return -EINVAL; > + struct vfio_fsl_mc_device *vdev = device_data; > + unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos); > + loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK; > + struct vfio_fsl_mc_region *region; > + u64 data[8]; > + int i; > + > + if (index >= vdev->num_regions) > + return -EINVAL; > + > + region = &vdev->regions[index]; > + > + if (!(region->flags & VFIO_REGION_INFO_FLAG_READ)) > + return -EINVAL; > + > + if (!region->ioaddr) { > + region->ioaddr = ioremap(region->addr, region->size); > + if (!region->ioaddr) > + return -ENOMEM; > + } > + > + if (count != 64 || off != 0) > + return -EINVAL; > + > + for (i = 7; i >= 0; i--) > + data[i] = readq(region->ioaddr + i * sizeof(uint64_t)); > + > + if (copy_to_user(buf, data, 64)) > + return -EFAULT; > + > + return count; > +} > + > +#define MC_CMD_COMPLETION_TIMEOUT_MS 5000 > +#define MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS 500 > + > +static int vfio_fsl_mc_send_command(void __iomem *ioaddr, uint64_t *cmd_data) > +{ > + int i; > + enum mc_cmd_status status; > + unsigned long timeout_usecs = MC_CMD_COMPLETION_TIMEOUT_MS * 1000; > + > + /* Write at command parameter into portal */ > + for (i = 7; i >= 1; i--) > + writeq_relaxed(cmd_data[i], ioaddr + i * sizeof(uint64_t)); > + > + /* Write command header in the end */ > + writeq(cmd_data[0], ioaddr); > + > + /* Wait for response before returning to user-space > + * This can be optimized in future to even prepare response > + * before returning to user-space and avoid read ioctl. > + */ > + for (;;) { > + u64 header; > + struct mc_cmd_header *resp_hdr; > + > + header = cpu_to_le64(readq_relaxed(ioaddr)); > + > + resp_hdr = (struct mc_cmd_header *)&header; > + status = (enum mc_cmd_status)resp_hdr->status; > + if (status != MC_CMD_STATUS_READY) > + break; > + > + udelay(MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS); > + timeout_usecs -= MC_CMD_COMPLETION_POLLING_MAX_SLEEP_USECS; > + if (timeout_usecs == 0) > + return -ETIMEDOUT; > + } > + > + return 0; > } > > static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf, > size_t count, loff_t *ppos) > { > - return -EINVAL; > + struct vfio_fsl_mc_device *vdev = device_data; > + unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos); > + loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK; > + struct vfio_fsl_mc_region *region; > + u64 data[8]; > + int ret; > + > + if (index >= vdev->num_regions) > + return -EINVAL; > + > + region = &vdev->regions[index]; > + > + if (!(region->flags & VFIO_REGION_INFO_FLAG_WRITE)) > + return -EINVAL; > + > + if (!region->ioaddr) { > + region->ioaddr = ioremap(region->addr, region->size); > + if (!region->ioaddr) > + return -ENOMEM; > + } > + > + if (count != 64 || off != 0) > + return -EINVAL; > + > + if (copy_from_user(&data, buf, 64)) > + return -EFAULT; > + > + ret = vfio_fsl_mc_send_command(region->ioaddr, data); > + if (ret) > + return ret; > + > + return count; > + > } > > static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region, > diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h > index bbfca8b55f8a..e6804e516c4a 100644 > --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h > +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h > @@ -32,6 +32,7 @@ struct vfio_fsl_mc_region { > u32 type; > u64 addr; > resource_size_t size; > + void __iomem *ioaddr; > }; > > struct vfio_fsl_mc_device { > Thanks Eric