From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23770C4338F for ; Thu, 19 Aug 2021 16:27:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7DFA60551 for ; Thu, 19 Aug 2021 16:27:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C7DFA60551 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:54504 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mGksy-0006Bd-W3 for qemu-devel@archiver.kernel.org; Thu, 19 Aug 2021 12:27:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54842) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGks3-00059n-G6 for qemu-devel@nongnu.org; Thu, 19 Aug 2021 12:26:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46068) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGks1-0000Q1-RW for qemu-devel@nongnu.org; Thu, 19 Aug 2021 12:26:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1629390379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rhNPASckNZvnpFIiG6lVchMBaWRjor5dePWYvzcMItM=; b=ayh2JPmL1G0486pX+RBAt20ZowZNZ/rX9wa8QZJed+6jWTE997C+ysZeXHTJ/B5LW3caNI KgpXriWDo4EaiYp7SP/Lqh7VMy9CyWCGhk0dlsRuCm8K01s2z6zbVRcuJQDsp20+nmOcnm PcmfsCcQ7aIe5JrTN7qL3Q4MqbxNqs0= Received: from mail-ot1-f71.google.com (mail-ot1-f71.google.com [209.85.210.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-25-mnPkoRq6NPipV9GlgIWGhA-1; Thu, 19 Aug 2021 12:26:18 -0400 X-MC-Unique: mnPkoRq6NPipV9GlgIWGhA-1 Received: by mail-ot1-f71.google.com with SMTP id v42-20020a056830092a00b0051827b113ecso3022437ott.22 for ; Thu, 19 Aug 2021 09:26:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=69ALGd1yxKTzvnMbtojToEMrZyLhp6EJw4gjewGE2AQ=; b=RNJB2Xuo7bZgW+9/nFD9ah/bZfLRaqfL+uEHLD/9kpg8QKX4c2WOtAXf47agIzxuBx U7MDfdwow0mG2VB+exavJt2qDeMURuLxZF2tOLFYJtzp1Xb5pimlzHpzPIVFdCQYfeva KIVS7vsOhySEnK89J7+Gn4DPceRUuaxhwx9pp7mbKo6AM2qqVEJsDmdrlhqmVPAdYc7C ISlHpAuYAI0TLwl55WojGb2cteygzm0xGGVRrlNBiMs4Grw2NQ7A5Xh6w402lyi8JpB9 R+BTYhtMCFz1sMBnr7PhTEVuUEIa6rFro13MsZj/Ys49ByP5ecEUw/I3gN1jlVXtkJP5 ghug== X-Gm-Message-State: AOAM5311AsPiHHP5xUffD9UUaqIbNvBygKwnzgUFpy52qlO3x+kikYvl 06NjQUmwo589MhDaKgmyX4g35szato6SyrzUeC9EfvTweBH6/vSW1lPdFfPIkr7nXDjG3susV4L JyvN8Lx/Ul2udKWk= X-Received: by 2002:a05:6830:9c7:: with SMTP id y7mr12674048ott.111.1629390377785; Thu, 19 Aug 2021 09:26:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxsqvv1qtxa/VBqew3KeXEHmMwUxeFcKJBf/lnz5zx74uBEk6pXc3oxlh8Nl+GJSyvhRRJjmw== X-Received: by 2002:a05:6830:9c7:: with SMTP id y7mr12674028ott.111.1629390377517; Thu, 19 Aug 2021 09:26:17 -0700 (PDT) Received: from redhat.com ([198.99.80.109]) by smtp.gmail.com with ESMTPSA id x12sm747091oie.56.2021.08.19.09.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Aug 2021 09:26:16 -0700 (PDT) Date: Thu, 19 Aug 2021 10:26:15 -0600 From: Alex Williamson To: John Johnson Subject: Re: [PATCH RFC v2 01/16] vfio-user: introduce vfio-user protocol specification Message-ID: <20210819102615.0c5ac7fd.alex.williamson@redhat.com> In-Reply-To: References: <20210817170408.78be6ac7.alex.williamson@redhat.com> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=alex.williamson@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=216.205.24.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Elena Ufimtseva , Jag Raman , Swapnil Ingle , John Levon , QEMU Devel Mailing List , Stefan Hajnoczi , "thanos.makatos@nutanix.com" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Thu, 19 Aug 2021 15:32:16 +0000 John Johnson wrote: > > On Aug 17, 2021, at 7:04 PM, Alex Williamson wrote: > >=20 > >=20 > > The address/size paradigm falls into the same issues as the vfio kernel > > interface where we can't map or unmap the entire 64-bit address space, > > ie. size is limited to 2^64 - 1. The kernel interface also requires > > PAGE_SIZE granularity for the DMA, which means the practical limit is > > 2^64 - PAGE_SIZE. If we had a redo on the kernel interface we'd use > > start/end so we can express a size of (end - start + 1). > >=20 > > Is following the vfio kernel interface sufficiently worthwhile for > > compatibility to incur this same limitation? I don't recall if we've > > already discussed this, but perhaps worth a note in this design doc if > > similarity to the kernel interface is being favored here. See for > > example QEMU commit 1b296c3def4b ("vfio: Don't issue full 2^64 unmap"). > > Thanks, > > =20 >=20 >=20 > =09I=E2=80=99d prefer to stay as close to the kernel i/f as we can. > An earlier version of the spec used a vhost-user derived structure > for MAP & UNMAP. This made it more difficult to add the bitmap > field when vfio added migration capability, so we switched to the > ioctl() structure. >=20 > =09It looks like vfio_dma_unmap() takes a 64b =E2=80=98size=E2=80=99 arg > (ram_addr_t) How did you unmap an entire 64b address space? It's called from the MemoryListener which operates on MemoryRegionSections, which uses Int128 that get's chunked to ram_addr_t for vfio_dma_unmap(). We do now have VFIO_DMA_UNMAP_FLAG_ALL in the kernel API which gives us an option to clear the whole 64bit address space in one ioctl, but it's not a high priority to make use of in QEMU since it still needs to handle older kernels. > The comment there mentions a bug where iova+size wraps the end of the > 64b space. Right, that's a separate issue that's just a bug in the kernel. That's been fixed, but the QEMU code exists for now as a workaround for any broken kernels in the wild. Thanks, Alex