From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B2BFC433ED for ; Thu, 29 Apr 2021 17:57:39 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F2886144E for ; Thu, 29 Apr 2021 17:57:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F2886144E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcAuv-0005u2-AY for qemu-devel@archiver.kernel.org; Thu, 29 Apr 2021 13:57:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42374) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcAst-0004Gm-1A for qemu-devel@nongnu.org; Thu, 29 Apr 2021 13:55:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25423) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcAsl-0006QZ-EA for qemu-devel@nongnu.org; Thu, 29 Apr 2021 13:55:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619718921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=f6RmY9lSzVscYshg1sKs49EYcSMfINot3XsTvQ80A3k=; b=YjMHWBXMTZuEcGmORwJMU+eS2orbvydDeJTyGoPfQx9sbprmFMLrH+dpxrTtUqM4M5A/4F sKAQ4ztA4K7ElDwPXFICiMJUX+hzJSBltmIa9oYn6GGmxSq99GRSnk2TCpRto6Ra5c/pde ahfsY/NTxA8Dkp8WUDZAXCDMsf4qFdc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-550-eMN-ADivM7-diwiZq2ozUg-1; Thu, 29 Apr 2021 13:55:19 -0400 X-MC-Unique: eMN-ADivM7-diwiZq2ozUg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AD9FE8042C7; Thu, 29 Apr 2021 17:55:18 +0000 (UTC) Received: from work-vm (ovpn-114-250.ams2.redhat.com [10.36.114.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E9D3A19726; Thu, 29 Apr 2021 17:55:12 +0000 (UTC) Date: Thu, 29 Apr 2021 18:55:09 +0100 From: "Dr. David Alan Gilbert" To: Alex Williamson Subject: Re: [PATCH] make vfio and DAX cache work together Message-ID: References: <20210426152203.379dab00@redhat.com> <20210427121850.68d2a8dd@redhat.com> <20210428133740.6ccbbba6@redhat.com> <20210429070901.52402ac2@x1.home.shazbot.org> MIME-Version: 1.0 In-Reply-To: <20210429070901.52402ac2@x1.home.shazbot.org> User-Agent: Mutt/2.0.6 (2021-03-06) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dgilbert@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.22, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dev Audsin , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" * Alex Williamson (alex.williamson@redhat.com) wrote: > On Thu, 29 Apr 2021 09:44:51 +0100 > "Dr. David Alan Gilbert" wrote: > > > * Alex Williamson (alex.williamson@redhat.com) wrote: > > > On Wed, 28 Apr 2021 20:17:23 +0100 > > > "Dr. David Alan Gilbert" wrote: > > > > > > > * Dev Audsin (dev.devaqemu@gmail.com) wrote: > > > > > Thanks Dave for your explanation. > > > > > Any suggestions on how to make VFIO not attempt to map into the > > > > > unaccessible and unallocated RAM. > > > > > > > > I'm not sure;: > > > > > > > > static bool vfio_listener_skipped_section(MemoryRegionSection *section) > > > > { > > > > return (!memory_region_is_ram(section->mr) && > > > > !memory_region_is_iommu(section->mr)) || > > > > section->offset_within_address_space & (1ULL << 63); > > > > } > > > > > > > > I'm declaring that region with memory_region_init_ram_ptr; should I be? > > > > it's not quite like RAM. > > > > But then I *do* want a kvm slot for it, and I do want it to be accessed > > > > by mapping rather htan calling IO functions; that makes me think mr->ram > > > > has to be true. > > > > But then do we need to add another flag to memory-regions; if we do, > > > > what is it; > > > > a) We don't want an 'is_virtio_fs' - it needs to be more generic > > > > b) 'no_vfio' also feels wrong > > > > > > > > Is perhaps 'not_lockable' the right thing to call it? > > > > > > This reasoning just seems to lead back to "it doesn't work, therefore > > > don't do it" rather than identifying the property of the region that > > > makes it safe not to map it for device DMA (assuming that's actually > > > the case). > > > > Yes, I'm struggling to get to what that generic form of that property > > is, possibly because I've not got an example of another case to compare > > it with. > > > > > It's clearly "RAM" as far as QEMU is concerned given how > > > it's created, but does it actually appear in the VM as generic physical > > > RAM that the guest OS could program to the device as a DMA target? If > > > not, what property makes that so, create a flag for that. Thanks, > > > > The guest sees it as a PCI-bar; so it knows it's not 'generic physical > > RAM' - but can a guest set other BARs (like frame buffers or pmem) as > > DMA targets? If so, how do I distinguish our bar? > > They can, this is how peer-to-peer DMA between devices works. However, > we can perhaps take advantage that drivers are generally a bit more > cautious in probing that this type of DMA works before relying on it, > and declare it with memory_region_init_ram_device_ptr() which vfio > would not consider fatal if it fails to map it. The other semantic > difference is that ram_device_mem_ops are used for read/write access to > avoid some of the opcodes that are not meant to be used for physical > device memory with the default memcpy ops. If you expect this region > to be mapped as a kvm memory slot, presumably these would never get > used anyway. Thanks, Oh, nice, I hadn't spotted memory_region_init_ram_device_ptr(); diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c index 7afd9495c9..11fb9b5979 100644 --- a/hw/virtio/vhost-user-fs.c +++ b/hw/virtio/vhost-user-fs.c @@ -604,7 +604,7 @@ static void vuf_device_realize(DeviceState *dev, Error **errp) return; } - memory_region_init_ram_ptr(&fs->cache, OBJECT(vdev), + memory_region_init_ram_device_ptr(&fs->cache, OBJECT(vdev), "virtio-fs-cache", fs->conf.cache_size, cache_ptr); } apparently still works for us; Dev does that fix it for you? Dave > Alex -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK