From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com (ale.deltatee.com [207.54.116.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id F40EA21951C81 for ; Tue, 25 Apr 2017 09:58:48 -0700 (PDT) References: <1490911959-5146-1-git-send-email-logang@deltatee.com> <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> From: Logan Gunthorpe Message-ID: <190d9a89-e227-e410-5886-ee2e20a9b941@deltatee.com> Date: Tue, 25 Apr 2017 10:58:40 -0600 MIME-Version: 1.0 In-Reply-To: <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Subject: Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Marta Rybczynska Cc: Jens Axboe , Keith Busch , "James E.J. Bottomley" , "Martin K. Petersen" , linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org, Steve Wise , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, Sinan Kaya , Alex Williamson , linux-scsi@vger.kernel.org, Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig , Jason Gunthorpe List-ID: On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be addressed by 'arch_p2p_cross_segment' as proposed by Jason. Logan _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 From: Logan Gunthorpe Subject: Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device Date: Tue, 25 Apr 2017 10:58:40 -0600 Message-ID: <190d9a89-e227-e410-5886-ee2e20a9b941@deltatee.com> References: <1490911959-5146-1-git-send-email-logang@deltatee.com> <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <117239810.400362790.1493121533234.JavaMail.zimbra-FNhOzJFKnXGHXe+LvDLADg@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: Marta Rybczynska Cc: Jens Axboe , Keith Busch , "James E.J. Bottomley" , "Martin K. Petersen" , linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Steve Wise , linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, Sinan Kaya , Alex Williamson , linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig , Jason Gunthorpe List-Id: linux-rdma@vger.kernel.org On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be addressed by 'arch_p2p_cross_segment' as proposed by Jason. Logan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1949825AbdDYQ7J (ORCPT ); Tue, 25 Apr 2017 12:59:09 -0400 Received: from ale.deltatee.com ([207.54.116.67]:49357 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1432405AbdDYQ7A (ORCPT ); Tue, 25 Apr 2017 12:59:00 -0400 To: Marta Rybczynska References: <1490911959-5146-1-git-send-email-logang@deltatee.com> <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Cc: Sinan Kaya , Christoph Hellwig , Sagi Grimberg , "James E.J. Bottomley" , "Martin K. Petersen" , Jens Axboe , Steve Wise , Stephen Bates , Max Gurtovoy , Dan Williams , Keith Busch , Jason Gunthorpe , linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@ml01.01.org, linux-kernel@vger.kernel.org, Alex Williamson , Bjorn Helgaas From: Logan Gunthorpe Message-ID: <190d9a89-e227-e410-5886-ee2e20a9b941@deltatee.com> Date: Tue, 25 Apr 2017 10:58:40 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 172.16.1.111 X-SA-Exim-Rcpt-To: bhelgaas@google.com, alex.williamson@redhat.com, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org, linux-scsi@vger.kernel.org, linux-pci@vger.kernel.org, jgunthorpe@obsidianresearch.com, keith.busch@intel.com, dan.j.williams@intel.com, maxg@mellanox.com, sbates@raithlin.com, swise@opengridcomputing.com, axboe@kernel.dk, martin.petersen@oracle.com, jejb@linux.vnet.ibm.com, sagi@grimberg.me, hch@lst.de, okaya@codeaurora.org, mrybczyn@kalray.eu X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device X-SA-Exim-Version: 4.2.1 (built Mon, 26 Dec 2011 16:24:06 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be addressed by 'arch_p2p_cross_segment' as proposed by Jason. Logan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com ([207.54.116.67]:49357 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1432405AbdDYQ7A (ORCPT ); Tue, 25 Apr 2017 12:59:00 -0400 To: Marta Rybczynska References: <1490911959-5146-1-git-send-email-logang@deltatee.com> <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Cc: Sinan Kaya , Christoph Hellwig , Sagi Grimberg , "James E.J. Bottomley" , "Martin K. Petersen" , Jens Axboe , Steve Wise , Stephen Bates , Max Gurtovoy , Dan Williams , Keith Busch , Jason Gunthorpe , linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, Alex Williamson , Bjorn Helgaas From: Logan Gunthorpe Message-ID: <190d9a89-e227-e410-5886-ee2e20a9b941@deltatee.com> Date: Tue, 25 Apr 2017 10:58:40 -0600 MIME-Version: 1.0 In-Reply-To: <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Content-Type: text/plain; charset=utf-8 Subject: Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device Sender: linux-pci-owner@vger.kernel.org List-ID: On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be addressed by 'arch_p2p_cross_segment' as proposed by Jason. Logan From mboxrd@z Thu Jan 1 00:00:00 1970 From: logang@deltatee.com (Logan Gunthorpe) Date: Tue, 25 Apr 2017 10:58:40 -0600 Subject: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device In-Reply-To: <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> References: <1490911959-5146-1-git-send-email-logang@deltatee.com> <63d73995-4dfa-6270-ca13-972e0628011b@deltatee.com> <117239810.400362790.1493121533234.JavaMail.zimbra@kalray.eu> Message-ID: <190d9a89-e227-e410-5886-ee2e20a9b941@deltatee.com> On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be addressed by 'arch_p2p_cross_segment' as proposed by Jason. Logan