From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93A0CC43603 for ; Fri, 6 Dec 2019 06:10:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5719A21835 for ; Fri, 6 Dec 2019 06:10:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T5Bw9Rtk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726076AbfLFGKB (ORCPT ); Fri, 6 Dec 2019 01:10:01 -0500 Received: from mail-qt1-f195.google.com ([209.85.160.195]:34307 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726068AbfLFGKB (ORCPT ); Fri, 6 Dec 2019 01:10:01 -0500 Received: by mail-qt1-f195.google.com with SMTP id 5so6159200qtz.1 for ; Thu, 05 Dec 2019 22:10:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lTGkz4bGNLSBByKRY+XMc/8PwwmxOinFRPPxMBQvl9A=; b=T5Bw9RtkPmglwFHS+XTONH6lKglnA4/qzz5heLyDWw7UO7PX13JJLRVMnt9e/hH7Hu kamxdnrpnKaOyKInj39K/Znv2GriB7+w49A+AtrlJBJZuZxB6A+AUFBouUCuin94cfys jPIgrI3Xw7cBZ6SgtXbG7Zrun2zdy7xfZ18hxyLxMpxrIDNO8iZR+ogk7vqG/gv5UZc8 DweYMBoP+SPQFGSMSHmASaAnpPmred64CiPYtVqsyhsDgdn0Ba9zXfNDwy0Sba6Jt2AV vHNgUt3/6Gocfo8XHJbbBpI8aK9ngLfNmUyHesMOqGvM9ROKwOq4S0S4VT1sQR3+z68r lm0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lTGkz4bGNLSBByKRY+XMc/8PwwmxOinFRPPxMBQvl9A=; b=tuT1gCBGFzhOSay7qPUCfxy3z9uTNg/lqYjDTNMRfOZBje3vjgQQLKtAD/omMUl/uM UYoW3ip/AX5CXW8Qg4rUVfW71/LbaP16wDzbwmVcyzzra7lT0tlWOeHzu7ovNvnU0PFj np5zvOnJuICxupwkcTnryYERXUXcTNuqhWDv+8ptbAROtKoMYjcoTMVPXeO4XTfhsAaT BsbTqlzJdf8Lkv6dKaiXWDsJNrdAzeXEuO6FIbJY3g1pHYavlbdSyZOylJOzjgZjGqCm B3QXgdBi09jzSrdGHPkIFwKG63CcOksmxqLfRNx5QICRKRWlPV2zNZ6OVBcFVn7T3aWz JzYA== X-Gm-Message-State: APjAAAXNKhIynp0LoXdeYyKymuTSYXmhaBpVi5++qjRDFSGugc5lBHwm kYrnmFuVrFj2DJ4/LbMk2bwjNi6WyHagDoeDZiq2WA== X-Google-Smtp-Source: APXvYqwzTIJe5G1exbg4mHBzapxOvxRdfnENdXdJiMtFaRJEQuFuK+7guZGmowj2rQccgdW7QMNpaCMZVhV3Mz1hi4Y= X-Received: by 2002:ac8:7a78:: with SMTP id w24mr10649361qtt.321.1575612600205; Thu, 05 Dec 2019 22:10:00 -0800 (PST) MIME-Version: 1.0 References: <20191129183836.GA20312@google.com> In-Reply-To: <20191129183836.GA20312@google.com> From: Ranran Date: Fri, 6 Dec 2019 08:09:48 +0200 Message-ID: Subject: Re: [Bug 205701] New: Can't access RAM from PCIe To: Bjorn Helgaas Cc: bjorn@helgaas.com, linux-pci@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Fri, Nov 29, 2019 at 8:38 PM Bjorn Helgaas wrote: > > On Fri, Nov 29, 2019 at 06:10:51PM +0200, Ranran wrote: > > On Fri, Nov 29, 2019 at 4:58 PM Bjorn Helgaas wrote: > > > On Fri, Nov 29, 2019 at 06:59:48AM +0000, bugzilla-daemon@bugzilla.kernel.org wrote: > > > > https://bugzilla.kernel.org/show_bug.cgi?id=205701 > > > > ... > > > > > > > > Using Intel Xeon computer with linux kernel 4.18.0 centos8. > > > > Trying to access RAM (with DMA) using FPGA fails in this computer. > > > > > > > > 1. I tried to add intel_iommu=off - it did not help. > > > > > > > > 2. Installing windows on same PC - FPGA can access RAM using DMA without > > > > issues. > > > > > > > > 3. using another PC (Intel Duo) with same linux and OS - FPGA access works. > > > > > > > > FPGA access the RAM using a physical address provided by a kernel module which > > > > allocates physical continuous memory in PC. (the module works perfectly with > > > > Intel Duo on exactly same OS and kernel). > > > > > > Hi, thanks for the report! Can you please attach the complete dmesg > > > and "sudo lspci -vv" output for the working and non-working v4.18 > > > kernels to the bugzilla? > > > > > > Then please try to reproduce the problem on the current v5.4 kernel > > > and attach the v5.4 dmesg log. If v5.4 fails, we'll have to debug it. > > > If v5.4 works, figure out what fixed it (by comparing dmesg logs or by > > > bisection) and backport it to v4.18. > > > > > > Bjorn > > > > Hi, > > I've attached 2 files: > > 1. dmesg.log - is the dmesg you've requested. > > 2. dmesg_intel_iommu_off.log - dmesg when I added intel_iommu=off > > kernel parameter. > > Thanks, I attached these to the bugzilla. I think the linux-pci > mailing list rejected your mail since it wasn't plain-text. > > Please also attach the "sudo lspci -vv" output to the bugzilla and > indicate which device is your FPGA. I guess it might be 0000:20:00.0, > since it looks like it's being claimed by an out-of-tree module in > your dmesg_intel_iommu_off.log (but not dmesg.log). > > Please also attach the driver source so we can see how it is obtaining > and using the DMA buffer address. > > > I might try the new kernel, yet since we are required to use the > > installation of centos8 (centos8 was just published about 2 month ago > > and it comes with kernel 4.18.0), updating kernel might be > > problematic. > > Even if you can't use the v5.4 kernel for your project, if you can > establish that it works, then you have a clear path to finding the > fix. If v5.4 still *doesn't* work, then we'll be much more interested > in helping to fix that. > > > I would please like to ask if there is some workaround you can think of ? > > For example, might it help if I disable iommu (VT-d) in BIOS ? > > Usually when an IOMMU blocks a DMA, it seems like there's a note in > dmesg. I don't see that in either of your logs, but I'm not an IOMMU > expert, so it does seem reasonable to try disabling the IOMMU. > > Bjorn Hello, I have tried to upgrade to latest kernel 5.4 (elrepo in centos), but with this processor/board (system x3650, Xeon), it get hang during kernel boot, without any error in dmesg, just keeps waiting for nothing for couple of minutes and than drops to dracut. I am really stuck with this, is there any way to get progress with this bug ? Please help, Ran