From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 52745C04FFE for ; Sat, 11 May 2024 04:10:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.720203.1123287 (Exim 4.92) (envelope-from ) id 1s5e3r-0003WT-PX; Sat, 11 May 2024 04:10:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 720203.1123287; Sat, 11 May 2024 04:10:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s5e3r-0003WM-MC; Sat, 11 May 2024 04:10:15 +0000 Received: by outflank-mailman (input) for mailman id 720203; Sat, 11 May 2024 04:10:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s5e3p-0003WG-Ty for xen-devel@lists.xenproject.org; Sat, 11 May 2024 04:10:13 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 5b39f051-0f4c-11ef-b4bb-af5377834399; Sat, 11 May 2024 06:10:10 +0200 (CEST) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:8ac4:0:0:0:0:f7]) by mailhost.m5p.com (8.17.1/8.17.1) with ESMTPS id 44B49s3g016991 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 11 May 2024 00:10:00 -0400 (EDT) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.17.1/8.15.2/Submit) id 44B49s11016990; Fri, 10 May 2024 21:09:54 -0700 (PDT) (envelope-from ehem) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 5b39f051-0f4c-11ef-b4bb-af5377834399 Date: Fri, 10 May 2024 21:09:54 -0700 From: Elliott Mitchell To: Jan Beulich Cc: xen-devel@lists.xenproject.org, Andrew Cooper , Roger Pau =?iso-8859-1?Q?Monn=E9?= , Wei Liu , Kelly Choi Subject: Re: Serious AMD-Vi(?) issue Message-ID: References: <2aa4d1f4-ff37-4f12-bfbb-3ef5ad3f6fdd@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Thu, Apr 18, 2024 at 09:33:31PM -0700, Elliott Mitchell wrote: > > I suspect this is a case of there is some step which is missing from > Xen's IOMMU handling. Perhaps something which Linux does during an early > DMA setup stage, but the current Xen implementation does lazily? > Alternatively some flag setting or missing step? > > I should be able to do another test approach in a few weeks, but I would > love if something could be found sooner. Turned out to be disturbingly easy to get the first entry when it happened. Didn't even need `dbench`, it simply showed once the OS was fully loaded. I did get some additional data points. Appears this requires an AMD IOMMUv2. A test system with known functioning AMD IOMMUv1 didn't display the issue at all. (XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr fffffffdf8000000 flags 0x8 I (XEN) DDDD:bb:dd.f root @ 83b5f5 (3 levels) dfn=fffffffdf8000 (XEN) L3[1f7] = 0 np I find it surprising this required "iommu=debug" to get this level of detail. This amount of output seems more appropriate for "verbose". I strongly prefer to provide snippets. There is a fair bit of output, I'm unsure which portion is most pertinent. -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445