From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1162350AbdD0Gvy (ORCPT ); Thu, 27 Apr 2017 02:51:54 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:34627 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932726AbdD0Gvq (ORCPT ); Thu, 27 Apr 2017 02:51:46 -0400 Date: Thu, 27 Apr 2017 08:51:42 +0200 From: Ingo Molnar To: Shaohua Li Cc: linux-kernel@vger.kernel.org, gang.wei@intel.com, jroedel@suse.de, hpa@linux.intel.com, kernel-team@fb.com, ning.sun@intel.com, srihan@fb.com, alex.eydelberg@intel.com Subject: Re: [PATCH V2] x86/tboot: add an option to disable iommu force on Message-ID: <20170427065142.lnsdegq7zwxacqo2@gmail.com> References: <1c2cadcf5cd7d19cea93c56435610e61b551bd1e.1493223474.git.shli@fb.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1c2cadcf5cd7d19cea93c56435610e61b551bd1e.1493223474.git.shli@fb.com> User-Agent: NeoMutt/20170113 (1.7.2) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Shaohua Li wrote: > IOMMU harms performance signficantly when we run very fast networking > workloads. It's 40GB networking doing XDP test. Software overhead is > almost unaware, but it's the IOTLB miss (based on our analysis) which > kills the performance. We observed the same performance issue even with > software passthrough (identity mapping), only the hardware passthrough > survives. The pps with iommu (with software passthrough) is only about > ~30% of that without it. This is a limitation in hardware based on our > observation, so we'd like to disable the IOMMU force on, but we do want > to use TBOOT and we can sacrifice the DMA security bought by IOMMU. I > must admit I know nothing about TBOOT, but TBOOT guys (cc-ed) think not > eabling IOMMU is totally ok. > > So introduce a new boot option to disable the force on. It's kind of > silly we need to run into intel_iommu_init even without force on, but we > need to disable TBOOT PMR registers. For system without the boot option, > nothing is changed. > > Signed-off-by: Shaohua Li > --- > Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++ > arch/x86/kernel/tboot.c | 3 +++ > drivers/iommu/intel-iommu.c | 18 ++++++++++++++++++ > include/linux/dma_remapping.h | 1 + > 4 files changed, 31 insertions(+) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index 33a3b54..8a3fb0d 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -1579,6 +1579,15 @@ > extended tables themselves, and also PASID support. With > this option set, extended tables will not be used even > on hardware which claims to support them. > + tboot_noforce [Default Off] > + Do not force the Intel IOMMU enabled under tboot. > + By default, tboot will force Intel IOMMU on, which > + could harm performance of some high-throughput > + devices like 40GBit network cards, even if identity > + mapping is enabled. > + Note that using this option lowers the security > + provided by tboot because it makes the system > + vulnerable to DMA attacks. So what's the purpose of this kernel option? It sure isn't the proper solution for correctly architectured hardware/firmware (which can just choose not to expose the IOMMU!), and for one-time hacks for special embedded systems or for debugging why not just add an iommu=off option to force it off? This just increases complexity for no good reason. Thanks, Ingo