From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13940C433E1 for ; Wed, 20 May 2020 18:16:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB0A220671 for ; Wed, 20 May 2020 18:16:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589998611; bh=S/zutDDmcZ7lnGu6DMFPWTBTbQiSF9FetCYn1L66JTM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=K9iHmh72gZWbX28LsfVJZdmMmEDd3VyL6OUTpLO56Qpx3o+0yU487KryP7Zrrbm1K kItsroVzRXiMUB28G7eg3LRPBpuUYmOKQw5D7P48SfRN3vmpO2X+ZHanuGyB7nHn2X ccrdP7y6rOb+GZc6t/JFgxNqoHWM8an/D2TjFe50= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726697AbgETSQs (ORCPT ); Wed, 20 May 2020 14:16:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:40268 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726548AbgETSQs (ORCPT ); Wed, 20 May 2020 14:16:48 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CD9512072C; Wed, 20 May 2020 18:16:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589998607; bh=S/zutDDmcZ7lnGu6DMFPWTBTbQiSF9FetCYn1L66JTM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=pgqQmiSdsDa1cocMgY6htah52kcScko7buW2f2rIeAOWQoBSyXAS59mPsVr9BHIwG UHdF7VnMGEYDUlBqDw9wrIPu51AQixA+PnR7vS4rjGJUTCJvQTLvgSpDUVzPOgZ4fm //ES2r7oQT+sAtZ+chw517ruLZtwA8etc7EP5wjw= Date: Wed, 20 May 2020 20:16:45 +0200 From: Greg Kroah-Hartman To: David Rientjes Cc: Christoph Hellwig , Peter Gonda , "Grimm, Jon" , Tom Lendacky , Brijesh Singh , stable@vger.kernel.org Subject: Re: DMA API stable backports for AMD SEV Message-ID: <20200520181645.GA339194@kroah.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org On Tue, May 19, 2020 at 05:41:15PM -0700, David Rientjes wrote: > Hi Greg and everyone, > > On all kernels, SEV enabled guests hit might_sleep() warnings when a > driver (nvme in this case) allocates through the DMA API in a > non-blockable context: > > BUG: sleeping function called from invalid context at mm/vmalloc.c:1710 > in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 3383, name: fio > 2 locks held by fio/3383: > #0: ffff93b6a8568348 (&sb->s_type->i_mutex_key#16){+.+.}, at: ext4_file_write_iter+0xa2/0x5d0 > #1: ffffffffa52a61a0 (rcu_read_lock){....}, at: hctx_lock+0x1a/0xe0 > CPU: 0 PID: 3383 Comm: fio Tainted: G W 5.5.10 #14 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 > Call Trace: > dump_stack+0x98/0xd5 > ___might_sleep+0x175/0x260 > __might_sleep+0x4a/0x80 > _vm_unmap_aliases+0x45/0x250 > vm_unmap_aliases+0x19/0x20 > __set_memory_enc_dec+0xa4/0x130 > set_memory_decrypted+0x10/0x20 > dma_direct_alloc_pages+0x148/0x150 > dma_direct_alloc+0xe/0x10 > dma_alloc_attrs+0x86/0xc0 > dma_pool_alloc+0x16f/0x2b0 > nvme_queue_rq+0x878/0xc30 [nvme] > __blk_mq_try_issue_directly+0x135/0x200 > blk_mq_request_issue_directly+0x4f/0x80 > blk_mq_try_issue_list_directly+0x46/0xb0 > blk_mq_sched_insert_requests+0x19b/0x2b0 > blk_mq_flush_plug_list+0x22f/0x3b0 > blk_flush_plug_list+0xd1/0x100 > blk_finish_plug+0x2c/0x40 > iomap_dio_rw+0x427/0x490 > ext4_file_write_iter+0x181/0x5d0 > aio_write+0x109/0x1b0 > io_submit_one+0x7d0/0xfa0 > __x64_sys_io_submit+0xa2/0x280 > do_syscall_64+0x5f/0x250 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > There is a series of patches in Christoph's dma-mapping.git repo in the > for-next branch on track for 5.8: > > 1d659236fb43 dma-pool: scale the default DMA coherent pool size with memory capacity > 82fef0ad811f x86/mm: unencrypted non-blocking DMA allocations use coherent pools > 2edc5bb3c5cc dma-pool: add pool sizes to debugfs > 76a19940bd62 dma-direct: atomic allocations must come from atomic coherent pools > 54adadf9b085 dma-pool: dynamically expanding atomic pools > c84dc6e68a1d dma-pool: add additional coherent pools to map to gfp mask > e860c299ac0d dma-remap: separate DMA atomic pools from direct remap code > > We'd like to prepare backports to LTS kernels so that our guest images are > not modified by us and don't exhibit this issue. > > They are bigger than we'd like: > > arch/x86/Kconfig | 1 + > drivers/iommu/dma-iommu.c | 5 +- > include/linux/dma-direct.h | 2 + > include/linux/dma-mapping.h | 6 +- > kernel/dma/Kconfig | 6 +- > kernel/dma/Makefile | 1 + > kernel/dma/direct.c | 56 ++++++-- > kernel/dma/pool.c | 264 ++++++++++++++++++++++++++++++++++++ > kernel/dma/remap.c | 121 +---------------- > 9 files changed, 324 insertions(+), 138 deletions(-) > create mode 100644 kernel/dma/pool.c > > But they apply relatively cleanly to more modern kernels like 5.4. We'd > like to backport these all the way to 4.19, however, otherwise guests > encounter these bugs. > > The changes to kernel/dma/remap.c, for example, simply moves code to the > new pool.c. But that original code is actually in arch/arm64 in 4.19 and > was moved in 5.0: > > commit 0c3b3171ceccb8830c2bb5adff1b4e9b204c1450 > Author: Christoph Hellwig > Date: Sun Nov 4 20:29:28 2018 +0100 > > dma-mapping: move the arm64 noncoherent alloc/free support to common code > > commit f0edfea8ef93ed6cc5f747c46c85c8e53e0798a0 > Author: Christoph Hellwig > Date: Fri Aug 24 10:31:08 2018 +0200 > > dma-mapping: move the remap helpers to a separate file > > And there are most certainly more dependencies to get a cleanly applying > series to 4.19.123. So the backports could be quite extensive. > > Peter Gonda is currently handling these and we're > looking for advice: should we compile a full list of required backports > that would be needed to get a series that would only consist of minor > conflicts or is this going to be a non-starter? A full series would be good. Once these hit Linus's tree and show up in a -rc or two, feel free to send on the backports and we can look at them then. thanks, greg k-h