From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D5F0C43603 for ; Fri, 20 Dec 2019 18:51:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 12CD320866 for ; Fri, 20 Dec 2019 18:51:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="R4OJMfvk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12CD320866 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B7CE08E01B6; Fri, 20 Dec 2019 13:51:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B55678E019D; Fri, 20 Dec 2019 13:51:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A90FB8E01B6; Fri, 20 Dec 2019 13:51:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id 925C68E019D for ; Fri, 20 Dec 2019 13:51:03 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 531F3180AD81A for ; Fri, 20 Dec 2019 18:51:03 +0000 (UTC) X-FDA: 76286412006.23.veil84_7d9488a969c29 X-HE-Tag: veil84_7d9488a969c29 X-Filterd-Recvd-Size: 8047 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Fri, 20 Dec 2019 18:51:02 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id c6so4106936wrm.18 for ; Fri, 20 Dec 2019 10:51:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MFWtbCdLFKvgTQnCDajtn6la3N66KGU5yNNvG/61cdA=; b=R4OJMfvkFEzKmiLpUg8ebUMmAa2064EkZTO6u8p06KiIqUvtQYB6gllRhlHS2EpLce tO4RMoM00TclV/BmSVqnolwNXAiVxFtOajWHlWbplyeh/B7u3pIaURd2KcXcoKT/0+xi l8K06L4LgbRbLNe30rg4arbHp3CJo1box9hhB7wq1F6pMpswwHkCyS6PMcUEglfxg6Hx B41O+orjF2Im+q9ok86DdOFVDJUu11nXrv+C8pgkEWb0npytiJKW/3S5cQUEgUWelRKQ gQcEcpgHamaJxA+jAXhwih/svMRucZe8JG7S80tVM9XvpJj5xi5lY1KDvLCTPNJOkSV6 GzpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MFWtbCdLFKvgTQnCDajtn6la3N66KGU5yNNvG/61cdA=; b=Z0feZEy9jd46tb8p6riTABBmia9jdYQWYVS+JldvdpdqSnu/sfX7k7ftwO7SQGXG9C Q0Y1RPY1LUGTgnZ+0GCpxBM2Y6YxFR7gEMSDW+15LmAyMefJ+9xVMYNJRVUoBoOHXYYz 6FB7k7FsNC94fWSIbRy4EEGMcG6rrglURs1u4HGrkpuJ56Tp0YiS++yRlUJXOyCE4MF7 wn0C+M6VWg6EU2f47nGt/nTfAr+eDzzAqkpOOxqzFPPiVOmXWAD0jF7TpvNiDYstfL3s 0hEHr+fDEd36Tu3NUAeS/rioEsO5D5xtbEbA/L/Vqu4C9txsHjROVsCNyF6eNn+kZChD f3SA== X-Gm-Message-State: APjAAAXQLsE+DPw6+otMvLABy9/+ccLALjIXDRnPpIXl03pbl6iEZ3hq EKyl3zOA0f+C8bGcUTiVTJgLZDA+jX4= X-Google-Smtp-Source: APXvYqw+Tfg9BLMyz7voeK2zLF8rrUyPwjV9HsySqZWhVoblo//XGBpQN3D5P1yXHmRtuF4G2g6hVrYSEBg= X-Received: by 2002:adf:82a7:: with SMTP id 36mr17609322wrc.203.1576867861376; Fri, 20 Dec 2019 10:51:01 -0800 (PST) Date: Fri, 20 Dec 2019 19:49:32 +0100 In-Reply-To: <20191220184955.223741-1-glider@google.com> Message-Id: <20191220184955.223741-20-glider@google.com> Mime-Version: 1.0 References: <20191220184955.223741-1-glider@google.com> X-Mailer: git-send-email 2.24.1.735.g03f4e72817-goog Subject: [PATCH RFC v4 19/42] kmsan: x86/asm: softirq: add KMSAN IRQ entry hooks From: glider@google.com To: Jens Axboe , Andy Lutomirski , Vegard Nossum , Dmitry Vyukov , Marco Elver , Andrey Konovalov , Christoph Hellwig , linux-mm@kvack.org Cc: glider@google.com, viro@zeniv.linux.org.uk, adilger.kernel@dilger.ca, akpm@linux-foundation.org, aryabinin@virtuozzo.com, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@infradead.org, darrick.wong@oracle.com, davem@davemloft.net, dmitry.torokhov@gmail.com, ebiggers@google.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, iii@linux.ibm.com, mingo@elte.hu, jasowang@redhat.com, m.szyprowski@samsung.com, mark.rutland@arm.com, martin.petersen@oracle.com, schwidefsky@de.ibm.com, willy@infradead.org, mst@redhat.com, mhocko@suse.com, monstr@monstr.eu, pmladek@suse.com, cai@lca.pw, rdunlap@infradead.org, robin.murphy@arm.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, gor@linux.ibm.com, wsa@the-dreams.de Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add assembly helpers to entry_64.S that invoke hooks from kmsan_entry.c and notify KMSAN about interrupts. Also call these hooks from kernel/softirq.c This is needed to switch between several KMSAN contexts holding function parameter metadata. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Jens Axboe Cc: Andy Lutomirski Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Marco Elver Cc: Andrey Konovalov Cc: Christoph Hellwig Cc: linux-mm@kvack.org --- v4: - moved softirq changes to this patch Change-Id: I3037d51672fe69d09e588b27adb2d9fdc6ad3a7d --- arch/x86/entry/entry_64.S | 16 ++++++++++++++++ kernel/softirq.c | 5 +++++ 2 files changed, 21 insertions(+) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 76942cbd95a1..293922680a39 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -575,6 +576,7 @@ SYM_CODE_START(interrupt_entry) 1: ENTER_IRQ_STACK old_rsp=%rdi save_ret=1 + KMSAN_INTERRUPT_ENTER /* We entered an interrupt context - irqs are off: */ TRACE_IRQS_OFF @@ -604,12 +606,14 @@ SYM_CODE_START_LOCAL(common_interrupt) addq $-0x80, (%rsp) /* Adjust vector to [-256, -1] range */ call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call do_IRQ /* rdi points to pt_regs */ /* 0(%rsp): old RSP */ ret_from_intr: DISABLE_INTERRUPTS(CLBR_ANY) TRACE_IRQS_OFF + KMSAN_INTERRUPT_EXIT LEAVE_IRQ_STACK testb $3, CS(%rsp) @@ -801,6 +805,7 @@ SYM_CODE_START(\sym) .Lcommon_\sym: call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call \do_sym /* rdi points to pt_regs */ jmp ret_from_intr SYM_CODE_END(\sym) @@ -908,15 +913,18 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt .if \shift_ist != -1 subq $\ist_offset, CPU_TSS_IST(\shift_ist) + KMSAN_IST_ENTER(\shift_ist) .endif .if \read_cr2 movq %r12, %rdx /* Move CR2 into 3rd argument */ .endif + KMSAN_UNPOISON_PT_REGS call \do_sym .if \shift_ist != -1 + KMSAN_IST_EXIT(\shift_ist) addq $\ist_offset, CPU_TSS_IST(\shift_ist) .endif @@ -1079,7 +1087,9 @@ SYM_FUNC_START(do_softirq_own_stack) pushq %rbp mov %rsp, %rbp ENTER_IRQ_STACK regs=0 old_rsp=%r11 + KMSAN_SOFTIRQ_ENTER call __do_softirq + KMSAN_SOFTIRQ_EXIT LEAVE_IRQ_STACK regs=0 leaveq ret @@ -1466,9 +1476,12 @@ SYM_CODE_START(nmi) * done with the NMI stack. */ + KMSAN_NMI_ENTER movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* * Return back to user mode. We must *not* do the normal exit @@ -1678,10 +1691,13 @@ end_repeat_nmi: call paranoid_entry UNWIND_HINT_REGS + KMSAN_NMI_ENTER /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */ movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* Always restore stashed CR3 value (see paranoid_entry) */ RESTORE_CR3 scratch_reg=%r15 save_reg=%r14 diff --git a/kernel/softirq.c b/kernel/softirq.c index 0427a86743a4..98c5f4062cbf 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -370,7 +371,9 @@ static inline void invoke_softirq(void) * it is the irq stack, because it should be near empty * at this stage. */ + kmsan_context_enter(); __do_softirq(); + kmsan_context_exit(); #else /* * Otherwise, irq_exit() is called on the task stack that can @@ -600,7 +603,9 @@ static void run_ksoftirqd(unsigned int cpu) * We can safely run softirq on inline stack, as we are not deep * in the task stack here. */ + kmsan_context_enter(); __do_softirq(); + kmsan_context_exit(); local_irq_enable(); cond_resched(); return; -- 2.24.1.735.g03f4e72817-goog