linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Ard Biesheuvel <ardb@kernel.org>
To: linux-crypto@vger.kernel.org
Cc: mark.rutland@arm.com, herbert@gondor.apana.org.au,
	Eric Biggers <ebiggers@google.com>,
	catalin.marinas@arm.com, Dave Martin <dave.martin@arm.com>,
	will@kernel.org, Ard Biesheuvel <ardb@kernel.org>,
	linux-arm-kernel@lists.infradead.org
Subject: [PATCH 1/9] arm64: assembler: add cond_yield macro
Date: Thu, 28 Jan 2021 14:06:17 +0100	[thread overview]
Message-ID: <20210128130625.54076-2-ardb@kernel.org> (raw)
In-Reply-To: <20210128130625.54076-1-ardb@kernel.org>

Add a macro cond_yield that branches to a specified label when called if
the TIF_NEED_RESCHED flag is set and decreasing the preempt count would
make the task preemptible again, resulting in a schedule to occur. This
can be used by kernel mode SIMD code that keeps a lot of state in SIMD
registers, which would make chunking the input in order to perform the
cond_resched() check from C code disproportionately costly.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/include/asm/assembler.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
index bf125c591116..5f977a7c6b43 100644
--- a/arch/arm64/include/asm/assembler.h
+++ b/arch/arm64/include/asm/assembler.h
@@ -745,6 +745,22 @@ USER(\label, ic	ivau, \tmp2)			// invalidate I line PoU
 .Lyield_out_\@ :
 	.endm
 
+	/*
+	 * Check whether preempt-disabled code should yield as soon as it
+	 * is able. This is the case if re-enabling preemption a single
+	 * time results in a preempt count of zero, and the TIF_NEED_RESCHED
+	 * flag is set. (Note that the latter is stored negated in the
+	 * top word of the thread_info::preempt_count field)
+	 */
+	.macro		cond_yield, lbl:req, tmp:req
+#ifdef CONFIG_PREEMPTION
+	get_current_task \tmp
+	ldr		\tmp, [\tmp, #TSK_TI_PREEMPT]
+	cmp		\tmp, #PREEMPT_DISABLE_OFFSET
+	beq		\lbl
+#endif
+	.endm
+
 /*
  * This macro emits a program property note section identifying
  * architecture features which require special handling, mainly for
-- 
2.29.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2021-01-28 13:08 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-28 13:06 [PATCH 0/9] arm64: rework NEON yielding to avoid scheduling from asm code Ard Biesheuvel
2021-01-28 13:06 ` Ard Biesheuvel [this message]
2021-01-28 20:24   ` [PATCH 1/9] arm64: assembler: add cond_yield macro Will Deacon
2021-01-28 20:25     ` Will Deacon
2021-01-28 20:26       ` Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 2/9] crypto: arm64/sha1-ce - simplify NEON yield Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 3/9] crypto: arm64/sha2-ce " Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 4/9] crypto: arm64/sha3-ce " Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 5/9] crypto: arm64/sha512-ce " Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 6/9] crypto: arm64/aes-neonbs - remove NEON yield calls Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 7/9] crypto: arm64/aes-ce-mac - simplify NEON yield Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 8/9] crypto: arm64/crc-t10dif - move NEON yield to C code Ard Biesheuvel
2021-01-28 13:06 ` [PATCH 9/9] arm64: assembler: remove conditional NEON yield macros Ard Biesheuvel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210128130625.54076-2-ardb@kernel.org \
    --to=ardb@kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=dave.martin@arm.com \
    --cc=ebiggers@google.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-crypto@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).