* [RFC PATCH v2] IV Generation algorithms for dm-crypt
@ 2016-12-13 8:49 Binoy Jayan
2016-12-13 8:49 ` [RFC PATCH v2] crypto: Add IV generation algorithms Binoy Jayan
0 siblings, 1 reply; 17+ messages in thread
From: Binoy Jayan @ 2016-12-13 8:49 UTC (permalink / raw)
To: Oded, Ofir
Cc: Herbert Xu, David S. Miller, linux-crypto, Mark Brown,
Arnd Bergmann, linux-kernel, Alasdair Kergon, Mike Snitzer,
dm-devel, Shaohua Li, linux-raid, Rajendra, Binoy Jayan
===============================================================================
GENIV Template cipher
===============================================================================
Currently, the iv generation algorithms are implemented in dm-crypt.c. The goal
is to move these algorithms from the dm layer to the kernel crypto layer by
implementing them as template ciphers so they can be used in relation with
algorithms like aes, and with multiple modes like cbc, ecb etc. As part of this
patchset, the iv-generation code is moved from the dm layer to the crypto layer
and adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
at a time. Each bio contains the in memory representation of physically
contiguous disk blocks. Since the bio itself may not be contiguous in main
memory, the dm layer sets up a chained scatterlist of these blocks split into
physically contiguous segments in memory so that DMA can be performed.
v1 --> v2
----------
1. dm-crypt changes to process larger block sizes (one segment in a bio)
2. Incorporated changes w.r.t. comments from Herbert.
v1: https://patchwork.kernel.org/patch/9439175
One challenge in doing so is that the IVs are generated based on a 512-byte
sector number. This infact limits the block sizes to 512 bytes. But this should
not be a problem if a hardware with iv generation support is used. The geniv
itself splits the segments into sectors so it could choose the IV based on
sector number. But it could be modelled in hardware effectively by not
splitting up the segments in the bio.
Another challenge faced is that dm-crypt has an option to use multiple keys.
The key selection is done based on the sector number. If the whole bio is
encrypted / decrypted with the same key, the encrypted volumes will not be
compatible with the original dm-crypt [without the changes].
The following ASCII art decomposes the kernel crypto API layers when using the
skcipher with the automated IV generation. The shown example is used by the
DM layer. For other use cases of cbc(aes), the ASCII art applies as well, but
the caller may not use the same with a separate IV generator. In this case, the
caller must generate the IV. The depicted example decomposes <ivgen>(cbc(aes))
based on the generic C implementations (geniv.c, cbc.c and aes-generic.c).
The generic implementation depicts the dependency between the templates ciphers
used in implementing geniv using the kernel crypto API.
Here, <geniv> indicates one of the following algorithms:
1. plain
2. plain64
3. essiv
4. benbi
5. null
6. lmk
7. tcw
It is possible that some streamlined cipher implementations (like AES-NI)
provide implementations merging aspects which in the view of the kernel crypto
API cannot be decomposed into layers any more. Each block in the following
ASCII art is an independent cipher instance obtained from the kernel crypto
API. Each block is accessed by the caller or by other blocks using the API
functions defined by the kernel crypto API for the cipher implementation type.
The blocks below indicate the cipher type as well as the specific logic
implemented in the cipher.
The ASCII art picture also indicates the call structure, i.e. who calls which
component. The arrows point to the invoked block where the caller uses the API
applicable to the cipher type specified for the block. For the purpose of
illustration, here we take the example of the aes mode 'cbc'. However, the IV
generation algorithm could be used with other aes modes like ecb as well.
-------------------------------------------------------------------------------
Geniv implementation
-------------------------------------------------------------------------------
NB: The ASCII art below is best viewed in a fixed-width font.
crypt_convert_block() (DM Layer)
|
| (1)
|
v
+------------+ +-----------+ +-----------+ +-----------+
| | | | | | (2) | |
| skcipher | | skcipher | | skcipher |----+ | skcipher | Blocks for
| (plain/64) | | (benbi) | | (essiv) | | | (null) | lmk, tcw
+------------+ +-----------+ +-----------+ | +-----------+
| | | v |
| (3) | (3) (3) | +-----------+ |
| | | | | |
| | | | ahash | | (3)
| | | | | |
| | | +-----------+ |
| | v | (Crypto API
| | +-----------+ | Layer)
| v | | |
+------------------------> | skcipher | <-------------+
| (cbc) |
+-----------+ (AES Mode Template cipher)
| (4)
v
+-----------+
| |
| cipher | (Base generic-AES cipher)
| (aes) |
+-----------+
The following call sequence is applicable when the DM layer triggers an
encryption operation with the crypt_convert_block() function. During
configuration, the administrator sets up the use of <geniv>(cbc(aes)) as the
template cipher. 'geniv' can be one among plain, plain64, essiv, benbi, null,
lmk, or tcw which are all implemented as seperate templates.
The following are the template ciphers implemented as part of 'geniv.c'
1. plain(cbc(aes))
2. plain64(cbc(aes))
3. essiv(cbc(aes))
4. benbi(cbc(aes))
5. null(cbc(aes))
6. lmk(cbc(aes))
7. tcw(cbc(aes))
The following call sequence is now depicted in the ASCII art above:
1. crypt_convert_block invokes crypto_skcipher_encrypt() to trigger encryption
operation of a single block (i.e. sector) with the IV same as the sector no.
For example, with essiv, the IV generation implementation is registered with
a call to 'crypto_register_template(&crypto_essiv_tmpl)'
2. During instantiation of the 'geniv' handle, the IV generation algorithm is
instantiated. For the purpose of illustration, we take the example of essiv.
In this case, the ahash cipher is instantiated to calculate the hash of the
sector to generate the IV.
3. Now, geniv uses the skcipher api calls to invoke the associated cipher. In
our case, during the instantiation of geniv, the cipher handle for cbc is
provided to geniv. The geniv skcipher type implementation now invokes the
skcipher api with the instantiated cbc(aes) cipher handle. During the
instantiation of the cbc(aes) cipher, the cipher type generic-aes is also
instantiated. That means that the SKCIPHER implementation of cbc(aes) only
implements the Cipher-block chaining mode. After performing block chaining
operation, the cipher implementation of aes is invoked. The skcipher of
cbc(aes) now invokes the cipher api with the aes cipher handle to encrypt
one block.
-------------------------------------------------------------------------------
To be discussed
-------------------------------------------------------------------------------
1. Changes to testmgr.c
2. How to use multiple keys.
When using multiple keys with the original dm-crypt, the key selection is
made based on the sector number as:
key_index = sector & (key_count - 1)
This restricts the usage of the same key for encrypting/decrypting a single
bio. One way to solve this is to move the key management code from dm-crypt
to cryto layer. But this seems tricky when using template ciphers because,
when multiple ciphers are instantiated from dm layer, each cipher instance
set with a unique subkey (part of the bigger master key) and these instances
themselves do not have access to each other's instances or contexts. This
way, a single instance cannot encryt/decrypt a whole bio. I have not been
able to get around this problem.
-------------------------------------------------------------------------------
Test procedure
-------------------------------------------------------------------------------
The algorithms are tested using 'cryptsetup' utility to create LUKS
compatible volumes on Qemu.
NB: '/dev/sdb' is a second disk volume (configured in qemu)
# One time setup - Format the device compatible with LUKS.
# Choose one of the following IV generation alorithms at a time
cryptsetup -y -c aes-cbc-plain -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes-cbc-plain64 -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes-cbc-essiv:sha256 -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes-cbc-benbi -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes-cbc-null -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes-cbc-lmk -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes-cbc-tcw -s 256 --hash sha256 luksFormat /dev/sdb
# With a keycount >= 2
# These were tested ok but not compatible with the older dm-crypt
# because the keys were selected based on the IV number when multiple
# keys are involved. This needs to be fixed.
cryptsetup -y -c aes:2-cbc-plain -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes:2-cbc-plain64 -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes:2-cbc-essiv:sha256 -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes:2-cbc-null -s 256 --hash sha256 luksFormat /dev/sdb
cryptsetup -y -c aes:2-cbc-lmk -s 256 --hash sha256 luksFormat /dev/sdb
# Add additional key - optional
cryptsetup luksAddKey /dev/sdb
# The above lists only a limited number of tests with the aes cipher.
# The IV generation algorithms may also be tested with other ciphers as well.
cryptsetup luksDump --dump-master-key /dev/sdb
# create a luks volume and open the device
cryptsetup luksOpen /dev/sdb crypt_fun
dmsetup table --showkeys
# Write some data to the device
cat data.txt > /dev/mapper/crypt_fun
# Read 100 bytes back
dd if=/dev/mapper/crypt_fun of=out.txt bs=100 count=1
cat out.txt
mkfs.ext4 -j /dev/mapper/crypt_fun
# Mount if fs creation succeeds
mount -t ext4 /dev/mapper/crypt_fun /mnt
<-- Use the encrypted file system -->
umount /mnt
cryptsetup luksClose crypt_fun
cryptsetup luksRemoveKey /dev/sdb
This seems to work well. The file system mounts successfully and the files
written to in the file system remain persistent across reboots.
Binoy Jayan (1):
crypto: Add IV generation algorithms
crypto/Kconfig | 10 +
crypto/Makefile | 1 +
crypto/geniv.c | 1294 ++++++++++++++++++++++++++++++++++++++++++++++++
drivers/md/dm-crypt.c | 894 +++++----------------------------
include/crypto/geniv.h | 60 +++
5 files changed, 1499 insertions(+), 760 deletions(-)
create mode 100644 crypto/geniv.c
create mode 100644 include/crypto/geniv.h
--
Binoy Jayan
^ permalink raw reply [flat|nested] 17+ messages in thread
* [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 8:49 [RFC PATCH v2] IV Generation algorithms for dm-crypt Binoy Jayan
@ 2016-12-13 8:49 ` Binoy Jayan
2016-12-13 10:01 ` Milan Broz
` (2 more replies)
0 siblings, 3 replies; 17+ messages in thread
From: Binoy Jayan @ 2016-12-13 8:49 UTC (permalink / raw)
To: Oded, Ofir
Cc: Herbert Xu, David S. Miller, linux-crypto, Mark Brown,
Arnd Bergmann, linux-kernel, Alasdair Kergon, Mike Snitzer,
dm-devel, Shaohua Li, linux-raid, Rajendra, Binoy Jayan
Currently, the iv generation algorithms are implemented in dm-crypt.c.
The goal is to move these algorithms from the dm layer to the kernel
crypto layer by implementing them as template ciphers so they can be
implemented in hardware for performance. As part of this patchset, the
iv-generation code is moved from the dm layer to the crypto layer and
adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
at a time. Each bio contains the in memory representation of physically
contiguous disk blocks. The dm layer sets up a chained scatterlist of
these blocks split into physically contiguous segments in memory so that
DMA can be performed. The iv generation algorithms implemented in geniv.c
include plain, plain64, essiv, benbi, null, lmk and tcw.
When using multiple keys with the original dm-crypt, the key selection is
made based on the sector number as:
key_index = sector & (key_count - 1)
This restricts the usage of the same key for encrypting/decrypting a
single bio. One way to solve this is to move the key management code from
dm-crypt to cryto layer. But this seems tricky when using template ciphers
because, when multiple ciphers are instantiated from dm layer, each cipher
instance set with a unique subkey (part of the bigger master key) and
these instances themselves do not have access to each other's instances
or contexts. This way, a single instance cannot encryt/decrypt a whole bio.
This has to be fixed.
Not-signed-off-by: Binoy Jayan <binoy.jayan@linaro.org>
---
crypto/Kconfig | 10 +
crypto/Makefile | 1 +
crypto/geniv.c | 1294 ++++++++++++++++++++++++++++++++++++++++++++++++
drivers/md/dm-crypt.c | 894 +++++----------------------------
include/crypto/geniv.h | 60 +++
5 files changed, 1499 insertions(+), 760 deletions(-)
create mode 100644 crypto/geniv.c
create mode 100644 include/crypto/geniv.h
diff --git a/crypto/Kconfig b/crypto/Kconfig
index 84d7148..dc33a33 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -326,6 +326,16 @@ config CRYPTO_CTS
This mode is required for Kerberos gss mechanism support
for AES encryption.
+config CRYPTO_GENIV
+ tristate "IV Generation for dm-crypt"
+ select CRYPTO_BLKCIPHER
+ help
+ GENIV: IV Generation for dm-crypt
+ Algorithms to generate Initialization Vector for ciphers
+ used by dm-crypt. The iv generation algorithms implemented
+ as part of geniv include plain, plain64, essiv, benbi, null,
+ lmk and tcw.
+
config CRYPTO_ECB
tristate "ECB support"
select CRYPTO_BLKCIPHER
diff --git a/crypto/Makefile b/crypto/Makefile
index bd6a029..627ec76 100644
--- a/crypto/Makefile
+++ b/crypto/Makefile
@@ -75,6 +75,7 @@ obj-$(CONFIG_CRYPTO_TGR192) += tgr192.o
obj-$(CONFIG_CRYPTO_GF128MUL) += gf128mul.o
obj-$(CONFIG_CRYPTO_ECB) += ecb.o
obj-$(CONFIG_CRYPTO_CBC) += cbc.o
+obj-$(CONFIG_CRYPTO_GENIV) += geniv.o
obj-$(CONFIG_CRYPTO_PCBC) += pcbc.o
obj-$(CONFIG_CRYPTO_CTS) += cts.o
obj-$(CONFIG_CRYPTO_LRW) += lrw.o
diff --git a/crypto/geniv.c b/crypto/geniv.c
new file mode 100644
index 0000000..ac81a49
--- /dev/null
+++ b/crypto/geniv.c
@@ -0,0 +1,1294 @@
+/*
+ * geniv: IV generation algorithms
+ *
+ * Copyright (c) 2016, Linaro Ltd.
+ * Copyright (C) 2006-2015 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2013 Milan Broz <gmazyland@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+
+#include <crypto/algapi.h>
+#include <crypto/internal/skcipher.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/log2.h>
+#include <linux/module.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
+#include <linux/completion.h>
+#include <linux/crypto.h>
+#include <linux/workqueue.h>
+#include <linux/backing-dev.h>
+#include <linux/atomic.h>
+#include <linux/rbtree.h>
+#include <crypto/hash.h>
+#include <crypto/md5.h>
+#include <crypto/algapi.h>
+#include <crypto/skcipher.h>
+#include <asm/unaligned.h>
+#include <crypto/geniv.h>
+
+struct geniv_ctx;
+struct crypto_geniv_req_ctx;
+
+/* Sub request for each of the skcipher_request's for a segment */
+struct crypto_geniv_subreq {
+ struct skcipher_request req CRYPTO_MINALIGN_ATTR;
+ struct crypto_geniv_req_ctx *rctx;
+};
+
+struct crypto_geniv_req_ctx {
+ struct crypto_geniv_subreq *subreqs;
+ struct scatterlist *src;
+ struct scatterlist *dst;
+ bool is_write;
+ sector_t iv_sector;
+ unsigned int nents;
+ u8 *iv;
+ struct completion restart;
+ atomic_t req_pending;
+ struct skcipher_request *req;
+};
+
+struct geniv_operations {
+ int (*ctr)(struct geniv_ctx *ctx);
+ void (*dtr)(struct geniv_ctx *ctx);
+ int (*init)(struct geniv_ctx *ctx);
+ int (*wipe)(struct geniv_ctx *ctx);
+ int (*generator)(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n);
+ int (*post)(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n);
+};
+
+struct geniv_essiv_private {
+ struct crypto_ahash *hash_tfm;
+ u8 *salt;
+};
+
+struct geniv_benbi_private {
+ int shift;
+};
+
+struct geniv_lmk_private {
+ struct crypto_shash *hash_tfm;
+ u8 *seed;
+};
+
+struct geniv_tcw_private {
+ struct crypto_shash *crc32_tfm;
+ u8 *iv_seed;
+ u8 *whitening;
+};
+
+struct geniv_ctx {
+ struct crypto_skcipher *child;
+ unsigned int tfms_count;
+ char *ivmode;
+ unsigned int iv_size;
+ char *ivopts;
+ char *cipher;
+ struct geniv_operations *iv_gen_ops;
+ union {
+ struct geniv_essiv_private essiv;
+ struct geniv_benbi_private benbi;
+ struct geniv_lmk_private lmk;
+ struct geniv_tcw_private tcw;
+ } iv_gen_private;
+ void *iv_private;
+ struct crypto_skcipher *tfm;
+ unsigned int key_size;
+ unsigned int key_extra_size;
+ unsigned int key_parts; /* independent parts in key buffer */
+ enum setkey_op keyop;
+ char *msg;
+ u8 *key;
+};
+
+static struct crypto_skcipher *any_tfm(struct geniv_ctx *ctx)
+{
+ return ctx->tfm;
+}
+
+static inline
+struct crypto_geniv_req_ctx *geniv_req_ctx(struct skcipher_request *req)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ unsigned long align = crypto_skcipher_alignmask(tfm);
+
+ return (void *) PTR_ALIGN((u8 *)skcipher_request_ctx(req), align + 1);
+}
+
+static int crypt_iv_plain_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *iv = rctx->iv;
+
+ memset(iv, 0, ctx->iv_size);
+ *(__le32 *)iv = cpu_to_le32(rctx->iv_sector & 0xffffffff);
+
+ return 0;
+}
+
+static int crypt_iv_plain64_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *iv = rctx->iv;
+
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
+
+ return 0;
+}
+
+/* Initialise ESSIV - compute salt but no local memory allocations */
+static int crypt_iv_essiv_init(struct geniv_ctx *ctx)
+{
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+ struct scatterlist sg;
+ struct crypto_cipher *essiv_tfm;
+ int err;
+ AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
+
+ sg_init_one(&sg, ctx->key, ctx->key_size);
+ ahash_request_set_tfm(req, essiv->hash_tfm);
+ ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
+ ahash_request_set_crypt(req, &sg, essiv->salt, ctx->key_size);
+
+ err = crypto_ahash_digest(req);
+ ahash_request_zero(req);
+ if (err)
+ return err;
+
+ essiv_tfm = ctx->iv_private;
+
+ err = crypto_cipher_setkey(essiv_tfm, essiv->salt,
+ crypto_ahash_digestsize(essiv->hash_tfm));
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/* Wipe salt and reset key derived from volume key */
+static int crypt_iv_essiv_wipe(struct geniv_ctx *ctx)
+{
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+ unsigned int salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
+ struct crypto_cipher *essiv_tfm;
+ int r, err = 0;
+
+ memset(essiv->salt, 0, salt_size);
+
+ essiv_tfm = ctx->iv_private;
+ r = crypto_cipher_setkey(essiv_tfm, essiv->salt, salt_size);
+ if (r)
+ err = r;
+
+ return err;
+}
+
+/* Set up per cpu cipher state */
+static struct crypto_cipher *setup_essiv_cpu(struct geniv_ctx *ctx,
+ u8 *salt, unsigned int saltsize)
+{
+ struct crypto_cipher *essiv_tfm;
+ int err;
+
+ /* Setup the essiv_tfm with the given salt */
+ essiv_tfm = crypto_alloc_cipher(ctx->cipher, 0, CRYPTO_ALG_ASYNC);
+
+ if (IS_ERR(essiv_tfm)) {
+ pr_err("Error allocating crypto tfm for ESSIV\n");
+ return essiv_tfm;
+ }
+
+ if (crypto_cipher_blocksize(essiv_tfm) !=
+ crypto_skcipher_ivsize(any_tfm(ctx))) {
+ pr_err("Block size of ESSIV cipher does not match IV size of block cipher\n");
+ crypto_free_cipher(essiv_tfm);
+ return ERR_PTR(-EINVAL);
+ }
+
+ err = crypto_cipher_setkey(essiv_tfm, salt, saltsize);
+ if (err) {
+ pr_err("Failed to set key for ESSIV cipher\n");
+ crypto_free_cipher(essiv_tfm);
+ return ERR_PTR(err);
+ }
+ return essiv_tfm;
+}
+
+static void crypt_iv_essiv_dtr(struct geniv_ctx *ctx)
+{
+ struct crypto_cipher *essiv_tfm;
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+
+ crypto_free_ahash(essiv->hash_tfm);
+ essiv->hash_tfm = NULL;
+
+ kzfree(essiv->salt);
+ essiv->salt = NULL;
+
+ essiv_tfm = ctx->iv_private;
+
+ if (essiv_tfm)
+ crypto_free_cipher(essiv_tfm);
+
+ ctx->iv_private = NULL;
+}
+
+static int crypt_iv_essiv_ctr(struct geniv_ctx *ctx)
+{
+ struct crypto_cipher *essiv_tfm = NULL;
+ struct crypto_ahash *hash_tfm = NULL;
+ u8 *salt = NULL;
+ int err;
+
+ if (!ctx->ivopts) {
+ pr_err("Digest algorithm missing for ESSIV mode\n");
+ return -EINVAL;
+ }
+
+ /* Allocate hash algorithm */
+ hash_tfm = crypto_alloc_ahash(ctx->ivopts, 0, CRYPTO_ALG_ASYNC);
+ if (IS_ERR(hash_tfm)) {
+ err = PTR_ERR(hash_tfm);
+ pr_err("Error initializing ESSIV hash. err=%d\n", err);
+ goto bad;
+ }
+
+ salt = kzalloc(crypto_ahash_digestsize(hash_tfm), GFP_KERNEL);
+ if (!salt) {
+ err = -ENOMEM;
+ goto bad;
+ }
+
+ ctx->iv_gen_private.essiv.salt = salt;
+ ctx->iv_gen_private.essiv.hash_tfm = hash_tfm;
+
+ essiv_tfm = setup_essiv_cpu(ctx, salt,
+ crypto_ahash_digestsize(hash_tfm));
+ if (IS_ERR(essiv_tfm)) {
+ crypt_iv_essiv_dtr(ctx);
+ return PTR_ERR(essiv_tfm);
+ }
+ ctx->iv_private = essiv_tfm;
+
+ return 0;
+
+bad:
+ if (hash_tfm && !IS_ERR(hash_tfm))
+ crypto_free_ahash(hash_tfm);
+ kfree(salt);
+ return err;
+}
+
+static int crypt_iv_essiv_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *iv = rctx->iv;
+ struct crypto_cipher *essiv_tfm = ctx->iv_private;
+
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
+ crypto_cipher_encrypt_one(essiv_tfm, iv, iv);
+
+ return 0;
+}
+
+static int crypt_iv_benbi_ctr(struct geniv_ctx *ctx)
+{
+ unsigned int bs = crypto_skcipher_blocksize(any_tfm(ctx));
+ int log = ilog2(bs);
+
+ /* we need to calculate how far we must shift the sector count
+ * to get the cipher block count, we use this shift in _gen
+ */
+
+ if (1 << log != bs) {
+ pr_err("cypher blocksize is not a power of 2\n");
+ return -EINVAL;
+ }
+
+ if (log > 9) {
+ pr_err("cypher blocksize is > 512\n");
+ return -EINVAL;
+ }
+
+ ctx->iv_gen_private.benbi.shift = 9 - log;
+
+ return 0;
+}
+
+static int crypt_iv_benbi_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *iv = rctx->iv;
+ __be64 val;
+
+ memset(iv, 0, ctx->iv_size - sizeof(u64)); /* rest is cleared below */
+
+ val = cpu_to_be64(((u64) rctx->iv_sector <<
+ ctx->iv_gen_private.benbi.shift) + 1);
+ put_unaligned(val, (__be64 *)(iv + ctx->iv_size - sizeof(u64)));
+
+ return 0;
+}
+
+static int crypt_iv_null_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *iv = rctx->iv;
+
+ memset(iv, 0, ctx->iv_size);
+ return 0;
+}
+
+static void crypt_iv_lmk_dtr(struct geniv_ctx *ctx)
+{
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+
+ if (lmk->hash_tfm && !IS_ERR(lmk->hash_tfm))
+ crypto_free_shash(lmk->hash_tfm);
+ lmk->hash_tfm = NULL;
+
+ kzfree(lmk->seed);
+ lmk->seed = NULL;
+}
+
+static int crypt_iv_lmk_ctr(struct geniv_ctx *ctx)
+{
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+
+ lmk->hash_tfm = crypto_alloc_shash("md5", 0, 0);
+ if (IS_ERR(lmk->hash_tfm)) {
+ pr_err("Error initializing LMK hash; err=%ld\n",
+ PTR_ERR(lmk->hash_tfm));
+ return PTR_ERR(lmk->hash_tfm);
+ }
+
+ /* No seed in LMK version 2 */
+ if (ctx->key_parts == ctx->tfms_count) {
+ lmk->seed = NULL;
+ return 0;
+ }
+
+ lmk->seed = kzalloc(LMK_SEED_SIZE, GFP_KERNEL);
+ if (!lmk->seed) {
+ crypt_iv_lmk_dtr(ctx);
+ pr_err("Error kmallocing seed storage in LMK\n");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static int crypt_iv_lmk_init(struct geniv_ctx *ctx)
+{
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+ int subkey_size = ctx->key_size / ctx->key_parts;
+
+ /* LMK seed is on the position of LMK_KEYS + 1 key */
+ if (lmk->seed)
+ memcpy(lmk->seed, ctx->key + (ctx->tfms_count * subkey_size),
+ crypto_shash_digestsize(lmk->hash_tfm));
+
+ return 0;
+}
+
+static int crypt_iv_lmk_wipe(struct geniv_ctx *ctx)
+{
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+
+ if (lmk->seed)
+ memset(lmk->seed, 0, LMK_SEED_SIZE);
+
+ return 0;
+}
+
+static int crypt_iv_lmk_one(struct geniv_ctx *ctx, u8 *iv,
+ struct crypto_geniv_req_ctx *rctx, u8 *data)
+{
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+ struct md5_state md5state;
+ __le32 buf[4];
+ int i, r;
+ SHASH_DESC_ON_STACK(desc, lmk->hash_tfm);
+
+ desc->tfm = lmk->hash_tfm;
+ desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
+
+ r = crypto_shash_init(desc);
+ if (r)
+ return r;
+
+ if (lmk->seed) {
+ r = crypto_shash_update(desc, lmk->seed, LMK_SEED_SIZE);
+ if (r)
+ return r;
+ }
+
+ /* Sector is always 512B, block size 16, add data of blocks 1-31 */
+ r = crypto_shash_update(desc, data + 16, 16 * 31);
+ if (r)
+ return r;
+
+ /* Sector is cropped to 56 bits here */
+ buf[0] = cpu_to_le32(rctx->iv_sector & 0xFFFFFFFF);
+ buf[1] = cpu_to_le32((((u64)rctx->iv_sector >> 32) & 0x00FFFFFF)
+ | 0x80000000);
+ buf[2] = cpu_to_le32(4024);
+ buf[3] = 0;
+ r = crypto_shash_update(desc, (u8 *)buf, sizeof(buf));
+ if (r)
+ return r;
+
+ /* No MD5 padding here */
+ r = crypto_shash_export(desc, &md5state);
+ if (r)
+ return r;
+
+ for (i = 0; i < MD5_HASH_WORDS; i++)
+ __cpu_to_le32s(&md5state.hash[i]);
+ memcpy(iv, &md5state.hash, ctx->iv_size);
+
+ return 0;
+}
+
+static int crypt_iv_lmk_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *src;
+ u8 *iv = rctx->iv;
+ int r = 0;
+
+ if (rctx->is_write) {
+ src = kmap_atomic(sg_page(&rctx->src[n]));
+ r = crypt_iv_lmk_one(ctx, iv, rctx, src + rctx->src[n].offset);
+ kunmap_atomic(src);
+ } else
+ memset(iv, 0, ctx->iv_size);
+
+ return r;
+}
+
+static int crypt_iv_lmk_post(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *dst;
+ u8 *iv = rctx->iv;
+ int r;
+
+ if (rctx->is_write)
+ return 0;
+
+ dst = kmap_atomic(sg_page(&rctx->dst[n]));
+ r = crypt_iv_lmk_one(ctx, iv, rctx, dst + rctx->dst[n].offset);
+
+ /* Tweak the first block of plaintext sector */
+ if (!r)
+ crypto_xor(dst + rctx->dst[n].offset, iv, ctx->iv_size);
+
+ kunmap_atomic(dst);
+ return r;
+}
+
+static void crypt_iv_tcw_dtr(struct geniv_ctx *ctx)
+{
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+
+ kzfree(tcw->iv_seed);
+ tcw->iv_seed = NULL;
+ kzfree(tcw->whitening);
+ tcw->whitening = NULL;
+
+ if (tcw->crc32_tfm && !IS_ERR(tcw->crc32_tfm))
+ crypto_free_shash(tcw->crc32_tfm);
+ tcw->crc32_tfm = NULL;
+}
+
+static int crypt_iv_tcw_ctr(struct geniv_ctx *ctx)
+{
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+
+ if (ctx->key_size <= (ctx->iv_size + TCW_WHITENING_SIZE)) {
+ pr_err("Wrong key size (%d) for TCW. Choose a value > %d bytes\n",
+ ctx->key_size,
+ ctx->iv_size + TCW_WHITENING_SIZE);
+ return -EINVAL;
+ }
+
+ tcw->crc32_tfm = crypto_alloc_shash("crc32", 0, 0);
+ if (IS_ERR(tcw->crc32_tfm)) {
+ pr_err("Error initializing CRC32 in TCW; err=%ld\n",
+ PTR_ERR(tcw->crc32_tfm));
+ return PTR_ERR(tcw->crc32_tfm);
+ }
+
+ tcw->iv_seed = kzalloc(ctx->iv_size, GFP_KERNEL);
+ tcw->whitening = kzalloc(TCW_WHITENING_SIZE, GFP_KERNEL);
+ if (!tcw->iv_seed || !tcw->whitening) {
+ crypt_iv_tcw_dtr(ctx);
+ pr_err("Error allocating seed storage in TCW\n");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static int crypt_iv_tcw_init(struct geniv_ctx *ctx)
+{
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ int key_offset = ctx->key_size - ctx->iv_size - TCW_WHITENING_SIZE;
+
+ memcpy(tcw->iv_seed, &ctx->key[key_offset], ctx->iv_size);
+ memcpy(tcw->whitening, &ctx->key[key_offset + ctx->iv_size],
+ TCW_WHITENING_SIZE);
+
+ return 0;
+}
+
+static int crypt_iv_tcw_wipe(struct geniv_ctx *ctx)
+{
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+
+ memset(tcw->iv_seed, 0, ctx->iv_size);
+ memset(tcw->whitening, 0, TCW_WHITENING_SIZE);
+
+ return 0;
+}
+
+static int crypt_iv_tcw_whitening(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, u8 *data)
+{
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ __le64 sector = cpu_to_le64(rctx->iv_sector);
+ u8 buf[TCW_WHITENING_SIZE];
+ int i, r;
+ SHASH_DESC_ON_STACK(desc, tcw->crc32_tfm);
+
+ /* xor whitening with sector number */
+ memcpy(buf, tcw->whitening, TCW_WHITENING_SIZE);
+ crypto_xor(buf, (u8 *)§or, 8);
+ crypto_xor(&buf[8], (u8 *)§or, 8);
+
+ /* calculate crc32 for every 32bit part and xor it */
+ desc->tfm = tcw->crc32_tfm;
+ desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
+ for (i = 0; i < 4; i++) {
+ r = crypto_shash_init(desc);
+ if (r)
+ goto out;
+ r = crypto_shash_update(desc, &buf[i * 4], 4);
+ if (r)
+ goto out;
+ r = crypto_shash_final(desc, &buf[i * 4]);
+ if (r)
+ goto out;
+ }
+ crypto_xor(&buf[0], &buf[12], 4);
+ crypto_xor(&buf[4], &buf[8], 4);
+
+ /* apply whitening (8 bytes) to whole sector */
+ for (i = 0; i < (SECTOR_SIZE / 8); i++)
+ crypto_xor(data + i * 8, buf, 8);
+out:
+ memzero_explicit(buf, sizeof(buf));
+ return r;
+}
+
+static int crypt_iv_tcw_gen(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *iv = rctx->iv;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ __le64 sector = cpu_to_le64(rctx->iv_sector);
+ u8 *src;
+ int r = 0;
+
+ /* Remove whitening from ciphertext */
+ if (!rctx->is_write) {
+ src = kmap_atomic(sg_page(&rctx->src[n]));
+ r = crypt_iv_tcw_whitening(ctx, rctx,
+ src + rctx->src[n].offset);
+ kunmap_atomic(src);
+ }
+
+ /* Calculate IV */
+ memcpy(iv, tcw->iv_seed, ctx->iv_size);
+ crypto_xor(iv, (u8 *)§or, 8);
+ if (ctx->iv_size > 8)
+ crypto_xor(&iv[8], (u8 *)§or, ctx->iv_size - 8);
+
+ return r;
+}
+
+static int crypt_iv_tcw_post(struct geniv_ctx *ctx,
+ struct crypto_geniv_req_ctx *rctx, int n)
+{
+ u8 *dst;
+ int r;
+
+ if (!rctx->is_write)
+ return 0;
+
+ /* Apply whitening on ciphertext */
+ dst = kmap_atomic(sg_page(&rctx->dst[n]));
+ r = crypt_iv_tcw_whitening(ctx, rctx, dst + rctx->dst[n].offset);
+ kunmap_atomic(dst);
+
+ return r;
+}
+
+static struct geniv_operations crypt_iv_plain_ops = {
+ .generator = crypt_iv_plain_gen
+};
+
+static struct geniv_operations crypt_iv_plain64_ops = {
+ .generator = crypt_iv_plain64_gen
+};
+
+static struct geniv_operations crypt_iv_essiv_ops = {
+ .ctr = crypt_iv_essiv_ctr,
+ .dtr = crypt_iv_essiv_dtr,
+ .init = crypt_iv_essiv_init,
+ .wipe = crypt_iv_essiv_wipe,
+ .generator = crypt_iv_essiv_gen
+};
+
+static struct geniv_operations crypt_iv_benbi_ops = {
+ .ctr = crypt_iv_benbi_ctr,
+ .generator = crypt_iv_benbi_gen
+};
+
+static struct geniv_operations crypt_iv_null_ops = {
+ .generator = crypt_iv_null_gen
+};
+
+static struct geniv_operations crypt_iv_lmk_ops = {
+ .ctr = crypt_iv_lmk_ctr,
+ .dtr = crypt_iv_lmk_dtr,
+ .init = crypt_iv_lmk_init,
+ .wipe = crypt_iv_lmk_wipe,
+ .generator = crypt_iv_lmk_gen,
+ .post = crypt_iv_lmk_post
+};
+
+static struct geniv_operations crypt_iv_tcw_ops = {
+ .ctr = crypt_iv_tcw_ctr,
+ .dtr = crypt_iv_tcw_dtr,
+ .init = crypt_iv_tcw_init,
+ .wipe = crypt_iv_tcw_wipe,
+ .generator = crypt_iv_tcw_gen,
+ .post = crypt_iv_tcw_post
+};
+
+static int geniv_setkey_set(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init)
+ ret = ctx->iv_gen_ops->init(ctx);
+ return ret;
+}
+
+static int geniv_setkey_wipe(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->wipe) {
+ ret = ctx->iv_gen_ops->wipe(ctx);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
+static int geniv_setkey_init_ctx(struct geniv_ctx *ctx)
+{
+ int ret = -EINVAL;
+
+ pr_debug("IV Generation algorithm : %s\n", ctx->ivmode);
+
+ if (ctx->ivmode == NULL)
+ ctx->iv_gen_ops = NULL;
+ else if (strcmp(ctx->ivmode, "plain") == 0)
+ ctx->iv_gen_ops = &crypt_iv_plain_ops;
+ else if (strcmp(ctx->ivmode, "plain64") == 0)
+ ctx->iv_gen_ops = &crypt_iv_plain64_ops;
+ else if (strcmp(ctx->ivmode, "essiv") == 0)
+ ctx->iv_gen_ops = &crypt_iv_essiv_ops;
+ else if (strcmp(ctx->ivmode, "benbi") == 0)
+ ctx->iv_gen_ops = &crypt_iv_benbi_ops;
+ else if (strcmp(ctx->ivmode, "null") == 0)
+ ctx->iv_gen_ops = &crypt_iv_null_ops;
+ else if (strcmp(ctx->ivmode, "lmk") == 0)
+ ctx->iv_gen_ops = &crypt_iv_lmk_ops;
+ else if (strcmp(ctx->ivmode, "tcw") == 0) {
+ ctx->iv_gen_ops = &crypt_iv_tcw_ops;
+ ctx->key_parts += 2; /* IV + whitening */
+ ctx->key_extra_size = ctx->iv_size + TCW_WHITENING_SIZE;
+ } else {
+ ret = -EINVAL;
+ pr_err("Invalid IV mode %s\n", ctx->ivmode);
+ goto end;
+ }
+
+ /* Allocate IV */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->ctr) {
+ ret = ctx->iv_gen_ops->ctr(ctx);
+ if (ret < 0) {
+ pr_err("Error creating IV for %s\n", ctx->ivmode);
+ goto end;
+ }
+ }
+
+ /* Initialize IV (set keys for ESSIV etc) */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init) {
+ ret = ctx->iv_gen_ops->init(ctx);
+ if (ret < 0)
+ pr_err("Error creating IV for %s\n", ctx->ivmode);
+ }
+ ret = 0;
+end:
+ return ret;
+}
+
+/* Initialize the cipher's context with the key, ivmode and other parameters.
+ * Also allocate IV generation template ciphers and initialize them.
+ */
+
+static int geniv_setkey_init(struct crypto_skcipher *parent,
+ struct geniv_key_info *info)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+
+ ctx->tfm = parent;
+ ctx->iv_size = crypto_skcipher_ivsize(parent);
+ ctx->tfms_count = info->tfms_count;
+ ctx->cipher = info->cipher;
+ ctx->key = info->key;
+ ctx->key_size = info->key_size;
+ ctx->key_parts = info->key_parts;
+ ctx->ivmode = info->ivmode;
+ ctx->ivopts = info->ivopts;
+ return geniv_setkey_init_ctx(ctx);
+}
+
+static int crypto_geniv_setkey(struct crypto_skcipher *parent,
+ const u8 *key, unsigned int keylen)
+{
+ int err;
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ struct crypto_skcipher *child = ctx->child;
+ struct geniv_key_info *info = (struct geniv_key_info *) key;
+
+ pr_debug("SETKEY Operation : %d\n", info->keyop);
+
+ switch (info->keyop) {
+ case SETKEY_OP_INIT:
+ err = geniv_setkey_init(parent, info);
+ break;
+ case SETKEY_OP_SET:
+ err = geniv_setkey_set(ctx);
+ break;
+ case SETKEY_OP_WIPE:
+ err = geniv_setkey_wipe(ctx);
+ break;
+ }
+
+ crypto_skcipher_clear_flags(child, CRYPTO_TFM_REQ_MASK);
+ crypto_skcipher_set_flags(child, crypto_skcipher_get_flags(parent) &
+ CRYPTO_TFM_REQ_MASK);
+ err = crypto_skcipher_setkey(child, info->subkey, info->subkey_size);
+ crypto_skcipher_set_flags(parent, crypto_skcipher_get_flags(child) &
+ CRYPTO_TFM_RES_MASK);
+ return err;
+}
+
+/* Asynchronous IO completion callback for each sector in a segment. When all
+ * pending i/o are completed the parent cipher's async function is called.
+ */
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error)
+{
+ struct crypto_geniv_subreq *subreq =
+ (struct crypto_geniv_subreq *) async_req->data;
+ struct crypto_geniv_req_ctx *rctx = subreq->rctx;
+ struct skcipher_request *req = rctx->req;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ int n = subreq - rctx->subreqs;
+
+ /*
+ * A request from crypto driver backlog is going to be processed now,
+ * finish the completion and continue in crypt_convert().
+ * (Callback will be called for the second time for this request.)
+ */
+ if (error == -EINPROGRESS) {
+ complete(&rctx->restart);
+ return;
+ }
+
+ if (!error && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ error = ctx->iv_gen_ops->post(ctx, rctx, n);
+
+ /* req_pending needs to be checked before req->base.complete is called
+ * as we need 'req_pending' to be equal to 1 to ensure all subrequests
+ * are processed before freeing subreq array
+ */
+ if (!atomic_dec_and_test(&rctx->req_pending)) {
+ /* Call the parent cipher's completion function */
+ skcipher_request_complete(req, error);
+ kfree(rctx->subreqs);
+ kfree(rctx->src);
+ kfree(rctx->dst);
+ }
+}
+
+/* Split scatterlist of segments into scatterlist of sectors so that unique IVs
+ * could be generated for each 512-byte sector. This split may not be necessary
+ * for example when these ciphers are modelled in hardware, where in can make
+ * use of the hardware's IV generation capabilities.
+ */
+static unsigned int geniv_split_req(struct scatterlist *sg,
+ struct scatterlist **sg2_ptr,
+ unsigned int segments)
+{
+ unsigned int i, j, nents = 0, off, len;
+ struct scatterlist *sg2;
+
+ for (i = 0; i < segments ; i++)
+ nents += sg[i].length / SECTOR_SIZE;
+
+ pr_debug("geniv: splitting scatterlist with %d segments into %d ents\n",
+ segments, nents);
+ sg2 = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
+ *sg2_ptr = sg2;
+ for (i = 0, j = 0; i < segments ; i++) {
+
+ off = sg[i].offset;
+ len = sg[i].length;
+
+ for (; len > 0; j++) {
+ sg_set_page(&sg2[j], sg_page(&sg[i]), SECTOR_SIZE, off);
+ off += SECTOR_SIZE;
+ len -= SECTOR_SIZE;
+ }
+ }
+ return nents;
+}
+
+/* Common encryt/decrypt function for geniv template cipher. Before the crypto
+ * operation, it splits the memory segments (in the scatterlist) into 512 byte
+ * sectors. The initialization vector(IV) used is based on a unique sector
+ * number which is generated here.
+ */
+static inline int crypto_geniv_crypt(struct skcipher_request *req, bool encrypt)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct crypto_geniv_req_ctx *rctx = geniv_req_ctx(req);
+ struct crypto_geniv_subreq *subreqs;
+ struct geniv_req_info *rinfo = (struct geniv_req_info *) req->iv;
+ int i, bytes, cryptlen, ret = 0, n1, n2;
+ char *str = encrypt ? "encrypt" : "decrypt";
+
+ /* Instance of 'struct geniv_req_info' is stored in IV ptr */
+ rctx->is_write = rinfo->is_write;
+ rctx->iv_sector = rinfo->iv_sector;
+ rctx->nents = rinfo->nents;
+ rctx->iv = rinfo->iv;
+
+ pr_debug("geniv:%s: starting sector=%d, #segments=%u\n", str,
+ (unsigned int) rctx->iv_sector, rctx->nents);
+
+ cryptlen = req->cryptlen;
+ n1 = geniv_split_req(req->src, &rctx->src, rctx->nents);
+ n2 = geniv_split_req(req->dst, &rctx->dst, rctx->nents);
+ rctx->nents = n1 > n2 ? n1 : n2;
+
+ subreqs = kcalloc(rctx->nents, sizeof(struct crypto_geniv_subreq),
+ GFP_KERNEL);
+ rctx->subreqs = subreqs;
+ rctx->req = req;
+
+ init_completion(&rctx->restart);
+ atomic_set(&rctx->req_pending, 1);
+ for (i = 0; i < rctx->nents; i++) {
+ struct skcipher_request *subreq = &subreqs[i].req;
+
+ subreqs[i].rctx = rctx;
+ atomic_inc(&rctx->req_pending);
+ if (ctx->iv_gen_ops)
+ ret = ctx->iv_gen_ops->generator(ctx, rctx, i);
+
+ if (ret < 0) {
+ pr_err("Error in generating IV ret: %d\n", ret);
+ goto end;
+ }
+
+ skcipher_request_set_tfm(subreq, ctx->child);
+ skcipher_request_set_callback(subreq, req->base.flags,
+ geniv_async_done, &subreqs[i]);
+
+ bytes = cryptlen < SECTOR_SIZE ? cryptlen : SECTOR_SIZE;
+
+ skcipher_request_set_crypt(subreq, &rctx->src[i],
+ &rctx->dst[i], bytes, rctx->iv);
+ cryptlen -= bytes;
+
+ if (encrypt)
+ ret = crypto_skcipher_encrypt(subreq);
+ else
+ ret = crypto_skcipher_decrypt(subreq);
+
+
+ if (!ret && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ ret = ctx->iv_gen_ops->post(ctx, rctx, i);
+
+ switch (ret) {
+ /*
+ * The request was queued by a crypto driver
+ * but the driver request queue is full, let's wait.
+ */
+ case -EBUSY:
+ wait_for_completion(&rctx->restart);
+ reinit_completion(&rctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function geniv_async_done() is called.
+ */
+ case -EINPROGRESS:
+ rctx->iv_sector++;
+ cond_resched();
+ break;
+ /*
+ * The request was already processed (synchronously).
+ */
+ case 0:
+ atomic_dec(&rctx->req_pending);
+ rctx->iv_sector++;
+ cond_resched();
+ continue;
+
+ /* There was an error while processing the request. */
+ default:
+ atomic_dec(&rctx->req_pending);
+ return ret;
+ }
+
+ if (ret)
+ break;
+ }
+
+ if (atomic_read(&rctx->req_pending) == 1) {
+ pr_debug("geniv:%s: Freeing subreq and scatterlists\n", str);
+ kfree(subreqs);
+ kfree(rctx->src);
+ kfree(rctx->dst);
+ }
+
+end:
+ return ret;
+}
+
+static int crypto_geniv_encrypt(struct skcipher_request *req)
+{
+ return crypto_geniv_crypt(req, true);
+}
+
+static int crypto_geniv_decrypt(struct skcipher_request *req)
+{
+ return crypto_geniv_crypt(req, false);
+}
+
+static int crypto_geniv_init_tfm(struct crypto_skcipher *tfm)
+{
+ struct skcipher_instance *inst = skcipher_alg_instance(tfm);
+ struct crypto_skcipher_spawn *spawn = skcipher_instance_ctx(inst);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct crypto_skcipher *cipher;
+ unsigned long align;
+ unsigned int reqsize;
+
+ cipher = crypto_spawn_skcipher2(spawn);
+ if (IS_ERR(cipher))
+ return PTR_ERR(cipher);
+
+ ctx->child = cipher;
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(tfm);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct crypto_geniv_req_ctx) +
+ crypto_skcipher_reqsize(cipher);
+ crypto_skcipher_set_reqsize(tfm, reqsize);
+
+ return 0;
+}
+
+static void crypto_geniv_exit_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->dtr)
+ ctx->iv_gen_ops->dtr(ctx);
+
+ crypto_free_skcipher(ctx->child);
+}
+
+static void crypto_geniv_free(struct skcipher_instance *inst)
+{
+ struct crypto_skcipher_spawn *spawn = skcipher_instance_ctx(inst);
+
+ crypto_drop_skcipher(spawn);
+ kfree(inst);
+}
+
+static int crypto_geniv_create(struct crypto_template *tmpl,
+ struct rtattr **tb, char *algname)
+{
+ struct crypto_attr_type *algt;
+ struct skcipher_instance *inst;
+ struct skcipher_alg *alg;
+ struct crypto_skcipher_spawn *spawn;
+ const char *cipher_name;
+ int err;
+
+ algt = crypto_get_attr_type(tb);
+
+ if (IS_ERR(algt))
+ return PTR_ERR(algt);
+
+ if ((algt->type ^ CRYPTO_ALG_TYPE_SKCIPHER) & algt->mask)
+ return -EINVAL;
+
+ cipher_name = crypto_attr_alg_name(tb[1]);
+
+ if (IS_ERR(cipher_name))
+ return PTR_ERR(cipher_name);
+
+ inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ spawn = skcipher_instance_ctx(inst);
+
+ crypto_set_skcipher_spawn(spawn, skcipher_crypto_instance(inst));
+ err = crypto_grab_skcipher2(spawn, cipher_name, 0,
+ crypto_requires_sync(algt->type,
+ algt->mask));
+
+ if (err)
+ goto err_free_inst;
+
+ alg = crypto_spawn_skcipher_alg(spawn);
+
+ /* We only support 16-byte blocks. */
+ err = -EINVAL;
+
+ if (!is_power_of_2(alg->base.cra_blocksize))
+ goto err_drop_spawn;
+
+ err = -ENAMETOOLONG;
+ if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ algname, alg->base.cra_name) >= CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+ if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME,
+ "%s(%s)", algname, alg->base.cra_driver_name) >=
+ CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+
+ inst->alg.base.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER;
+ inst->alg.base.cra_priority = alg->base.cra_priority;
+ inst->alg.base.cra_blocksize = alg->base.cra_blocksize;
+ inst->alg.base.cra_alignmask = alg->base.cra_alignmask;
+ inst->alg.base.cra_flags = alg->base.cra_flags & CRYPTO_ALG_ASYNC;
+ inst->alg.ivsize = alg->base.cra_blocksize;
+ inst->alg.chunksize = crypto_skcipher_alg_chunksize(alg);
+ inst->alg.min_keysize = crypto_skcipher_alg_min_keysize(alg);
+ inst->alg.max_keysize = crypto_skcipher_alg_max_keysize(alg);
+
+ inst->alg.setkey = crypto_geniv_setkey;
+ inst->alg.encrypt = crypto_geniv_encrypt;
+ inst->alg.decrypt = crypto_geniv_decrypt;
+
+ inst->alg.base.cra_ctxsize = sizeof(struct geniv_ctx);
+
+ inst->alg.init = crypto_geniv_init_tfm;
+ inst->alg.exit = crypto_geniv_exit_tfm;
+
+ inst->free = crypto_geniv_free;
+
+ err = skcipher_register_instance(tmpl, inst);
+ if (err)
+ goto err_drop_spawn;
+
+out:
+ return err;
+
+err_drop_spawn:
+ crypto_drop_skcipher(spawn);
+err_free_inst:
+ kfree(inst);
+ goto out;
+}
+
+static int crypto_plain_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "plain");
+}
+
+static int crypto_plain64_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "plain64");
+}
+
+static int crypto_essiv_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "essiv");
+}
+
+static int crypto_benbi_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "benbi");
+}
+
+static int crypto_null_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "null");
+}
+
+static int crypto_lmk_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "lmk");
+}
+
+static int crypto_tcw_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return crypto_geniv_create(tmpl, tb, "tcw");
+}
+
+static struct crypto_template crypto_plain_tmpl = {
+ .name = "plain",
+ .create = crypto_plain_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_plain64_tmpl = {
+ .name = "plain64",
+ .create = crypto_plain64_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_essiv_tmpl = {
+ .name = "essiv",
+ .create = crypto_essiv_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_benbi_tmpl = {
+ .name = "benbi",
+ .create = crypto_benbi_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_null_tmpl = {
+ .name = "null",
+ .create = crypto_null_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_lmk_tmpl = {
+ .name = "lmk",
+ .create = crypto_lmk_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_tcw_tmpl = {
+ .name = "tcw",
+ .create = crypto_tcw_create,
+ .module = THIS_MODULE,
+};
+
+static int __init crypto_geniv_module_init(void)
+{
+ int err;
+
+ err = crypto_register_template(&crypto_plain_tmpl);
+ if (err)
+ goto out;
+
+ err = crypto_register_template(&crypto_plain64_tmpl);
+ if (err)
+ goto out_undo_plain;
+
+ err = crypto_register_template(&crypto_essiv_tmpl);
+ if (err)
+ goto out_undo_plain64;
+
+ err = crypto_register_template(&crypto_benbi_tmpl);
+ if (err)
+ goto out_undo_essiv;
+
+ err = crypto_register_template(&crypto_null_tmpl);
+ if (err)
+ goto out_undo_benbi;
+
+ err = crypto_register_template(&crypto_lmk_tmpl);
+ if (err)
+ goto out_undo_null;
+
+ err = crypto_register_template(&crypto_tcw_tmpl);
+ if (!err)
+ goto out;
+
+ crypto_unregister_template(&crypto_lmk_tmpl);
+out_undo_null:
+ crypto_unregister_template(&crypto_null_tmpl);
+out_undo_benbi:
+ crypto_unregister_template(&crypto_benbi_tmpl);
+out_undo_essiv:
+ crypto_unregister_template(&crypto_essiv_tmpl);
+out_undo_plain64:
+ crypto_unregister_template(&crypto_plain64_tmpl);
+out_undo_plain:
+ crypto_unregister_template(&crypto_plain_tmpl);
+out:
+ return err;
+}
+
+static void __exit crypto_geniv_module_exit(void)
+{
+ crypto_unregister_template(&crypto_plain_tmpl);
+ crypto_unregister_template(&crypto_plain64_tmpl);
+ crypto_unregister_template(&crypto_essiv_tmpl);
+ crypto_unregister_template(&crypto_benbi_tmpl);
+ crypto_unregister_template(&crypto_null_tmpl);
+ crypto_unregister_template(&crypto_lmk_tmpl);
+ crypto_unregister_template(&crypto_tcw_tmpl);
+}
+
+module_init(crypto_geniv_module_init);
+module_exit(crypto_geniv_module_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("IV generation algorithms");
+MODULE_ALIAS_CRYPTO("geniv");
+
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index a276883..1d565d8 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -29,10 +29,12 @@
#include <crypto/md5.h>
#include <crypto/algapi.h>
#include <crypto/skcipher.h>
+#include <crypto/geniv.h>
#include <linux/device-mapper.h>
#define DM_MSG_PREFIX "crypt"
+#define MAX_SG_LIST 1024
/*
* context holding the current state of a multi-part conversion
@@ -67,47 +69,13 @@ struct dm_crypt_io {
struct dm_crypt_request {
struct convert_context *ctx;
- struct scatterlist sg_in;
- struct scatterlist sg_out;
+ struct scatterlist *sg_in;
+ struct scatterlist *sg_out;
sector_t iv_sector;
};
struct crypt_config;
-struct crypt_iv_operations {
- int (*ctr)(struct crypt_config *cc, struct dm_target *ti,
- const char *opts);
- void (*dtr)(struct crypt_config *cc);
- int (*init)(struct crypt_config *cc);
- int (*wipe)(struct crypt_config *cc);
- int (*generator)(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq);
- int (*post)(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq);
-};
-
-struct iv_essiv_private {
- struct crypto_ahash *hash_tfm;
- u8 *salt;
-};
-
-struct iv_benbi_private {
- int shift;
-};
-
-#define LMK_SEED_SIZE 64 /* hash + 0 */
-struct iv_lmk_private {
- struct crypto_shash *hash_tfm;
- u8 *seed;
-};
-
-#define TCW_WHITENING_SIZE 16
-struct iv_tcw_private {
- struct crypto_shash *crc32_tfm;
- u8 *iv_seed;
- u8 *whitening;
-};
-
/*
* Crypt: maps a linear range of a block device
* and encrypts / decrypts at the same time.
@@ -141,13 +109,6 @@ struct crypt_config {
char *cipher;
char *cipher_string;
- struct crypt_iv_operations *iv_gen_ops;
- union {
- struct iv_essiv_private essiv;
- struct iv_benbi_private benbi;
- struct iv_lmk_private lmk;
- struct iv_tcw_private tcw;
- } iv_gen_private;
sector_t iv_offset;
unsigned int iv_size;
@@ -241,567 +202,6 @@ static struct crypto_skcipher *any_tfm(struct crypt_config *cc)
* http://article.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/454
*/
-static int crypt_iv_plain_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- memset(iv, 0, cc->iv_size);
- *(__le32 *)iv = cpu_to_le32(dmreq->iv_sector & 0xffffffff);
-
- return 0;
-}
-
-static int crypt_iv_plain64_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- memset(iv, 0, cc->iv_size);
- *(__le64 *)iv = cpu_to_le64(dmreq->iv_sector);
-
- return 0;
-}
-
-/* Initialise ESSIV - compute salt but no local memory allocations */
-static int crypt_iv_essiv_init(struct crypt_config *cc)
-{
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
- AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
- struct scatterlist sg;
- struct crypto_cipher *essiv_tfm;
- int err;
-
- sg_init_one(&sg, cc->key, cc->key_size);
- ahash_request_set_tfm(req, essiv->hash_tfm);
- ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
- ahash_request_set_crypt(req, &sg, essiv->salt, cc->key_size);
-
- err = crypto_ahash_digest(req);
- ahash_request_zero(req);
- if (err)
- return err;
-
- essiv_tfm = cc->iv_private;
-
- err = crypto_cipher_setkey(essiv_tfm, essiv->salt,
- crypto_ahash_digestsize(essiv->hash_tfm));
- if (err)
- return err;
-
- return 0;
-}
-
-/* Wipe salt and reset key derived from volume key */
-static int crypt_iv_essiv_wipe(struct crypt_config *cc)
-{
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
- unsigned salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
- struct crypto_cipher *essiv_tfm;
- int r, err = 0;
-
- memset(essiv->salt, 0, salt_size);
-
- essiv_tfm = cc->iv_private;
- r = crypto_cipher_setkey(essiv_tfm, essiv->salt, salt_size);
- if (r)
- err = r;
-
- return err;
-}
-
-/* Set up per cpu cipher state */
-static struct crypto_cipher *setup_essiv_cpu(struct crypt_config *cc,
- struct dm_target *ti,
- u8 *salt, unsigned saltsize)
-{
- struct crypto_cipher *essiv_tfm;
- int err;
-
- /* Setup the essiv_tfm with the given salt */
- essiv_tfm = crypto_alloc_cipher(cc->cipher, 0, CRYPTO_ALG_ASYNC);
- if (IS_ERR(essiv_tfm)) {
- ti->error = "Error allocating crypto tfm for ESSIV";
- return essiv_tfm;
- }
-
- if (crypto_cipher_blocksize(essiv_tfm) !=
- crypto_skcipher_ivsize(any_tfm(cc))) {
- ti->error = "Block size of ESSIV cipher does "
- "not match IV size of block cipher";
- crypto_free_cipher(essiv_tfm);
- return ERR_PTR(-EINVAL);
- }
-
- err = crypto_cipher_setkey(essiv_tfm, salt, saltsize);
- if (err) {
- ti->error = "Failed to set key for ESSIV cipher";
- crypto_free_cipher(essiv_tfm);
- return ERR_PTR(err);
- }
-
- return essiv_tfm;
-}
-
-static void crypt_iv_essiv_dtr(struct crypt_config *cc)
-{
- struct crypto_cipher *essiv_tfm;
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
-
- crypto_free_ahash(essiv->hash_tfm);
- essiv->hash_tfm = NULL;
-
- kzfree(essiv->salt);
- essiv->salt = NULL;
-
- essiv_tfm = cc->iv_private;
-
- if (essiv_tfm)
- crypto_free_cipher(essiv_tfm);
-
- cc->iv_private = NULL;
-}
-
-static int crypt_iv_essiv_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
-{
- struct crypto_cipher *essiv_tfm = NULL;
- struct crypto_ahash *hash_tfm = NULL;
- u8 *salt = NULL;
- int err;
-
- if (!opts) {
- ti->error = "Digest algorithm missing for ESSIV mode";
- return -EINVAL;
- }
-
- /* Allocate hash algorithm */
- hash_tfm = crypto_alloc_ahash(opts, 0, CRYPTO_ALG_ASYNC);
- if (IS_ERR(hash_tfm)) {
- ti->error = "Error initializing ESSIV hash";
- err = PTR_ERR(hash_tfm);
- goto bad;
- }
-
- salt = kzalloc(crypto_ahash_digestsize(hash_tfm), GFP_KERNEL);
- if (!salt) {
- ti->error = "Error kmallocing salt storage in ESSIV";
- err = -ENOMEM;
- goto bad;
- }
-
- cc->iv_gen_private.essiv.salt = salt;
- cc->iv_gen_private.essiv.hash_tfm = hash_tfm;
-
- essiv_tfm = setup_essiv_cpu(cc, ti, salt,
- crypto_ahash_digestsize(hash_tfm));
- if (IS_ERR(essiv_tfm)) {
- crypt_iv_essiv_dtr(cc);
- return PTR_ERR(essiv_tfm);
- }
- cc->iv_private = essiv_tfm;
-
- return 0;
-
-bad:
- if (hash_tfm && !IS_ERR(hash_tfm))
- crypto_free_ahash(hash_tfm);
- kfree(salt);
- return err;
-}
-
-static int crypt_iv_essiv_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- struct crypto_cipher *essiv_tfm = cc->iv_private;
-
- memset(iv, 0, cc->iv_size);
- *(__le64 *)iv = cpu_to_le64(dmreq->iv_sector);
- crypto_cipher_encrypt_one(essiv_tfm, iv, iv);
-
- return 0;
-}
-
-static int crypt_iv_benbi_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
-{
- unsigned bs = crypto_skcipher_blocksize(any_tfm(cc));
- int log = ilog2(bs);
-
- /* we need to calculate how far we must shift the sector count
- * to get the cipher block count, we use this shift in _gen */
-
- if (1 << log != bs) {
- ti->error = "cypher blocksize is not a power of 2";
- return -EINVAL;
- }
-
- if (log > 9) {
- ti->error = "cypher blocksize is > 512";
- return -EINVAL;
- }
-
- cc->iv_gen_private.benbi.shift = 9 - log;
-
- return 0;
-}
-
-static void crypt_iv_benbi_dtr(struct crypt_config *cc)
-{
-}
-
-static int crypt_iv_benbi_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- __be64 val;
-
- memset(iv, 0, cc->iv_size - sizeof(u64)); /* rest is cleared below */
-
- val = cpu_to_be64(((u64)dmreq->iv_sector << cc->iv_gen_private.benbi.shift) + 1);
- put_unaligned(val, (__be64 *)(iv + cc->iv_size - sizeof(u64)));
-
- return 0;
-}
-
-static int crypt_iv_null_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- memset(iv, 0, cc->iv_size);
-
- return 0;
-}
-
-static void crypt_iv_lmk_dtr(struct crypt_config *cc)
-{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
-
- if (lmk->hash_tfm && !IS_ERR(lmk->hash_tfm))
- crypto_free_shash(lmk->hash_tfm);
- lmk->hash_tfm = NULL;
-
- kzfree(lmk->seed);
- lmk->seed = NULL;
-}
-
-static int crypt_iv_lmk_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
-{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
-
- lmk->hash_tfm = crypto_alloc_shash("md5", 0, 0);
- if (IS_ERR(lmk->hash_tfm)) {
- ti->error = "Error initializing LMK hash";
- return PTR_ERR(lmk->hash_tfm);
- }
-
- /* No seed in LMK version 2 */
- if (cc->key_parts == cc->tfms_count) {
- lmk->seed = NULL;
- return 0;
- }
-
- lmk->seed = kzalloc(LMK_SEED_SIZE, GFP_KERNEL);
- if (!lmk->seed) {
- crypt_iv_lmk_dtr(cc);
- ti->error = "Error kmallocing seed storage in LMK";
- return -ENOMEM;
- }
-
- return 0;
-}
-
-static int crypt_iv_lmk_init(struct crypt_config *cc)
-{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
- int subkey_size = cc->key_size / cc->key_parts;
-
- /* LMK seed is on the position of LMK_KEYS + 1 key */
- if (lmk->seed)
- memcpy(lmk->seed, cc->key + (cc->tfms_count * subkey_size),
- crypto_shash_digestsize(lmk->hash_tfm));
-
- return 0;
-}
-
-static int crypt_iv_lmk_wipe(struct crypt_config *cc)
-{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
-
- if (lmk->seed)
- memset(lmk->seed, 0, LMK_SEED_SIZE);
-
- return 0;
-}
-
-static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq,
- u8 *data)
-{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
- SHASH_DESC_ON_STACK(desc, lmk->hash_tfm);
- struct md5_state md5state;
- __le32 buf[4];
- int i, r;
-
- desc->tfm = lmk->hash_tfm;
- desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
-
- r = crypto_shash_init(desc);
- if (r)
- return r;
-
- if (lmk->seed) {
- r = crypto_shash_update(desc, lmk->seed, LMK_SEED_SIZE);
- if (r)
- return r;
- }
-
- /* Sector is always 512B, block size 16, add data of blocks 1-31 */
- r = crypto_shash_update(desc, data + 16, 16 * 31);
- if (r)
- return r;
-
- /* Sector is cropped to 56 bits here */
- buf[0] = cpu_to_le32(dmreq->iv_sector & 0xFFFFFFFF);
- buf[1] = cpu_to_le32((((u64)dmreq->iv_sector >> 32) & 0x00FFFFFF) | 0x80000000);
- buf[2] = cpu_to_le32(4024);
- buf[3] = 0;
- r = crypto_shash_update(desc, (u8 *)buf, sizeof(buf));
- if (r)
- return r;
-
- /* No MD5 padding here */
- r = crypto_shash_export(desc, &md5state);
- if (r)
- return r;
-
- for (i = 0; i < MD5_HASH_WORDS; i++)
- __cpu_to_le32s(&md5state.hash[i]);
- memcpy(iv, &md5state.hash, cc->iv_size);
-
- return 0;
-}
-
-static int crypt_iv_lmk_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- u8 *src;
- int r = 0;
-
- if (bio_data_dir(dmreq->ctx->bio_in) == WRITE) {
- src = kmap_atomic(sg_page(&dmreq->sg_in));
- r = crypt_iv_lmk_one(cc, iv, dmreq, src + dmreq->sg_in.offset);
- kunmap_atomic(src);
- } else
- memset(iv, 0, cc->iv_size);
-
- return r;
-}
-
-static int crypt_iv_lmk_post(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- u8 *dst;
- int r;
-
- if (bio_data_dir(dmreq->ctx->bio_in) == WRITE)
- return 0;
-
- dst = kmap_atomic(sg_page(&dmreq->sg_out));
- r = crypt_iv_lmk_one(cc, iv, dmreq, dst + dmreq->sg_out.offset);
-
- /* Tweak the first block of plaintext sector */
- if (!r)
- crypto_xor(dst + dmreq->sg_out.offset, iv, cc->iv_size);
-
- kunmap_atomic(dst);
- return r;
-}
-
-static void crypt_iv_tcw_dtr(struct crypt_config *cc)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
-
- kzfree(tcw->iv_seed);
- tcw->iv_seed = NULL;
- kzfree(tcw->whitening);
- tcw->whitening = NULL;
-
- if (tcw->crc32_tfm && !IS_ERR(tcw->crc32_tfm))
- crypto_free_shash(tcw->crc32_tfm);
- tcw->crc32_tfm = NULL;
-}
-
-static int crypt_iv_tcw_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
-
- if (cc->key_size <= (cc->iv_size + TCW_WHITENING_SIZE)) {
- ti->error = "Wrong key size for TCW";
- return -EINVAL;
- }
-
- tcw->crc32_tfm = crypto_alloc_shash("crc32", 0, 0);
- if (IS_ERR(tcw->crc32_tfm)) {
- ti->error = "Error initializing CRC32 in TCW";
- return PTR_ERR(tcw->crc32_tfm);
- }
-
- tcw->iv_seed = kzalloc(cc->iv_size, GFP_KERNEL);
- tcw->whitening = kzalloc(TCW_WHITENING_SIZE, GFP_KERNEL);
- if (!tcw->iv_seed || !tcw->whitening) {
- crypt_iv_tcw_dtr(cc);
- ti->error = "Error allocating seed storage in TCW";
- return -ENOMEM;
- }
-
- return 0;
-}
-
-static int crypt_iv_tcw_init(struct crypt_config *cc)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- int key_offset = cc->key_size - cc->iv_size - TCW_WHITENING_SIZE;
-
- memcpy(tcw->iv_seed, &cc->key[key_offset], cc->iv_size);
- memcpy(tcw->whitening, &cc->key[key_offset + cc->iv_size],
- TCW_WHITENING_SIZE);
-
- return 0;
-}
-
-static int crypt_iv_tcw_wipe(struct crypt_config *cc)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
-
- memset(tcw->iv_seed, 0, cc->iv_size);
- memset(tcw->whitening, 0, TCW_WHITENING_SIZE);
-
- return 0;
-}
-
-static int crypt_iv_tcw_whitening(struct crypt_config *cc,
- struct dm_crypt_request *dmreq,
- u8 *data)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- __le64 sector = cpu_to_le64(dmreq->iv_sector);
- u8 buf[TCW_WHITENING_SIZE];
- SHASH_DESC_ON_STACK(desc, tcw->crc32_tfm);
- int i, r;
-
- /* xor whitening with sector number */
- memcpy(buf, tcw->whitening, TCW_WHITENING_SIZE);
- crypto_xor(buf, (u8 *)§or, 8);
- crypto_xor(&buf[8], (u8 *)§or, 8);
-
- /* calculate crc32 for every 32bit part and xor it */
- desc->tfm = tcw->crc32_tfm;
- desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
- for (i = 0; i < 4; i++) {
- r = crypto_shash_init(desc);
- if (r)
- goto out;
- r = crypto_shash_update(desc, &buf[i * 4], 4);
- if (r)
- goto out;
- r = crypto_shash_final(desc, &buf[i * 4]);
- if (r)
- goto out;
- }
- crypto_xor(&buf[0], &buf[12], 4);
- crypto_xor(&buf[4], &buf[8], 4);
-
- /* apply whitening (8 bytes) to whole sector */
- for (i = 0; i < ((1 << SECTOR_SHIFT) / 8); i++)
- crypto_xor(data + i * 8, buf, 8);
-out:
- memzero_explicit(buf, sizeof(buf));
- return r;
-}
-
-static int crypt_iv_tcw_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- __le64 sector = cpu_to_le64(dmreq->iv_sector);
- u8 *src;
- int r = 0;
-
- /* Remove whitening from ciphertext */
- if (bio_data_dir(dmreq->ctx->bio_in) != WRITE) {
- src = kmap_atomic(sg_page(&dmreq->sg_in));
- r = crypt_iv_tcw_whitening(cc, dmreq, src + dmreq->sg_in.offset);
- kunmap_atomic(src);
- }
-
- /* Calculate IV */
- memcpy(iv, tcw->iv_seed, cc->iv_size);
- crypto_xor(iv, (u8 *)§or, 8);
- if (cc->iv_size > 8)
- crypto_xor(&iv[8], (u8 *)§or, cc->iv_size - 8);
-
- return r;
-}
-
-static int crypt_iv_tcw_post(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- u8 *dst;
- int r;
-
- if (bio_data_dir(dmreq->ctx->bio_in) != WRITE)
- return 0;
-
- /* Apply whitening on ciphertext */
- dst = kmap_atomic(sg_page(&dmreq->sg_out));
- r = crypt_iv_tcw_whitening(cc, dmreq, dst + dmreq->sg_out.offset);
- kunmap_atomic(dst);
-
- return r;
-}
-
-static struct crypt_iv_operations crypt_iv_plain_ops = {
- .generator = crypt_iv_plain_gen
-};
-
-static struct crypt_iv_operations crypt_iv_plain64_ops = {
- .generator = crypt_iv_plain64_gen
-};
-
-static struct crypt_iv_operations crypt_iv_essiv_ops = {
- .ctr = crypt_iv_essiv_ctr,
- .dtr = crypt_iv_essiv_dtr,
- .init = crypt_iv_essiv_init,
- .wipe = crypt_iv_essiv_wipe,
- .generator = crypt_iv_essiv_gen
-};
-
-static struct crypt_iv_operations crypt_iv_benbi_ops = {
- .ctr = crypt_iv_benbi_ctr,
- .dtr = crypt_iv_benbi_dtr,
- .generator = crypt_iv_benbi_gen
-};
-
-static struct crypt_iv_operations crypt_iv_null_ops = {
- .generator = crypt_iv_null_gen
-};
-
-static struct crypt_iv_operations crypt_iv_lmk_ops = {
- .ctr = crypt_iv_lmk_ctr,
- .dtr = crypt_iv_lmk_dtr,
- .init = crypt_iv_lmk_init,
- .wipe = crypt_iv_lmk_wipe,
- .generator = crypt_iv_lmk_gen,
- .post = crypt_iv_lmk_post
-};
-
-static struct crypt_iv_operations crypt_iv_tcw_ops = {
- .ctr = crypt_iv_tcw_ctr,
- .dtr = crypt_iv_tcw_dtr,
- .init = crypt_iv_tcw_init,
- .wipe = crypt_iv_tcw_wipe,
- .generator = crypt_iv_tcw_gen,
- .post = crypt_iv_tcw_post
-};
-
static void crypt_convert_init(struct crypt_config *cc,
struct convert_context *ctx,
struct bio *bio_out, struct bio *bio_in,
@@ -836,52 +236,6 @@ static u8 *iv_of_dmreq(struct crypt_config *cc,
crypto_skcipher_alignmask(any_tfm(cc)) + 1);
}
-static int crypt_convert_block(struct crypt_config *cc,
- struct convert_context *ctx,
- struct skcipher_request *req)
-{
- struct bio_vec bv_in = bio_iter_iovec(ctx->bio_in, ctx->iter_in);
- struct bio_vec bv_out = bio_iter_iovec(ctx->bio_out, ctx->iter_out);
- struct dm_crypt_request *dmreq;
- u8 *iv;
- int r;
-
- dmreq = dmreq_of_req(cc, req);
- iv = iv_of_dmreq(cc, dmreq);
-
- dmreq->iv_sector = ctx->cc_sector;
- dmreq->ctx = ctx;
- sg_init_table(&dmreq->sg_in, 1);
- sg_set_page(&dmreq->sg_in, bv_in.bv_page, 1 << SECTOR_SHIFT,
- bv_in.bv_offset);
-
- sg_init_table(&dmreq->sg_out, 1);
- sg_set_page(&dmreq->sg_out, bv_out.bv_page, 1 << SECTOR_SHIFT,
- bv_out.bv_offset);
-
- bio_advance_iter(ctx->bio_in, &ctx->iter_in, 1 << SECTOR_SHIFT);
- bio_advance_iter(ctx->bio_out, &ctx->iter_out, 1 << SECTOR_SHIFT);
-
- if (cc->iv_gen_ops) {
- r = cc->iv_gen_ops->generator(cc, iv, dmreq);
- if (r < 0)
- return r;
- }
-
- skcipher_request_set_crypt(req, &dmreq->sg_in, &dmreq->sg_out,
- 1 << SECTOR_SHIFT, iv);
-
- if (bio_data_dir(ctx->bio_in) == WRITE)
- r = crypto_skcipher_encrypt(req);
- else
- r = crypto_skcipher_decrypt(req);
-
- if (!r && cc->iv_gen_ops && cc->iv_gen_ops->post)
- r = cc->iv_gen_ops->post(cc, iv, dmreq);
-
- return r;
-}
-
static void kcryptd_async_done(struct crypto_async_request *async_req,
int error);
@@ -916,57 +270,94 @@ static void crypt_free_req(struct crypt_config *cc,
/*
* Encrypt / decrypt data from one bio to another one (can be the same one)
*/
-static int crypt_convert(struct crypt_config *cc,
- struct convert_context *ctx)
+
+static int crypt_convert_bio(struct crypt_config *cc,
+ struct convert_context *ctx)
{
+ unsigned int cryptlen, n1, n2, nents, i = 0, bytes = 0;
+ struct skcipher_request *req;
+ struct dm_crypt_request *dmreq;
+ struct geniv_req_info rinfo;
+ struct bio_vec bv_in, bv_out;
int r;
+ u8 *iv;
atomic_set(&ctx->cc_pending, 1);
+ crypt_alloc_req(cc, ctx);
- while (ctx->iter_in.bi_size && ctx->iter_out.bi_size) {
+ req = ctx->req;
+ dmreq = dmreq_of_req(cc, req);
+ iv = iv_of_dmreq(cc, dmreq);
- crypt_alloc_req(cc, ctx);
+ n1 = bio_segments(ctx->bio_in);
+ n2 = bio_segments(ctx->bio_in);
+ nents = n1 > n2 ? n1 : n2;
+ nents = nents > MAX_SG_LIST ? MAX_SG_LIST : nents;
+ cryptlen = ctx->iter_in.bi_size;
- atomic_inc(&ctx->cc_pending);
+ DMDEBUG("dm-crypt:%s: segments:[in=%u, out=%u] bi_size=%u\n",
+ bio_data_dir(ctx->bio_in) == WRITE ? "write" : "read",
+ n1, n2, cryptlen);
- r = crypt_convert_block(cc, ctx, ctx->req);
+ dmreq->sg_in = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
+ dmreq->sg_out = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
- switch (r) {
- /*
- * The request was queued by a crypto driver
- * but the driver request queue is full, let's wait.
- */
- case -EBUSY:
- wait_for_completion(&ctx->restart);
- reinit_completion(&ctx->restart);
- /* fall through */
- /*
- * The request is queued and processed asynchronously,
- * completion function kcryptd_async_done() will be called.
- */
- case -EINPROGRESS:
- ctx->req = NULL;
- ctx->cc_sector++;
- continue;
- /*
- * The request was already processed (synchronously).
- */
- case 0:
- atomic_dec(&ctx->cc_pending);
- ctx->cc_sector++;
- cond_resched();
- continue;
-
- /* There was an error while processing the request. */
- default:
- atomic_dec(&ctx->cc_pending);
- return r;
- }
+ dmreq->ctx = ctx;
+
+ sg_init_table(dmreq->sg_in, nents);
+ sg_init_table(dmreq->sg_out, nents);
+
+ while (ctx->iter_in.bi_size && ctx->iter_out.bi_size && i < nents) {
+ bv_in = bio_iter_iovec(ctx->bio_in, ctx->iter_in);
+ bv_out = bio_iter_iovec(ctx->bio_out, ctx->iter_out);
+
+ sg_set_page(&dmreq->sg_in[i], bv_in.bv_page, bv_in.bv_len,
+ bv_in.bv_offset);
+ sg_set_page(&dmreq->sg_out[i], bv_out.bv_page, bv_out.bv_len,
+ bv_out.bv_offset);
+
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in, bv_in.bv_len);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out, bv_out.bv_len);
+
+ bytes += bv_in.bv_len;
+ i++;
}
- return 0;
+ DMDEBUG("dm-crypt: Processed %u of %u bytes\n", bytes, cryptlen);
+
+ rinfo.is_write = bio_data_dir(ctx->bio_in) == WRITE;
+ rinfo.iv_sector = ctx->cc_sector;
+ rinfo.nents = nents;
+ rinfo.iv = iv;
+
+ skcipher_request_set_crypt(req, dmreq->sg_in, dmreq->sg_out,
+ bytes, &rinfo);
+
+ if (bio_data_dir(ctx->bio_in) == WRITE)
+ r = crypto_skcipher_encrypt(req);
+ else
+ r = crypto_skcipher_decrypt(req);
+
+ switch (r) {
+ /* The request was queued so wait. */
+ case -EBUSY:
+ wait_for_completion(&ctx->restart);
+ reinit_completion(&ctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function kcryptd_async_done() is called.
+ */
+ case -EINPROGRESS:
+ ctx->req = NULL;
+ cond_resched();
+ break;
+ }
+
+ return r;
}
+
static void crypt_free_buffer_pages(struct crypt_config *cc, struct bio *clone);
/*
@@ -1072,11 +463,17 @@ static void crypt_dec_pending(struct dm_crypt_io *io)
{
struct crypt_config *cc = io->cc;
struct bio *base_bio = io->base_bio;
+ struct dm_crypt_request *dmreq;
int error = io->error;
if (!atomic_dec_and_test(&io->io_pending))
return;
+ dmreq = dmreq_of_req(cc, io->ctx.req);
+ DMDEBUG("dm-crypt: Freeing scatterlists [sync]\n");
+ kfree(dmreq->sg_in);
+ kfree(dmreq->sg_out);
+
if (io->ctx.req)
crypt_free_req(cc, io->ctx.req, base_bio);
@@ -1315,7 +712,7 @@ static void kcryptd_crypt_write_convert(struct dm_crypt_io *io)
sector += bio_sectors(clone);
crypt_inc_pending(io);
- r = crypt_convert(cc, &io->ctx);
+ r = crypt_convert_bio(cc, &io->ctx);
if (r)
io->error = -EIO;
crypt_finished = atomic_dec_and_test(&io->ctx.cc_pending);
@@ -1345,7 +742,8 @@ static void kcryptd_crypt_read_convert(struct dm_crypt_io *io)
crypt_convert_init(cc, &io->ctx, io->base_bio, io->base_bio,
io->sector);
- r = crypt_convert(cc, &io->ctx);
+ r = crypt_convert_bio(cc, &io->ctx);
+
if (r < 0)
io->error = -EIO;
@@ -1373,12 +771,13 @@ static void kcryptd_async_done(struct crypto_async_request *async_req,
return;
}
- if (!error && cc->iv_gen_ops && cc->iv_gen_ops->post)
- error = cc->iv_gen_ops->post(cc, iv_of_dmreq(cc, dmreq), dmreq);
-
if (error < 0)
io->error = -EIO;
+ DMDEBUG("dm-crypt: Freeing scatterlists and request struct [async]\n");
+ kfree(dmreq->sg_in);
+ kfree(dmreq->sg_out);
+
crypt_free_req(cc, req_of_dmreq(cc, dmreq), io->base_bio);
if (!atomic_dec_and_test(&ctx->cc_pending))
@@ -1471,7 +870,8 @@ static int crypt_alloc_tfms(struct crypt_config *cc, char *ciphermode)
return 0;
}
-static int crypt_setkey_allcpus(struct crypt_config *cc)
+static int crypt_setkey_allcpus(struct crypt_config *cc, enum setkey_op keyop,
+ char *ivmode, char *ivopts)
{
unsigned subkey_size;
int err = 0, i, r;
@@ -1480,9 +880,13 @@ static int crypt_setkey_allcpus(struct crypt_config *cc)
subkey_size = (cc->key_size - cc->key_extra_size) >> ilog2(cc->tfms_count);
for (i = 0; i < cc->tfms_count; i++) {
- r = crypto_skcipher_setkey(cc->tfms[i],
- cc->key + (i * subkey_size),
- subkey_size);
+ DECLARE_GENIV_KEY(kinfo, keyop, cc->tfms_count, cc->cipher,
+ cc->key, cc->key_size,
+ cc->key + (subkey_size * i), subkey_size,
+ cc->key_parts, ivmode, ivopts);
+
+ r = crypto_skcipher_setkey(cc->tfms[i], (u8 *) &kinfo,
+ sizeof(kinfo));
if (r)
err = r;
}
@@ -1490,7 +894,8 @@ static int crypt_setkey_allcpus(struct crypt_config *cc)
return err;
}
-static int crypt_set_key(struct crypt_config *cc, char *key)
+static int crypt_set_key(struct crypt_config *cc, enum setkey_op keyop,
+ char *key, char *ivmode, char *ivopts)
{
int r = -EINVAL;
int key_string_len = strlen(key);
@@ -1508,7 +913,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
set_bit(DM_CRYPT_KEY_VALID, &cc->flags);
- r = crypt_setkey_allcpus(cc);
+ r = crypt_setkey_allcpus(cc, keyop, ivmode, ivopts);
out:
/* Hex key string not needed after here, so wipe it. */
@@ -1517,12 +922,24 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
return r;
}
+static int crypt_init_all_cpus(struct dm_target *ti, char *key,
+ char *ivmode, char *ivopts)
+{
+ struct crypt_config *cc = ti->private;
+ int ret;
+
+ ret = crypt_set_key(cc, SETKEY_OP_INIT, key, ivmode, ivopts);
+ if (ret < 0)
+ ti->error = "Error decoding and setting key";
+ return ret;
+}
+
static int crypt_wipe_key(struct crypt_config *cc)
{
clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
memset(&cc->key, 0, cc->key_size * sizeof(u8));
- return crypt_setkey_allcpus(cc);
+ return crypt_setkey_allcpus(cc, SETKEY_OP_WIPE, NULL, NULL);
}
static void crypt_dtr(struct dm_target *ti)
@@ -1550,9 +967,6 @@ static void crypt_dtr(struct dm_target *ti)
mempool_destroy(cc->page_pool);
mempool_destroy(cc->req_pool);
- if (cc->iv_gen_ops && cc->iv_gen_ops->dtr)
- cc->iv_gen_ops->dtr(cc);
-
if (cc->dev)
dm_put_device(ti, cc->dev);
@@ -1629,8 +1043,16 @@ static int crypt_ctr_cipher(struct dm_target *ti,
if (!cipher_api)
goto bad_mem;
- ret = snprintf(cipher_api, CRYPTO_MAX_ALG_NAME,
- "%s(%s)", chainmode, cipher);
+create_cipher:
+ /* For those ciphers which do not support IVs,
+ * use the 'null' template cipher
+ */
+
+ if (!ivmode)
+ ivmode = "null";
+
+ ret = snprintf(cipher_api, CRYPTO_MAX_ALG_NAME, "%s(%s(%s))",
+ ivmode, chainmode, cipher);
if (ret < 0) {
kfree(cipher_api);
goto bad_mem;
@@ -1652,23 +1074,10 @@ static int crypt_ctr_cipher(struct dm_target *ti,
else if (ivmode) {
DMWARN("Selected cipher does not support IVs");
ivmode = NULL;
+ goto create_cipher;
}
- /* Choose ivmode, see comments at iv code. */
- if (ivmode == NULL)
- cc->iv_gen_ops = NULL;
- else if (strcmp(ivmode, "plain") == 0)
- cc->iv_gen_ops = &crypt_iv_plain_ops;
- else if (strcmp(ivmode, "plain64") == 0)
- cc->iv_gen_ops = &crypt_iv_plain64_ops;
- else if (strcmp(ivmode, "essiv") == 0)
- cc->iv_gen_ops = &crypt_iv_essiv_ops;
- else if (strcmp(ivmode, "benbi") == 0)
- cc->iv_gen_ops = &crypt_iv_benbi_ops;
- else if (strcmp(ivmode, "null") == 0)
- cc->iv_gen_ops = &crypt_iv_null_ops;
- else if (strcmp(ivmode, "lmk") == 0) {
- cc->iv_gen_ops = &crypt_iv_lmk_ops;
+ if (strcmp(ivmode, "lmk") == 0) {
/*
* Version 2 and 3 is recognised according
* to length of provided multi-key string.
@@ -1680,39 +1089,14 @@ static int crypt_ctr_cipher(struct dm_target *ti,
cc->key_extra_size = cc->key_size / cc->key_parts;
}
} else if (strcmp(ivmode, "tcw") == 0) {
- cc->iv_gen_ops = &crypt_iv_tcw_ops;
cc->key_parts += 2; /* IV + whitening */
cc->key_extra_size = cc->iv_size + TCW_WHITENING_SIZE;
- } else {
- ret = -EINVAL;
- ti->error = "Invalid IV mode";
- goto bad;
}
/* Initialize and set key */
- ret = crypt_set_key(cc, key);
- if (ret < 0) {
- ti->error = "Error decoding and setting key";
+ ret = crypt_init_all_cpus(ti, key, ivmode, ivopts);
+ if (ret < 0)
goto bad;
- }
-
- /* Allocate IV */
- if (cc->iv_gen_ops && cc->iv_gen_ops->ctr) {
- ret = cc->iv_gen_ops->ctr(cc, ti, ivopts);
- if (ret < 0) {
- ti->error = "Error creating IV";
- goto bad;
- }
- }
-
- /* Initialize IV (set keys for ESSIV etc) */
- if (cc->iv_gen_ops && cc->iv_gen_ops->init) {
- ret = cc->iv_gen_ops->init(cc);
- if (ret < 0) {
- ti->error = "Error initialising IV";
- goto bad;
- }
- }
ret = 0;
bad:
@@ -1934,8 +1318,9 @@ static int crypt_map(struct dm_target *ti, struct bio *bio)
if (bio_data_dir(io->base_bio) == READ) {
if (kcryptd_io_read(io, GFP_NOWAIT))
kcryptd_queue_read(io);
- } else
+ } else {
kcryptd_queue_crypt(io);
+ }
return DM_MAPIO_SUBMITTED;
}
@@ -2014,7 +1399,6 @@ static void crypt_resume(struct dm_target *ti)
static int crypt_message(struct dm_target *ti, unsigned argc, char **argv)
{
struct crypt_config *cc = ti->private;
- int ret = -EINVAL;
if (argc < 2)
goto error;
@@ -2025,19 +1409,9 @@ static int crypt_message(struct dm_target *ti, unsigned argc, char **argv)
return -EINVAL;
}
if (argc == 3 && !strcasecmp(argv[1], "set")) {
- ret = crypt_set_key(cc, argv[2]);
- if (ret)
- return ret;
- if (cc->iv_gen_ops && cc->iv_gen_ops->init)
- ret = cc->iv_gen_ops->init(cc);
- return ret;
+ return crypt_set_key(cc, SETKEY_OP_SET, argv[2], 0, 0);
}
if (argc == 2 && !strcasecmp(argv[1], "wipe")) {
- if (cc->iv_gen_ops && cc->iv_gen_ops->wipe) {
- ret = cc->iv_gen_ops->wipe(cc);
- if (ret)
- return ret;
- }
return crypt_wipe_key(cc);
}
}
diff --git a/include/crypto/geniv.h b/include/crypto/geniv.h
new file mode 100644
index 0000000..df9f953
--- /dev/null
+++ b/include/crypto/geniv.h
@@ -0,0 +1,60 @@
+/*
+ * geniv: common data structures for IV generation algorithms
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+#ifndef _CRYPTO_GENIV_
+#define _CRYPTO_GENIV_
+
+#define SECTOR_SHIFT 9
+#define SECTOR_SIZE (1 << SECTOR_SHIFT)
+
+#define LMK_SEED_SIZE 64 /* hash + 0 */
+#define TCW_WHITENING_SIZE 16
+
+enum setkey_op {
+ SETKEY_OP_INIT,
+ SETKEY_OP_SET,
+ SETKEY_OP_WIPE,
+};
+
+struct geniv_key_info {
+ enum setkey_op keyop;
+ unsigned int tfms_count;
+ char *cipher;
+ u8 *key;
+ u8 *subkey;
+ unsigned int key_size;
+ unsigned int subkey_size;
+ unsigned int key_parts;
+ char *ivmode;
+ char *ivopts;
+};
+
+#define DECLARE_GENIV_KEY(c, op, n, p, k, sz, skey, ssz, kp, m, opts) \
+ struct geniv_key_info c = { \
+ .keyop = op, \
+ .tfms_count = n, \
+ .cipher = p, \
+ .key = k, \
+ .key_size = sz, \
+ .subkey = skey, \
+ .subkey_size = ssz, \
+ .key_parts = kp, \
+ .ivmode = m, \
+ .ivopts = opts, \
+ }
+
+struct geniv_req_info {
+ bool is_write;
+ sector_t iv_sector;
+ unsigned int nents;
+ u8 *iv;
+};
+
+#endif
+
--
Binoy Jayan
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 8:49 ` [RFC PATCH v2] crypto: Add IV generation algorithms Binoy Jayan
@ 2016-12-13 10:01 ` Milan Broz
2016-12-14 6:09 ` Binoy Jayan
` (2 more replies)
2017-01-03 14:23 ` Gilad Ben-Yossef
2017-01-11 14:55 ` Ondrej Mosnáček
2 siblings, 3 replies; 17+ messages in thread
From: Milan Broz @ 2016-12-13 10:01 UTC (permalink / raw)
To: Binoy Jayan, Oded, Ofir
Cc: Herbert Xu, David S. Miller, linux-crypto, Mark Brown,
Arnd Bergmann, linux-kernel, Alasdair Kergon, Mike Snitzer,
dm-devel, Shaohua Li, linux-raid, Rajendra
On 12/13/2016 09:49 AM, Binoy Jayan wrote:
> Currently, the iv generation algorithms are implemented in dm-crypt.c.
> The goal is to move these algorithms from the dm layer to the kernel
> crypto layer by implementing them as template ciphers so they can be
> implemented in hardware for performance. As part of this patchset, the
> iv-generation code is moved from the dm layer to the crypto layer and
> adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
> at a time. Each bio contains the in memory representation of physically
> contiguous disk blocks. The dm layer sets up a chained scatterlist of
> these blocks split into physically contiguous segments in memory so that
> DMA can be performed. The iv generation algorithms implemented in geniv.c
> include plain, plain64, essiv, benbi, null, lmk and tcw.
...
Just few notes...
The lmk (loopAES) and tcw (old TrueCrypt mode) IVs are in fact hacks,
it is combination of IV and modification of CBC mode (in post calls).
It is used only in these disk-encryption systems, it does not make sense
to use it elsewhere (moreover tcw IV is not safe, it is here only
for compatibility reasons).
I think that IV generators should not modify or read encrypted data directly,
it should only generate IV.
By the move everything to cryptoAPI we are basically introducing some strange mix
of IV and modes there, I wonder how this is going to be maintained.
Anyway, Herbert should say if it is ok...
But I would imagine that cryptoAPI should implement only "real" IV generators
and these disk-encryption add-ons keep inside dmcrypt.
(and dmcrypt should probably use normal IVs through crypto API and
build some wrapper around for hacks...)
...
>
> When using multiple keys with the original dm-crypt, the key selection is
> made based on the sector number as:
>
> key_index = sector & (key_count - 1)
>
> This restricts the usage of the same key for encrypting/decrypting a
> single bio. One way to solve this is to move the key management code from
> dm-crypt to cryto layer. But this seems tricky when using template ciphers
> because, when multiple ciphers are instantiated from dm layer, each cipher
> instance set with a unique subkey (part of the bigger master key) and
> these instances themselves do not have access to each other's instances
> or contexts. This way, a single instance cannot encryt/decrypt a whole bio.
> This has to be fixed.
I really do not think the disk encryption key management should be moved
outside of dm-crypt. We cannot then change key structure later easily.
But it is not only key management, you are basically moving much more internals
outside of dm-crypt:
> +struct geniv_ctx {
> + struct crypto_skcipher *child;
> + unsigned int tfms_count;
> + char *ivmode;
> + unsigned int iv_size;
> + char *ivopts;
> + char *cipher;
> + struct geniv_operations *iv_gen_ops;
> + union {
> + struct geniv_essiv_private essiv;
> + struct geniv_benbi_private benbi;
> + struct geniv_lmk_private lmk;
> + struct geniv_tcw_private tcw;
> + } iv_gen_private;
> + void *iv_private;
> + struct crypto_skcipher *tfm;
> + unsigned int key_size;
> + unsigned int key_extra_size;
> + unsigned int key_parts; /* independent parts in key buffer */
^^^ these key sizes you probably mean by key management.
It is based on way how the key is currently sent into kernel
(one hexa string in ioctl that needs to be split) and have to be changed in future.
> + enum setkey_op keyop;
> + char *msg;
> + u8 *key;
Milan
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 10:01 ` Milan Broz
@ 2016-12-14 6:09 ` Binoy Jayan
2016-12-16 5:55 ` Binoy Jayan
2016-12-22 8:55 ` Herbert Xu
2 siblings, 0 replies; 17+ messages in thread
From: Binoy Jayan @ 2016-12-14 6:09 UTC (permalink / raw)
To: Milan Broz
Cc: Oded, Ofir, Herbert Xu, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
Hi Milan,
Thank you for the reply.
On 13 December 2016 at 15:31, Milan Broz <gmazyland@gmail.com> wrote:
> I really do not think the disk encryption key management should be moved
> outside of dm-crypt. We cannot then change key structure later easily.
Yes, I agree. but the key selection based on sector number restricts the
option of having a larger block size used for encryption.
>> + unsigned int key_size;
>> + unsigned int key_extra_size;
>> + unsigned int key_parts; /* independent parts in key buffer */
>
> ^^^ these key sizes you probably mean by key management.
Yes, I mean splitting the keys into subkeys based on the keycount
parameter (as mentioned below) to the dm-crypt.
cipher[:keycount]-mode-iv:ivopts
aes:2-cbc-essiv:sha256
> It is based on way how the key is currently sent into kernel
> (one hexa string in ioctl that needs to be split) and have to be changed in future.
-Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 10:01 ` Milan Broz
2016-12-14 6:09 ` Binoy Jayan
@ 2016-12-16 5:55 ` Binoy Jayan
2016-12-22 8:55 ` Herbert Xu
2 siblings, 0 replies; 17+ messages in thread
From: Binoy Jayan @ 2016-12-16 5:55 UTC (permalink / raw)
To: Milan Broz
Cc: Oded, Ofir, Herbert Xu, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
Hi Milan,
On 13 December 2016 at 15:31, Milan Broz <gmazyland@gmail.com> wrote:
> I think that IV generators should not modify or read encrypted data directly,
> it should only generate IV.
I was trying to find more information about what you said and how a
iv generator should be written. I saw two examples of IV generators
too used with AEAD ciphers (crypto/seqiv.c and crypto/echainiv.c)
Excerpt from crypto api doc:
http://www.chronox.de/crypto-API/crypto/architecture.html#crypto-api-cipher-references-and-priority
2. Now, SEQIV uses the AEAD API function calls to invoke the associated
AEAD cipher. In our case, during the instantiation of SEQIV, the cipher
handle for GCM is provided to SEQIV. This means that SEQIV invokes
AEAD cipher operations with the GCM cipher handle.
Here, it says seqiv invokes cipher operations. However the code crypto/seqiv.c
does not look similar to how the modes are implemented which is confusing. I
was looking for an example of an IV generator used with a regular block cipher
and not a AEAD cipher. Could you point me out to some?
Thanks,
Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 10:01 ` Milan Broz
2016-12-14 6:09 ` Binoy Jayan
2016-12-16 5:55 ` Binoy Jayan
@ 2016-12-22 8:55 ` Herbert Xu
2016-12-22 10:55 ` Binoy Jayan
2 siblings, 1 reply; 17+ messages in thread
From: Herbert Xu @ 2016-12-22 8:55 UTC (permalink / raw)
To: Milan Broz
Cc: Binoy Jayan, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, linux-kernel, Alasdair Kergon,
Mike Snitzer, dm-devel, Shaohua Li, linux-raid, Rajendra
On Tue, Dec 13, 2016 at 11:01:08AM +0100, Milan Broz wrote:
>
> By the move everything to cryptoAPI we are basically introducing some strange mix
> of IV and modes there, I wonder how this is going to be maintained.
> Anyway, Herbert should say if it is ok...
Well there is precedent in how do the IPsec IV generation. In
that case the IV generators too are completely specific to that
application, i.e., IPsec. However, the way structured it allowed
us to have one single entry path from the IPsec stack into the
crypto layer regardless of whether you are using AEAD or traditional
encryption/hashing algorithms.
For IPsec we make the IV generators behave like normal AEAD
algorithms, except that they take the sequence number as the IV.
The goal here are obviously different. However, by employing
the same method as we do in IPsec, it appears to me that you
can effectively process multiple blocks at once instead of having
to supply one block at a time due to the IV generation issue.
> I really do not think the disk encryption key management should be moved
> outside of dm-crypt. We cannot then change key structure later easily.
It doesn't have to live outside of dm-crypt. You can register
these IV generators from there if you really want.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-22 8:55 ` Herbert Xu
@ 2016-12-22 10:55 ` Binoy Jayan
2016-12-23 7:51 ` Herbert Xu
0 siblings, 1 reply; 17+ messages in thread
From: Binoy Jayan @ 2016-12-22 10:55 UTC (permalink / raw)
To: Herbert Xu
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
Hi Herbert,
On 22 December 2016 at 14:25, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Tue, Dec 13, 2016 at 11:01:08AM +0100, Milan Broz wrote:
>>
>> By the move everything to cryptoAPI we are basically introducing some strange mix
>> of IV and modes there, I wonder how this is going to be maintained.
>> Anyway, Herbert should say if it is ok...
>
> Well there is precedent in how do the IPsec IV generation. In
> that case the IV generators too are completely specific to that
> application, i.e., IPsec. However, the way structured it allowed
> us to have one single entry path from the IPsec stack into the
> crypto layer regardless of whether you are using AEAD or traditional
> encryption/hashing algorithms.
>
> For IPsec we make the IV generators behave like normal AEAD
> algorithms, except that they take the sequence number as the IV.
>
> The goal here are obviously different. However, by employing
> the same method as we do in IPsec, it appears to me that you
> can effectively process multiple blocks at once instead of having
> to supply one block at a time due to the IV generation issue.
Thank you for clarifying that part.:)
So, I hope we can consider algorithms like lmk and tcw too as IV generation
algorithms, even though they manipulate encrypted data directly?
>> I really do not think the disk encryption key management should be moved
>> outside of dm-crypt. We cannot then change key structure later easily.
I agree with this too, only problem with this is when multiple keys are involved
(where the master key is split into 2 or more), and the key selection is made
with a modular division of the (512-byte) sector number by the number of keys.
> It doesn't have to live outside of dm-crypt. You can register
> these IV generators from there if you really want.
Sorry, but I didn't understand this part.
Thanks,
Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-22 10:55 ` Binoy Jayan
@ 2016-12-23 7:51 ` Herbert Xu
2016-12-29 9:23 ` Binoy Jayan
0 siblings, 1 reply; 17+ messages in thread
From: Herbert Xu @ 2016-12-23 7:51 UTC (permalink / raw)
To: Binoy Jayan
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
On Thu, Dec 22, 2016 at 04:25:12PM +0530, Binoy Jayan wrote:
>
> > It doesn't have to live outside of dm-crypt. You can register
> > these IV generators from there if you really want.
>
> Sorry, but I didn't understand this part.
What I mean is that moving the IV generators into the crypto API
does not mean the dm-crypt team giving up control over them. You
could continue to keep them within the dm-crypt code base and
still register them through the normal crypto API mechanism.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-23 7:51 ` Herbert Xu
@ 2016-12-29 9:23 ` Binoy Jayan
2016-12-30 10:27 ` Herbert Xu
0 siblings, 1 reply; 17+ messages in thread
From: Binoy Jayan @ 2016-12-29 9:23 UTC (permalink / raw)
To: Herbert Xu
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
Hi Herbert,
Sorry for the delayed response, I was busy with testing dm-crypt
with bonnie++ for regressions. I tried to find some alternative
way to keep the IV algorithms' registration in the dm-crypt.
Also there were some changes done in dm-crypt keys structure too
recently.
c538f6e dm crypt: add ability to use keys from the kernel key retention service
On Thu, Dec 22, 2016 at 04:25:12PM +0530, Binoy Jayan wrote:
>
> > It doesn't have to live outside of dm-crypt. You can register
> > these IV generators from there if you really want.
>
> Sorry, but I didn't understand this part.
What I mean is that moving the IV generators into the crypto API
does not mean the dm-crypt team giving up control over them. You
could continue to keep them within the dm-crypt code base and
still register them through the normal crypto API mechanism
When we keep these in dm-crypt and if more than one key is used
(it is actually more than one parts of the original key),
there are more than one cipher instance created - one for each
unique part of the key. Since the crypto requests are modelled
to go through the template ciphers in the order:
"essiv -> cbc -> aes"
a particular cipher instance of the IV (essiv in this example) is
responsible to encrypt an entire bigger block. If this bigger block
is to be later split into 512 bytes blocks and then encrypted using
the other cipher instance depending on the following formula:
key_index = sector & (key_count - 1)
it is not possible as the cipher instances do not have access to
each other's instances. So, number of keys used is crucial while
performing encryption.
If there was only a single key, it should not have been a problem.
But if there are more than one key, then encrypting a bigger block
with a single key would cause backward incompatibility.
I was wondering if this is acceptable.
bigger block: What I mean by bigger block here is the set of 512-byte
blocks that dm-crypt can be optimized to process at once.
Thanks,
Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-29 9:23 ` Binoy Jayan
@ 2016-12-30 10:27 ` Herbert Xu
2017-01-02 6:46 ` Binoy Jayan
0 siblings, 1 reply; 17+ messages in thread
From: Herbert Xu @ 2016-12-30 10:27 UTC (permalink / raw)
To: Binoy Jayan
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
On Thu, Dec 29, 2016 at 02:53:25PM +0530, Binoy Jayan wrote:
>
> When we keep these in dm-crypt and if more than one key is used
> (it is actually more than one parts of the original key),
> there are more than one cipher instance created - one for each
> unique part of the key. Since the crypto requests are modelled
> to go through the template ciphers in the order:
>
> "essiv -> cbc -> aes"
>
> a particular cipher instance of the IV (essiv in this example) is
> responsible to encrypt an entire bigger block. If this bigger block
> is to be later split into 512 bytes blocks and then encrypted using
> the other cipher instance depending on the following formula:
>
> key_index = sector & (key_count - 1)
This is just a matter of structuring the key for the IV generator.
The IV generator's key in this case should be a combination of the
key to the underlying CBC plus the set of all keys for the IV
generator itself. It should then allocate the required number of
tfms as is currently done by crypt_alloc_tfms in dm-crypt.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-30 10:27 ` Herbert Xu
@ 2017-01-02 6:46 ` Binoy Jayan
2017-01-02 6:53 ` Herbert Xu
0 siblings, 1 reply; 17+ messages in thread
From: Binoy Jayan @ 2017-01-02 6:46 UTC (permalink / raw)
To: Herbert Xu
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
Hi Herbert,
On 30 December 2016 at 15:57, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> This is just a matter of structuring the key for the IV generator.
> The IV generator's key in this case should be a combination of the
> key to the underlying CBC plus the set of all keys for the IV
> generator itself. It should then allocate the required number of
> tfms as is currently done by crypt_alloc_tfms in dm-crypt.
Since I used template ciphers for the iv algorithms, I use
crypto_spawn_skcipher_alg and skcipher_register_instance
for creating the underlying cbc algorithm. I guess you suggest
to change that to make use of crypto_alloc_skcipher.
Even if ciphers are allocated this way, all the encryption requests
for cbc should still go through IV generators? So that should mean,
create one instance of IV generator using 'crypto_alloc_skcipher'
and create tfms_count instances of the generator depending on the
number of keys.
Thanks,
Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2017-01-02 6:46 ` Binoy Jayan
@ 2017-01-02 6:53 ` Herbert Xu
2017-01-02 7:05 ` Binoy Jayan
2017-01-05 6:06 ` Binoy Jayan
0 siblings, 2 replies; 17+ messages in thread
From: Herbert Xu @ 2017-01-02 6:53 UTC (permalink / raw)
To: Binoy Jayan
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
On Mon, Jan 02, 2017 at 12:16:45PM +0530, Binoy Jayan wrote:
>
> Even if ciphers are allocated this way, all the encryption requests
> for cbc should still go through IV generators? So that should mean,
> create one instance of IV generator using 'crypto_alloc_skcipher'
> and create tfms_count instances of the generator depending on the
> number of keys.
Right. The actual number of underlying tfms that do the work
won't change compared to the status quo. We're just structuring
it such that if the overall scheme is supported by the hardware
then we can feed more than one sector at a time to it.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2017-01-02 6:53 ` Herbert Xu
@ 2017-01-02 7:05 ` Binoy Jayan
2017-01-05 6:06 ` Binoy Jayan
1 sibling, 0 replies; 17+ messages in thread
From: Binoy Jayan @ 2017-01-02 7:05 UTC (permalink / raw)
To: Herbert Xu
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
On 2 January 2017 at 12:23, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Mon, Jan 02, 2017 at 12:16:45PM +0530, Binoy Jayan wrote:
>>
>> Even if ciphers are allocated this way, all the encryption requests
>> for cbc should still go through IV generators? So that should mean,
>> create one instance of IV generator using 'crypto_alloc_skcipher'
>> and create tfms_count instances of the generator depending on the
>> number of keys.
>
> Right. The actual number of underlying tfms that do the work
> won't change compared to the status quo. We're just structuring
> it such that if the overall scheme is supported by the hardware
> then we can feed more than one sector at a time to it.
Thank you Herbert.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 8:49 ` [RFC PATCH v2] crypto: Add IV generation algorithms Binoy Jayan
2016-12-13 10:01 ` Milan Broz
@ 2017-01-03 14:23 ` Gilad Ben-Yossef
2017-01-04 5:20 ` Binoy Jayan
2017-01-11 14:55 ` Ondrej Mosnáček
2 siblings, 1 reply; 17+ messages in thread
From: Gilad Ben-Yossef @ 2017-01-03 14:23 UTC (permalink / raw)
To: Binoy Jayan
Cc: Oded, Ofir, Herbert Xu, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, linux-kernel, Alasdair Kergon,
Mike Snitzer, dm-devel, Shaohua Li, linux-raid, Rajendra,
gilad.benyossef
Hi Binoy,
On Tue, Dec 13, 2016 at 02:19:09PM +0530, Binoy Jayan wrote:
> Currently, the iv generation algorithms are implemented in dm-crypt.c.
> The goal is to move these algorithms from the dm layer to the kernel
> crypto layer by implementing them as template ciphers so they can be
> implemented in hardware for performance. As part of this patchset, the
> iv-generation code is moved from the dm layer to the crypto layer and
> adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
> at a time. Each bio contains the in memory representation of physically
> contiguous disk blocks. The dm layer sets up a chained scatterlist of
> these blocks split into physically contiguous segments in memory so that
> DMA can be performed. The iv generation algorithms implemented in geniv.c
> include plain, plain64, essiv, benbi, null, lmk and tcw.
>
Good idea. I wanted to test the patch but alas it does not apply cleanly.
You seem to have a blank line at the end of files and other small
transgressions that makes checkpatch grumpy.
<snip>
Also...
>
> Not-signed-off-by: Binoy Jayan <binoy.jayan@linaro.org>
What is Not-signed-off-by ? :-)
Thanks,
Gilad Ben-Yossef
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2017-01-03 14:23 ` Gilad Ben-Yossef
@ 2017-01-04 5:20 ` Binoy Jayan
0 siblings, 0 replies; 17+ messages in thread
From: Binoy Jayan @ 2017-01-04 5:20 UTC (permalink / raw)
To: Gilad Ben-Yossef
Cc: Oded, Ofir, Herbert Xu, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra, gilad.benyossef
Hi Gilad,
On 3 January 2017 at 19:53, Gilad Ben-Yossef <gilad@benyossef.com> wrote:
> Good idea. I wanted to test the patch but alas it does not apply cleanly.
> You seem to have a blank line at the end of files and other small
> transgressions that makes checkpatch grumpy.
I think that is because there were some key structure changes in dm-crypt
after I sent out v2. I have resolved them while working on v3. Please wait for
the next version of the patchset. I'll send it probably by next week.
I wanted to incorporate a few changes suggested by Herbert before sending them.
> What is Not-signed-off-by ? :-)
It was just an RFC patch, not ready for merging.
Thanks,
Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2017-01-02 6:53 ` Herbert Xu
2017-01-02 7:05 ` Binoy Jayan
@ 2017-01-05 6:06 ` Binoy Jayan
1 sibling, 0 replies; 17+ messages in thread
From: Binoy Jayan @ 2017-01-05 6:06 UTC (permalink / raw)
To: Herbert Xu
Cc: Milan Broz, Oded, Ofir, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, Linux kernel mailing list,
Alasdair Kergon, Mike Snitzer, dm-devel, Shaohua Li, linux-raid,
Rajendra
Hi Herbert,
On 2 January 2017 at 12:23, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Mon, Jan 02, 2017 at 12:16:45PM +0530, Binoy Jayan wrote:
>
> Right. The actual number of underlying tfms that do the work
> won't change compared to the status quo. We're just structuring
> it such that if the overall scheme is supported by the hardware
> then we can feed more than one sector at a time to it.
I was thinking of continuing to have the iv generation algorithms as template
ciphers instead of regular 'skcipher' as it is easier to inherit the parameters
from the underlying cipher (e.g. aes) like cra_blocksize, cra_alignmask,
ivsize, chunksize etc.
Usually, the underlying cipher for the template ciphers are instantiated
in the following function:
skcipher_instance:skcipher_alg:init()
Since the number of such cipher instances depend on the key count, which is
not known at the time of creation of the cipher (it's passed to as an argument
to the setkey api), the creation of those have to be delayed until the setkey
operation of the template cipher. But as Mark pointed out, the users of this
cipher may get confused if the creation of the underlying cipher fails while
trying to do a 'setkey' on the template cipher. I was wondering if I can create
a single instance of the cipher and assign it to tfms[0] and allocate the
remaining instances when the setkey operation is called later with the encoded
key_count so that errors during cipher creation are uncovered earlier.
Thanks,
Binoy
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [RFC PATCH v2] crypto: Add IV generation algorithms
2016-12-13 8:49 ` [RFC PATCH v2] crypto: Add IV generation algorithms Binoy Jayan
2016-12-13 10:01 ` Milan Broz
2017-01-03 14:23 ` Gilad Ben-Yossef
@ 2017-01-11 14:55 ` Ondrej Mosnáček
2 siblings, 0 replies; 17+ messages in thread
From: Ondrej Mosnáček @ 2017-01-11 14:55 UTC (permalink / raw)
To: Binoy Jayan
Cc: Oded, Ofir, Herbert Xu, David S. Miller, linux-crypto,
Mark Brown, Arnd Bergmann, linux-kernel, Alasdair Kergon,
Mike Snitzer, dm-devel, Shaohua Li, linux-raid, Rajendra
Hi Binoy,
2016-12-13 9:49 GMT+01:00 Binoy Jayan <binoy.jayan@linaro.org>:
> Currently, the iv generation algorithms are implemented in dm-crypt.c.
> The goal is to move these algorithms from the dm layer to the kernel
> crypto layer by implementing them as template ciphers so they can be
> implemented in hardware for performance. As part of this patchset, the
> iv-generation code is moved from the dm layer to the crypto layer and
> adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
> at a time. Each bio contains the in memory representation of physically
> contiguous disk blocks. The dm layer sets up a chained scatterlist of
> these blocks split into physically contiguous segments in memory so that
> DMA can be performed. The iv generation algorithms implemented in geniv.c
> include plain, plain64, essiv, benbi, null, lmk and tcw.
I like what you are trying to achieve, however I don't think the
solution you are heading towards (passing sector number to a special
crypto template) would be the best approach here. Milan is currently
trying to add authenticated encryption support to dm-crypt (see [1])
and as part of this change, a new random IV mode would be introduced.
This mode generates a random IV for each sector write, includes it in
the authenticated data and stores it in the sector's metadata (in a
separate part of the disk). In this case dm-crypt will need to have
control over the IV generation (or at least be able to somehow
retrieve it after the crypto operation).
That said, I believe a different approach would be preferable here. I
would suggest, instead of moving the IV generation to the crypto
layer, to add a new type of request to skcipher API (let's call it
'skcipher_bulk_request'), which could be used to submit several
messages at once (together in a single sg list), each with their own
IV, to a skcipher. This would allow drivers to optimize handling of
such requests (e.g. the SIMD ciphers could call kernel_fpu_begin/end
just once for the whole request). It could be done in such a way, that
implementing this type of requests would be optional and a fallback
implementation, which would just split the request into regular
skcipher_requests, would be automatically set for the ciphers that do
not set it themselves. That way this would require no changes to
crypto drivers in the beginning and optimizations could be added
incrementally.
The advantage of this approach to handling such "bulk" requests is
that crypto drivers could just optimize regular algorithms (xts(aes),
cbc(aes), etc.) and wouldn't need to mess with dm-crypt-specific IV
generation. This also means that other users that could potentially
benefit from bulking requests (perhaps network stack?) could use the
same functionality.
I have been playing with this idea for some time now and I should have
an RFC patchset ready soon...
Binoy, Herbert, what do you think about such approach?
[1] https://www.redhat.com/archives/dm-devel/2017-January/msg00028.html
> When using multiple keys with the original dm-crypt, the key selection is
> made based on the sector number as:
>
> key_index = sector & (key_count - 1)
>
> This restricts the usage of the same key for encrypting/decrypting a
> single bio. One way to solve this is to move the key management code from
> dm-crypt to cryto layer. But this seems tricky when using template ciphers
> because, when multiple ciphers are instantiated from dm layer, each cipher
> instance set with a unique subkey (part of the bigger master key) and
> these instances themselves do not have access to each other's instances
> or contexts. This way, a single instance cannot encryt/decrypt a whole bio.
> This has to be fixed.
Please note that the "keycount" parameter was added to dm-crypt solely
for the purpose of implementing the loop-AES partition format. In
general, the security benefit gained by using keycount > 1 is
debatable, so it does not really make sense to use it for anything
else than accessing legacy loopAES partitions. Since Milan decided to
add it as a generic parameter, instead of hard-coding the
functionality for the LMK mode, it can be technically used also in
other combinations, but IMHO it is perfectly reasonable to just give
up on optimizing the cases when keycount > 1. I believe the loop-AES
partition support is just not that important :)
Thanks,
Ondrej
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2017-01-11 14:55 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-13 8:49 [RFC PATCH v2] IV Generation algorithms for dm-crypt Binoy Jayan
2016-12-13 8:49 ` [RFC PATCH v2] crypto: Add IV generation algorithms Binoy Jayan
2016-12-13 10:01 ` Milan Broz
2016-12-14 6:09 ` Binoy Jayan
2016-12-16 5:55 ` Binoy Jayan
2016-12-22 8:55 ` Herbert Xu
2016-12-22 10:55 ` Binoy Jayan
2016-12-23 7:51 ` Herbert Xu
2016-12-29 9:23 ` Binoy Jayan
2016-12-30 10:27 ` Herbert Xu
2017-01-02 6:46 ` Binoy Jayan
2017-01-02 6:53 ` Herbert Xu
2017-01-02 7:05 ` Binoy Jayan
2017-01-05 6:06 ` Binoy Jayan
2017-01-03 14:23 ` Gilad Ben-Yossef
2017-01-04 5:20 ` Binoy Jayan
2017-01-11 14:55 ` Ondrej Mosnáček
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).