[PATCH 0/7] Add support for AEAD algorithms in Qualcomm Crypto Engine driver

[PATCH 0/7] Add support for AEAD algorithms in Qualcomm Crypto Engine driver

[Thara Gopinath]

Enable support for AEAD algorithms in Qualcomm CE driver.  The first three
patches in this series are cleanups and add a few missing pieces required
to add support for AEAD algorithms.  Patch 4 introduces supported AEAD
transformations on Qualcomm CE.  Patches 5 and 6 implements the h/w
infrastructure needed to enable and run the AEAD transformations on
Qualcomm CE.  Patch 7 adds support to queue fallback algorithms in case of
unsupported special inputs.

This series is dependant on https://lkml.org/lkml/2021/2/11/1052.

Thara Gopinath (7):
  crypto: qce: common: Add MAC failed error checking
  crypto: qce: common: Make result dump optional
  crypto: qce: Add mode for rfc4309
  crypto: qce: Add support for AEAD algorithms
  crypto: qce: common: Clean up qce_auth_cfg
  crypto: qce: common: Add support for AEAD algorithms
  crypto: qce: aead: Schedule fallback algorithm

 drivers/crypto/Kconfig      |  15 +
 drivers/crypto/qce/Makefile |   1 +
 drivers/crypto/qce/aead.c   | 817 ++++++++++++++++++++++++++++++++++++
 drivers/crypto/qce/aead.h   |  56 +++
 drivers/crypto/qce/common.c | 198 ++++++++-
 drivers/crypto/qce/common.h |   9 +-
 drivers/crypto/qce/core.c   |   4 +
 7 files changed, 1077 insertions(+), 23 deletions(-)
 create mode 100644 drivers/crypto/qce/aead.c
 create mode 100644 drivers/crypto/qce/aead.h

-- 
2.25.1

[PATCH 1/7] crypto: qce: common: Add MAC failed error checking

[Thara Gopinath]

MAC_FAILED gets set in the status register if authenthication fails
for ccm algorithms(during decryption). Add support to catch and flag
this error.

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/qce/common.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
index dceb9579d87a..7c3cb483749e 100644
--- a/drivers/crypto/qce/common.c
+++ b/drivers/crypto/qce/common.c
@@ -403,7 +403,8 @@ int qce_start(struct crypto_async_request *async_req, u32 type)
 }
 
 #define STATUS_ERRORS	\
-		(BIT(SW_ERR_SHIFT) | BIT(AXI_ERR_SHIFT) | BIT(HSD_ERR_SHIFT))
+		(BIT(SW_ERR_SHIFT) | BIT(AXI_ERR_SHIFT) |	\
+		 BIT(HSD_ERR_SHIFT) | BIT(MAC_FAILED_SHIFT))
 
 int qce_check_status(struct qce_device *qce, u32 *status)
 {
@@ -417,8 +418,12 @@ int qce_check_status(struct qce_device *qce, u32 *status)
 	 * use result_status from result dump the result_status needs to be byte
 	 * swapped, since we set the device to little endian.
 	 */
-	if (*status & STATUS_ERRORS || !(*status & BIT(OPERATION_DONE_SHIFT)))
-		ret = -ENXIO;
+	if (*status & STATUS_ERRORS || !(*status & BIT(OPERATION_DONE_SHIFT))) {
+		if (*status & BIT(MAC_FAILED_SHIFT))
+			ret = -EBADMSG;
+		else
+			ret = -ENXIO;
+	}
 
 	return ret;
 }
-- 
2.25.1

Re: [PATCH 1/7] crypto: qce: common: Add MAC failed error checking

[Bjorn Andersson]

On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:

> MAC_FAILED gets set in the status register if authenthication fails
> for ccm algorithms(during decryption). Add support to catch and flag
> this error.
> 
> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> ---
>  drivers/crypto/qce/common.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
> index dceb9579d87a..7c3cb483749e 100644
> --- a/drivers/crypto/qce/common.c
> +++ b/drivers/crypto/qce/common.c
> @@ -403,7 +403,8 @@ int qce_start(struct crypto_async_request *async_req, u32 type)
>  }
>  
>  #define STATUS_ERRORS	\
> -		(BIT(SW_ERR_SHIFT) | BIT(AXI_ERR_SHIFT) | BIT(HSD_ERR_SHIFT))
> +		(BIT(SW_ERR_SHIFT) | BIT(AXI_ERR_SHIFT) |	\
> +		 BIT(HSD_ERR_SHIFT) | BIT(MAC_FAILED_SHIFT))
>  
>  int qce_check_status(struct qce_device *qce, u32 *status)
>  {
> @@ -417,8 +418,12 @@ int qce_check_status(struct qce_device *qce, u32 *status)
>  	 * use result_status from result dump the result_status needs to be byte
>  	 * swapped, since we set the device to little endian.
>  	 */
> -	if (*status & STATUS_ERRORS || !(*status & BIT(OPERATION_DONE_SHIFT)))
> -		ret = -ENXIO;
> +	if (*status & STATUS_ERRORS || !(*status & BIT(OPERATION_DONE_SHIFT))) {
> +		if (*status & BIT(MAC_FAILED_SHIFT))

Afaict MAC_FAILED indicates a different category of errors from the
others. So I would prefer that the conditionals are flattened.

Is OPERATION_DONE set when MAC_FAILED?

If so:

if (errors || !done)
	return -ENXIO;
else if (*status & BIT(MAC_FAILED))
	return -EBADMSG;

Would be cleaner in my opinion.

Regards,
Bjorn

> +			ret = -EBADMSG;
> +		else
> +			ret = -ENXIO;
> +	}
>  
>  	return ret;
>  }
> -- 
> 2.25.1
> 

Re: [PATCH 1/7] crypto: qce: common: Add MAC failed error checking

[Thara Gopinath]

Hi Bjorn,

Thanks for the reviews.
I realized that I had these replies in my draft for a while and forgot 
to send them!

On 4/5/21 1:36 PM, Bjorn Andersson wrote:
> On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> 
>> MAC_FAILED gets set in the status register if authenthication fails
>> for ccm algorithms(during decryption). Add support to catch and flag
>> this error.
>>
>> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
>> ---
>>   drivers/crypto/qce/common.c | 11 ++++++++---
>>   1 file changed, 8 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
>> index dceb9579d87a..7c3cb483749e 100644
>> --- a/drivers/crypto/qce/common.c
>> +++ b/drivers/crypto/qce/common.c
>> @@ -403,7 +403,8 @@ int qce_start(struct crypto_async_request *async_req, u32 type)
>>   }
>>   
>>   #define STATUS_ERRORS	\
>> -		(BIT(SW_ERR_SHIFT) | BIT(AXI_ERR_SHIFT) | BIT(HSD_ERR_SHIFT))
>> +		(BIT(SW_ERR_SHIFT) | BIT(AXI_ERR_SHIFT) |	\
>> +		 BIT(HSD_ERR_SHIFT) | BIT(MAC_FAILED_SHIFT))
>>   
>>   int qce_check_status(struct qce_device *qce, u32 *status)
>>   {
>> @@ -417,8 +418,12 @@ int qce_check_status(struct qce_device *qce, u32 *status)
>>   	 * use result_status from result dump the result_status needs to be byte
>>   	 * swapped, since we set the device to little endian.
>>   	 */
>> -	if (*status & STATUS_ERRORS || !(*status & BIT(OPERATION_DONE_SHIFT)))
>> -		ret = -ENXIO;
>> +	if (*status & STATUS_ERRORS || !(*status & BIT(OPERATION_DONE_SHIFT))) {
>> +		if (*status & BIT(MAC_FAILED_SHIFT))
> 
> Afaict MAC_FAILED indicates a different category of errors from the
> others. So I would prefer that the conditionals are flattened.
> 
> Is OPERATION_DONE set when MAC_FAILED?
Yes it is. I will change the check to the pattern you have suggested. It 
is less confusing..

> 
> If so:
> 
> if (errors || !done)
> 	return -ENXIO;
> else if (*status & BIT(MAC_FAILED))
> 	return -EBADMSG;
> 
> Would be cleaner in my opinion.
> 
> Regards,
> Bjorn
> 
>> +			ret = -EBADMSG;
>> +		else
>> +			ret = -ENXIO;
>> +	}
>>   
>>   	return ret;
>>   }
>> -- 
>> 2.25.1
>>

-- 
Warm Regards
Thara

[PATCH 2/7] crypto: qce: common: Make result dump optional

[Thara Gopinath]

Qualcomm crypto engine allows for IV registers and status register
to be concatenated to the output. This option is enabled by setting the
RESULTS_DUMP field in GOPROC  register. This is useful for most of the
algorithms to either retrieve status of operation or in case of
authentication algorithms to retrieve the mac. But for ccm
algorithms, the mac is part of the output stream and not retrieved
from the IV registers, thus needing a separate buffer to retrieve it.
Make enabling RESULTS_DUMP field optional so that algorithms can choose
whether or not to enable the option.
Note that in this patch, the enabled algorithms always choose
RESULTS_DUMP to be enabled. But later with the introduction of ccm
algorithms, this changes.

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/qce/common.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
index 7c3cb483749e..2485aa371d83 100644
--- a/drivers/crypto/qce/common.c
+++ b/drivers/crypto/qce/common.c
@@ -88,9 +88,12 @@ static void qce_setup_config(struct qce_device *qce)
 	qce_write(qce, REG_CONFIG, config);
 }
 
-static inline void qce_crypto_go(struct qce_device *qce)
+static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
 {
-	qce_write(qce, REG_GOPROC, BIT(GO_SHIFT) | BIT(RESULTS_DUMP_SHIFT));
+	if (result_dump)
+		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT) | BIT(RESULTS_DUMP_SHIFT));
+	else
+		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
 }
 
 #ifdef CONFIG_CRYPTO_DEV_QCE_SHA
@@ -219,7 +222,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
 	config = qce_config_reg(qce, 1);
 	qce_write(qce, REG_CONFIG, config);
 
-	qce_crypto_go(qce);
+	qce_crypto_go(qce, true);
 
 	return 0;
 }
@@ -380,7 +383,7 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
 	config = qce_config_reg(qce, 1);
 	qce_write(qce, REG_CONFIG, config);
 
-	qce_crypto_go(qce);
+	qce_crypto_go(qce, true);
 
 	return 0;
 }
-- 
2.25.1

Re: [PATCH 2/7] crypto: qce: common: Make result dump optional

[Bjorn Andersson]

On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:

> Qualcomm crypto engine allows for IV registers and status register
> to be concatenated to the output. This option is enabled by setting the
> RESULTS_DUMP field in GOPROC  register. This is useful for most of the
> algorithms to either retrieve status of operation or in case of
> authentication algorithms to retrieve the mac. But for ccm
> algorithms, the mac is part of the output stream and not retrieved
> from the IV registers, thus needing a separate buffer to retrieve it.
> Make enabling RESULTS_DUMP field optional so that algorithms can choose
> whether or not to enable the option.
> Note that in this patch, the enabled algorithms always choose
> RESULTS_DUMP to be enabled. But later with the introduction of ccm
> algorithms, this changes.
> 
> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>

Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>

Regards,
Bjorn

> ---
>  drivers/crypto/qce/common.c | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
> index 7c3cb483749e..2485aa371d83 100644
> --- a/drivers/crypto/qce/common.c
> +++ b/drivers/crypto/qce/common.c
> @@ -88,9 +88,12 @@ static void qce_setup_config(struct qce_device *qce)
>  	qce_write(qce, REG_CONFIG, config);
>  }
>  
> -static inline void qce_crypto_go(struct qce_device *qce)
> +static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
>  {
> -	qce_write(qce, REG_GOPROC, BIT(GO_SHIFT) | BIT(RESULTS_DUMP_SHIFT));
> +	if (result_dump)
> +		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT) | BIT(RESULTS_DUMP_SHIFT));
> +	else
> +		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
>  }
>  
>  #ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> @@ -219,7 +222,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>  	config = qce_config_reg(qce, 1);
>  	qce_write(qce, REG_CONFIG, config);
>  
> -	qce_crypto_go(qce);
> +	qce_crypto_go(qce, true);
>  
>  	return 0;
>  }
> @@ -380,7 +383,7 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
>  	config = qce_config_reg(qce, 1);
>  	qce_write(qce, REG_CONFIG, config);
>  
> -	qce_crypto_go(qce);
> +	qce_crypto_go(qce, true);
>  
>  	return 0;
>  }
> -- 
> 2.25.1
> 

[PATCH 3/7] crypto: qce: Add mode for rfc4309

[Thara Gopinath]

rf4309 is the specification that uses aes ccm algorithms with IPsec
security packets. Add a submode to identify rfc4309 ccm(aes) algorithm
in the crypto driver.

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/qce/common.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/qce/common.h b/drivers/crypto/qce/common.h
index 3bc244bcca2d..3ffe719b79e4 100644
--- a/drivers/crypto/qce/common.h
+++ b/drivers/crypto/qce/common.h
@@ -51,9 +51,11 @@
 #define QCE_MODE_CCM			BIT(12)
 #define QCE_MODE_MASK			GENMASK(12, 8)
 
+#define QCE_MODE_CCM_RFC4309		BIT(13)
+
 /* cipher encryption/decryption operations */
-#define QCE_ENCRYPT			BIT(13)
-#define QCE_DECRYPT			BIT(14)
+#define QCE_ENCRYPT			BIT(14)
+#define QCE_DECRYPT			BIT(15)
 
 #define IS_DES(flags)			(flags & QCE_ALG_DES)
 #define IS_3DES(flags)			(flags & QCE_ALG_3DES)
@@ -73,6 +75,7 @@
 #define IS_CTR(mode)			(mode & QCE_MODE_CTR)
 #define IS_XTS(mode)			(mode & QCE_MODE_XTS)
 #define IS_CCM(mode)			(mode & QCE_MODE_CCM)
+#define IS_CCM_RFC4309(mode)		((mode) & QCE_MODE_CCM_RFC4309)
 
 #define IS_ENCRYPT(dir)			(dir & QCE_ENCRYPT)
 #define IS_DECRYPT(dir)			(dir & QCE_DECRYPT)
-- 
2.25.1

Re: [PATCH 3/7] crypto: qce: Add mode for rfc4309

[Bjorn Andersson]

On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:

> rf4309 is the specification that uses aes ccm algorithms with IPsec
> security packets. Add a submode to identify rfc4309 ccm(aes) algorithm
> in the crypto driver.
> 
> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> ---
>  drivers/crypto/qce/common.h | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/crypto/qce/common.h b/drivers/crypto/qce/common.h
> index 3bc244bcca2d..3ffe719b79e4 100644
> --- a/drivers/crypto/qce/common.h
> +++ b/drivers/crypto/qce/common.h
> @@ -51,9 +51,11 @@
>  #define QCE_MODE_CCM			BIT(12)
>  #define QCE_MODE_MASK			GENMASK(12, 8)
>  
> +#define QCE_MODE_CCM_RFC4309		BIT(13)
> +
>  /* cipher encryption/decryption operations */
> -#define QCE_ENCRYPT			BIT(13)
> -#define QCE_DECRYPT			BIT(14)
> +#define QCE_ENCRYPT			BIT(14)
> +#define QCE_DECRYPT			BIT(15)

Can't we move these further up, so that next time we want to add
something it doesn't require that we also move the ENC/DEC bits?

>  
>  #define IS_DES(flags)			(flags & QCE_ALG_DES)
>  #define IS_3DES(flags)			(flags & QCE_ALG_3DES)
> @@ -73,6 +75,7 @@
>  #define IS_CTR(mode)			(mode & QCE_MODE_CTR)
>  #define IS_XTS(mode)			(mode & QCE_MODE_XTS)
>  #define IS_CCM(mode)			(mode & QCE_MODE_CCM)
> +#define IS_CCM_RFC4309(mode)		((mode) & QCE_MODE_CCM_RFC4309)

While leaving room for the typical macro issues, none of the other
macros wrap the argument in parenthesis. Please follow the style of the
driver, and perhaps follow up with a cleanup patch that just wraps them
all in parenthesis?

Regards,
Bjorn

>  
>  #define IS_ENCRYPT(dir)			(dir & QCE_ENCRYPT)
>  #define IS_DECRYPT(dir)			(dir & QCE_DECRYPT)
> -- 
> 2.25.1
> 

Re: [PATCH 3/7] crypto: qce: Add mode for rfc4309

[Thara Gopinath]

On 4/5/21 6:32 PM, Bjorn Andersson wrote:
> On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> 
>> rf4309 is the specification that uses aes ccm algorithms with IPsec
>> security packets. Add a submode to identify rfc4309 ccm(aes) algorithm
>> in the crypto driver.
>>
>> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
>> ---
>>   drivers/crypto/qce/common.h | 7 +++++--
>>   1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/crypto/qce/common.h b/drivers/crypto/qce/common.h
>> index 3bc244bcca2d..3ffe719b79e4 100644
>> --- a/drivers/crypto/qce/common.h
>> +++ b/drivers/crypto/qce/common.h
>> @@ -51,9 +51,11 @@
>>   #define QCE_MODE_CCM			BIT(12)
>>   #define QCE_MODE_MASK			GENMASK(12, 8)
>>   
>> +#define QCE_MODE_CCM_RFC4309		BIT(13)
>> +
>>   /* cipher encryption/decryption operations */
>> -#define QCE_ENCRYPT			BIT(13)
>> -#define QCE_DECRYPT			BIT(14)
>> +#define QCE_ENCRYPT			BIT(14)
>> +#define QCE_DECRYPT			BIT(15)
> 
> Can't we move these further up, so that next time we want to add
> something it doesn't require that we also move the ENC/DEC bits?

Yes I will change it to BIT(30) and BIT(31)

> 
>>   
>>   #define IS_DES(flags)			(flags & QCE_ALG_DES)
>>   #define IS_3DES(flags)			(flags & QCE_ALG_3DES)
>> @@ -73,6 +75,7 @@
>>   #define IS_CTR(mode)			(mode & QCE_MODE_CTR)
>>   #define IS_XTS(mode)			(mode & QCE_MODE_XTS)
>>   #define IS_CCM(mode)			(mode & QCE_MODE_CCM)
>> +#define IS_CCM_RFC4309(mode)		((mode) & QCE_MODE_CCM_RFC4309)
> 
> While leaving room for the typical macro issues, none of the other
> macros wrap the argument in parenthesis. Please follow the style of the
> driver, and perhaps follow up with a cleanup patch that just wraps them
> all in parenthesis?

This does throw up a checkpatch warning if I don't wrap "mode" in 
parenthesis. How about I keep this for now and I will follow up with a 
clean up for rest of the macros later ?

> 
> Regards,
> Bjorn
> 
>>   
>>   #define IS_ENCRYPT(dir)			(dir & QCE_ENCRYPT)
>>   #define IS_DECRYPT(dir)			(dir & QCE_DECRYPT)
>> -- 
>> 2.25.1
>>

-- 
Warm Regards
Thara

Re: [PATCH 3/7] crypto: qce: Add mode for rfc4309

[Bjorn Andersson]

On Tue 13 Apr 14:30 CDT 2021, Thara Gopinath wrote:

> 
> 
> On 4/5/21 6:32 PM, Bjorn Andersson wrote:
> > On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> > 
> > > rf4309 is the specification that uses aes ccm algorithms with IPsec
> > > security packets. Add a submode to identify rfc4309 ccm(aes) algorithm
> > > in the crypto driver.
> > > 
> > > Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> > > ---
> > >   drivers/crypto/qce/common.h | 7 +++++--
> > >   1 file changed, 5 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/crypto/qce/common.h b/drivers/crypto/qce/common.h
> > > index 3bc244bcca2d..3ffe719b79e4 100644
> > > --- a/drivers/crypto/qce/common.h
> > > +++ b/drivers/crypto/qce/common.h
> > > @@ -51,9 +51,11 @@
> > >   #define QCE_MODE_CCM			BIT(12)
> > >   #define QCE_MODE_MASK			GENMASK(12, 8)
> > > +#define QCE_MODE_CCM_RFC4309		BIT(13)
> > > +
> > >   /* cipher encryption/decryption operations */
> > > -#define QCE_ENCRYPT			BIT(13)
> > > -#define QCE_DECRYPT			BIT(14)
> > > +#define QCE_ENCRYPT			BIT(14)
> > > +#define QCE_DECRYPT			BIT(15)
> > 
> > Can't we move these further up, so that next time we want to add
> > something it doesn't require that we also move the ENC/DEC bits?
> 
> Yes I will change it to BIT(30) and BIT(31)
> 
> > 
> > >   #define IS_DES(flags)			(flags & QCE_ALG_DES)
> > >   #define IS_3DES(flags)			(flags & QCE_ALG_3DES)
> > > @@ -73,6 +75,7 @@
> > >   #define IS_CTR(mode)			(mode & QCE_MODE_CTR)
> > >   #define IS_XTS(mode)			(mode & QCE_MODE_XTS)
> > >   #define IS_CCM(mode)			(mode & QCE_MODE_CCM)
> > > +#define IS_CCM_RFC4309(mode)		((mode) & QCE_MODE_CCM_RFC4309)
> > 
> > While leaving room for the typical macro issues, none of the other
> > macros wrap the argument in parenthesis. Please follow the style of the
> > driver, and perhaps follow up with a cleanup patch that just wraps them
> > all in parenthesis?
> 
> This does throw up a checkpatch warning if I don't wrap "mode" in
> parenthesis. How about I keep this for now and I will follow up with a clean
> up for rest of the macros later ?
> 

I don't have a problem with this approach.

Regards,
Bjorn

> > 
> > Regards,
> > Bjorn
> > 
> > >   #define IS_ENCRYPT(dir)			(dir & QCE_ENCRYPT)
> > >   #define IS_DECRYPT(dir)			(dir & QCE_DECRYPT)
> > > -- 
> > > 2.25.1
> > > 
> 
> -- 
> Warm Regards
> Thara

[PATCH 4/7] crypto: qce: Add support for AEAD algorithms

[Thara Gopinath]

Introduce support to enable following algorithms in Qualcomm Crypto Engine.

- authenc(hmac(sha1),cbc(des))
- authenc(hmac(sha1),cbc(des3_ede))
- authenc(hmac(sha256),cbc(des))
- authenc(hmac(sha256),cbc(des3_ede))
- authenc(hmac(sha256),cbc(aes))
- ccm(aes)
- rfc4309(ccm(aes))

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/Kconfig      |  15 +
 drivers/crypto/qce/Makefile |   1 +
 drivers/crypto/qce/aead.c   | 779 ++++++++++++++++++++++++++++++++++++
 drivers/crypto/qce/aead.h   |  53 +++
 drivers/crypto/qce/common.h |   2 +
 drivers/crypto/qce/core.c   |   4 +
 6 files changed, 854 insertions(+)
 create mode 100644 drivers/crypto/qce/aead.c
 create mode 100644 drivers/crypto/qce/aead.h

diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index e535f28a8028..8caf296acda4 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -645,6 +645,12 @@ config CRYPTO_DEV_QCE_SHA
 	select CRYPTO_SHA1
 	select CRYPTO_SHA256
 
+config CRYPTO_DEV_QCE_AEAD
+	bool
+	depends on CRYPTO_DEV_QCE
+	select CRYPTO_AUTHENC
+	select CRYPTO_LIB_DES
+
 choice
 	prompt "Algorithms enabled for QCE acceleration"
 	default CRYPTO_DEV_QCE_ENABLE_ALL
@@ -665,6 +671,7 @@ choice
 		bool "All supported algorithms"
 		select CRYPTO_DEV_QCE_SKCIPHER
 		select CRYPTO_DEV_QCE_SHA
+		select CRYPTO_DEV_QCE_AEAD
 		help
 		  Enable all supported algorithms:
 			- AES (CBC, CTR, ECB, XTS)
@@ -690,6 +697,14 @@ choice
 			- SHA1, HMAC-SHA1
 			- SHA256, HMAC-SHA256
 
+	config CRYPTO_DEV_QCE_ENABLE_AEAD
+		bool "AEAD algorithms only"
+		select CRYPTO_DEV_QCE_AEAD
+		help
+		  Enable AEAD algorithms only:
+			- authenc()
+			- ccm(aes)
+			- rfc4309(ccm(aes))
 endchoice
 
 config CRYPTO_DEV_QCE_SW_MAX_LEN
diff --git a/drivers/crypto/qce/Makefile b/drivers/crypto/qce/Makefile
index 14ade8a7d664..2cf8984e1b85 100644
--- a/drivers/crypto/qce/Makefile
+++ b/drivers/crypto/qce/Makefile
@@ -6,3 +6,4 @@ qcrypto-objs := core.o \
 
 qcrypto-$(CONFIG_CRYPTO_DEV_QCE_SHA) += sha.o
 qcrypto-$(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) += skcipher.o
+qcrypto-$(CONFIG_CRYPTO_DEV_QCE_AEAD) += aead.o
diff --git a/drivers/crypto/qce/aead.c b/drivers/crypto/qce/aead.c
new file mode 100644
index 000000000000..b594c4bb2640
--- /dev/null
+++ b/drivers/crypto/qce/aead.c
@@ -0,0 +1,779 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/*
+ * Copyright (C) 2021, Linaro Limited. All rights reserved.
+ */
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <crypto/gcm.h>
+#include <crypto/authenc.h>
+#include <crypto/internal/aead.h>
+#include <crypto/internal/des.h>
+#include <crypto/sha1.h>
+#include <crypto/sha2.h>
+#include <crypto/scatterwalk.h>
+#include "aead.h"
+
+#define CCM_NONCE_ADATA_SHIFT		6
+#define CCM_NONCE_AUTHSIZE_SHIFT	3
+#define MAX_CCM_ADATA_HEADER_LEN        6
+
+static LIST_HEAD(aead_algs);
+
+static void qce_aead_done(void *data)
+{
+	struct crypto_async_request *async_req = data;
+	struct aead_request *req = aead_request_cast(async_req);
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
+	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
+	struct qce_device *qce = tmpl->qce;
+	struct qce_result_dump *result_buf = qce->dma.result_buf;
+	enum dma_data_direction dir_src, dir_dst;
+	bool diff_dst;
+	int error;
+	u32 status;
+	unsigned int totallen;
+	unsigned char tag[SHA256_DIGEST_SIZE] = {0};
+	int ret = 0;
+
+	diff_dst = (req->src != req->dst) ? true : false;
+	dir_src = diff_dst ? DMA_TO_DEVICE : DMA_BIDIRECTIONAL;
+	dir_dst = diff_dst ? DMA_FROM_DEVICE : DMA_BIDIRECTIONAL;
+
+	error = qce_dma_terminate_all(&qce->dma);
+	if (error)
+		dev_dbg(qce->dev, "aead dma termination error (%d)\n",
+			error);
+	if (diff_dst)
+		dma_unmap_sg(qce->dev, rctx->src_sg, rctx->src_nents, dir_src);
+
+	dma_unmap_sg(qce->dev, rctx->dst_sg, rctx->dst_nents, dir_dst);
+
+	if (IS_CCM(rctx->flags)) {
+		if (req->assoclen) {
+			sg_free_table(&rctx->src_tbl);
+			if (diff_dst)
+				sg_free_table(&rctx->dst_tbl);
+		} else {
+			if (!(IS_DECRYPT(rctx->flags) && !diff_dst))
+				sg_free_table(&rctx->dst_tbl);
+		}
+	} else {
+		sg_free_table(&rctx->dst_tbl);
+	}
+
+	error = qce_check_status(qce, &status);
+	if (error < 0 && (error != -EBADMSG))
+		dev_err(qce->dev, "aead operation error (%x)\n", status);
+
+	if (IS_ENCRYPT(rctx->flags)) {
+		totallen = req->cryptlen + req->assoclen;
+		if (IS_CCM(rctx->flags))
+			scatterwalk_map_and_copy(rctx->ccmresult_buf, req->dst,
+						 totallen, ctx->authsize, 1);
+		else
+			scatterwalk_map_and_copy(result_buf->auth_iv, req->dst,
+						 totallen, ctx->authsize, 1);
+
+	} else if (!IS_CCM(rctx->flags)) {
+		totallen = req->cryptlen + req->assoclen - ctx->authsize;
+		scatterwalk_map_and_copy(tag, req->src, totallen, ctx->authsize, 0);
+		ret = memcmp(result_buf->auth_iv, tag, ctx->authsize);
+		if (ret) {
+			pr_err("Bad message error\n");
+			 error = -EBADMSG;
+		}
+	}
+
+	qce->async_req_done(qce, error);
+}
+
+static struct scatterlist *
+qce_aead_prepare_result_buf(struct sg_table *tbl, struct aead_request *req)
+{
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
+	struct qce_device *qce = tmpl->qce;
+
+	sg_init_one(&rctx->result_sg, qce->dma.result_buf, QCE_RESULT_BUF_SZ);
+	return qce_sgtable_add(tbl, &rctx->result_sg, QCE_RESULT_BUF_SZ);
+}
+
+static struct scatterlist *
+qce_aead_prepare_ccm_result_buf(struct sg_table *tbl, struct aead_request *req)
+{
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+
+	sg_init_one(&rctx->result_sg, rctx->ccmresult_buf, QCE_BAM_BURST_SIZE);
+	return qce_sgtable_add(tbl, &rctx->result_sg, QCE_BAM_BURST_SIZE);
+}
+
+static struct scatterlist *
+qce_aead_prepare_dst_buf(struct aead_request *req)
+{
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
+	struct qce_device *qce = tmpl->qce;
+	struct scatterlist *sg, *msg_sg, __sg[2];
+	gfp_t gfp;
+	unsigned int assoclen = req->assoclen;
+	unsigned int totallen;
+	int ret;
+
+	totallen = rctx->cryptlen + assoclen;
+	rctx->dst_nents = sg_nents_for_len(req->dst, totallen);
+	if (rctx->dst_nents < 0) {
+		dev_err(qce->dev, "Invalid numbers of dst SG.\n");
+		return ERR_PTR(-EINVAL);
+	}
+	if (IS_CCM(rctx->flags))
+		rctx->dst_nents += 2;
+	else
+		rctx->dst_nents += 1;
+
+	gfp = (req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP) ?
+						GFP_KERNEL : GFP_ATOMIC;
+	ret = sg_alloc_table(&rctx->dst_tbl, rctx->dst_nents, gfp);
+	if (ret)
+		return ERR_PTR(ret);
+
+	if (IS_CCM(rctx->flags) && assoclen) {
+		/* Get the dst buffer */
+		msg_sg = scatterwalk_ffwd(__sg, req->dst, assoclen);
+
+		sg = qce_sgtable_add(&rctx->dst_tbl, &rctx->adata_sg,
+				     rctx->assoclen);
+		if (IS_ERR(sg)) {
+			ret = PTR_ERR(sg);
+			goto dst_tbl_free;
+		}
+		/* dst buffer */
+		sg = qce_sgtable_add(&rctx->dst_tbl, msg_sg, rctx->cryptlen);
+		if (IS_ERR(sg)) {
+			ret = PTR_ERR(sg);
+			goto dst_tbl_free;
+		}
+		totallen = rctx->cryptlen + rctx->assoclen;
+	} else {
+		if (totallen) {
+			sg = qce_sgtable_add(&rctx->dst_tbl, req->dst, totallen);
+			if (IS_ERR(sg))
+				goto dst_tbl_free;
+		}
+	}
+	if (IS_CCM(rctx->flags))
+		sg = qce_aead_prepare_ccm_result_buf(&rctx->dst_tbl, req);
+	else
+		sg = qce_aead_prepare_result_buf(&rctx->dst_tbl, req);
+
+	if (IS_ERR(sg))
+		goto dst_tbl_free;
+
+	sg_mark_end(sg);
+	rctx->dst_sg = rctx->dst_tbl.sgl;
+	rctx->dst_nents = sg_nents_for_len(rctx->dst_sg, totallen) + 1;
+
+	return sg;
+
+dst_tbl_free:
+	sg_free_table(&rctx->dst_tbl);
+	return sg;
+}
+
+static int
+qce_aead_ccm_prepare_buf_assoclen(struct aead_request *req)
+{
+	struct scatterlist *sg, *msg_sg, __sg[2];
+	struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+	unsigned int assoclen = rctx->assoclen;
+	unsigned int adata_header_len, cryptlen, totallen;
+	gfp_t gfp;
+	bool diff_dst;
+	int ret;
+
+	if (IS_DECRYPT(rctx->flags))
+		cryptlen = rctx->cryptlen + ctx->authsize;
+	else
+		cryptlen = rctx->cryptlen;
+	totallen = cryptlen + req->assoclen;
+
+	/* Get the msg */
+	msg_sg = scatterwalk_ffwd(__sg, req->src, req->assoclen);
+
+	rctx->adata = kzalloc((ALIGN(assoclen, 16) + MAX_CCM_ADATA_HEADER_LEN) *
+			       sizeof(unsigned char), GFP_ATOMIC);
+	if (!rctx->adata)
+		return -ENOMEM;
+
+	/*
+	 * Format associated data (RFC3610 and NIST 800-38C)
+	 * Even though specification allows for AAD to be up to 2^64 - 1 bytes,
+	 * the assoclen field in aead_request is unsigned int and thus limits
+	 * the AAD to be up to 2^32 - 1 bytes. So we handle only two scenarios
+	 * while forming the header for AAD.
+	 */
+	if (assoclen < 0xff00) {
+		adata_header_len = 2;
+		*(__be16 *)rctx->adata = cpu_to_be16(assoclen);
+	} else {
+		adata_header_len = 6;
+		*(__be16 *)rctx->adata = cpu_to_be16(0xfffe);
+		*(__be32 *)(rctx->adata + 2) = cpu_to_be32(assoclen);
+	}
+
+	/* Copy the associated data */
+	if (sg_copy_to_buffer(req->src, sg_nents_for_len(req->src, assoclen),
+			      rctx->adata + adata_header_len,
+			      assoclen) != assoclen)
+		return -EINVAL;
+
+	/* Pad associated data to block size */
+	rctx->assoclen = ALIGN(assoclen + adata_header_len, 16);
+
+	diff_dst = (req->src != req->dst) ? true : false;
+
+	if (diff_dst)
+		rctx->src_nents = sg_nents_for_len(req->src, totallen) + 1;
+	else
+		rctx->src_nents = sg_nents_for_len(req->src, totallen) + 2;
+
+	gfp = (req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP) ? GFP_KERNEL : GFP_ATOMIC;
+	ret = sg_alloc_table(&rctx->src_tbl, rctx->src_nents, gfp);
+	if (ret)
+		return ret;
+
+	/* Associated Data */
+	sg_init_one(&rctx->adata_sg, rctx->adata, rctx->assoclen);
+	sg = qce_sgtable_add(&rctx->src_tbl, &rctx->adata_sg,
+			     rctx->assoclen);
+	if (IS_ERR(sg)) {
+		ret = PTR_ERR(sg);
+		goto err_free;
+	}
+	/* src msg */
+	sg = qce_sgtable_add(&rctx->src_tbl, msg_sg, cryptlen);
+	if (IS_ERR(sg)) {
+		ret = PTR_ERR(sg);
+		goto err_free;
+	}
+	if (!diff_dst) {
+		/*
+		 * When src and dst buffers are same, there is already space
+		 * in the buffer for padded 0's which is output in lieu of
+		 * the MAC that is input
+		 */
+		if (!IS_DECRYPT(rctx->flags)) {
+			sg = qce_aead_prepare_ccm_result_buf(&rctx->src_tbl, req);
+			if (IS_ERR(sg)) {
+				ret = PTR_ERR(sg);
+				goto err_free;
+			}
+		}
+	}
+	sg_mark_end(sg);
+	rctx->src_sg = rctx->src_tbl.sgl;
+	totallen = cryptlen + rctx->assoclen;
+	rctx->src_nents = sg_nents_for_len(rctx->src_sg, totallen);
+
+	if (diff_dst) {
+		sg = qce_aead_prepare_dst_buf(req);
+		if (IS_ERR(sg))
+			goto err_free;
+	} else {
+		if (IS_ENCRYPT(rctx->flags))
+			rctx->dst_nents = rctx->src_nents + 1;
+		else
+			rctx->dst_nents = rctx->src_nents;
+		rctx->dst_sg = rctx->src_sg;
+	}
+
+	return 0;
+err_free:
+	sg_free_table(&rctx->src_tbl);
+	return ret;
+}
+
+static int qce_aead_prepare_buf(struct aead_request *req)
+{
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
+	struct qce_device *qce = tmpl->qce;
+	struct scatterlist *sg;
+	bool diff_dst = (req->src != req->dst) ? true : false;
+	unsigned int totallen;
+
+	totallen = rctx->cryptlen + rctx->assoclen;
+
+	sg = qce_aead_prepare_dst_buf(req);
+	if (IS_ERR(sg))
+		return PTR_ERR(sg);
+	if (diff_dst) {
+		rctx->src_nents = sg_nents_for_len(req->src, totallen);
+		if (rctx->src_nents < 0) {
+			dev_err(qce->dev, "Invalid numbers of src SG.\n");
+			return -EINVAL;
+		}
+		rctx->src_sg = req->src;
+	} else {
+		rctx->src_nents = rctx->dst_nents - 1;
+		rctx->src_sg = rctx->dst_sg;
+	}
+	return 0;
+}
+
+static int qce_aead_ccm_prepare_buf(struct aead_request *req)
+{
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+	struct scatterlist *sg;
+	bool diff_dst = (req->src != req->dst) ? true : false;
+	unsigned int cryptlen;
+
+	if (rctx->assoclen)
+		return qce_aead_ccm_prepare_buf_assoclen(req);
+
+	if (IS_ENCRYPT(rctx->flags))
+		return qce_aead_prepare_buf(req);
+
+	cryptlen = rctx->cryptlen + ctx->authsize;
+	if (diff_dst) {
+		rctx->src_nents = sg_nents_for_len(req->src, cryptlen);
+		rctx->src_sg = req->src;
+		sg = qce_aead_prepare_dst_buf(req);
+		if (IS_ERR(sg))
+			return PTR_ERR(sg);
+	} else {
+		rctx->src_nents = sg_nents_for_len(req->src, cryptlen);
+		rctx->src_sg = req->src;
+		rctx->dst_nents = rctx->src_nents;
+		rctx->dst_sg = rctx->src_sg;
+	}
+
+	return 0;
+}
+
+static int qce_aead_create_ccm_nonce(struct qce_aead_reqctx *rctx, struct qce_aead_ctx *ctx)
+{
+	unsigned int msglen_size;
+	u8 msg_len[4];
+	int i;
+
+	if (!rctx || !rctx->iv)
+		return -EINVAL;
+
+	msglen_size = rctx->iv[0] + 1;
+
+	if (msglen_size > 4)
+		msglen_size = 4;
+
+	memcpy(&msg_len[0], &rctx->cryptlen, 4);
+
+	memcpy(&rctx->ccm_nonce[0], rctx->iv, rctx->ivsize);
+	if (rctx->assoclen)
+		rctx->ccm_nonce[0] |= 1 << CCM_NONCE_ADATA_SHIFT;
+	rctx->ccm_nonce[0] |= ((ctx->authsize - 2) / 2) <<
+				CCM_NONCE_AUTHSIZE_SHIFT;
+	for (i = 0; i < msglen_size; i++)
+		rctx->ccm_nonce[QCE_MAX_NONCE - i - 1] = msg_len[i];
+
+	return 0;
+}
+
+static int
+qce_aead_async_req_handle(struct crypto_async_request *async_req)
+{
+	struct aead_request *req = aead_request_cast(async_req);
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
+	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
+	struct qce_device *qce = tmpl->qce;
+	enum dma_data_direction dir_src, dir_dst;
+	unsigned int totallen;
+	bool diff_dst;
+	int ret;
+
+	if (IS_CCM_RFC4309(rctx->flags)) {
+		memset(rctx->ccm_rfc4309_iv, 0, QCE_MAX_IV_SIZE);
+		rctx->ccm_rfc4309_iv[0] = 3;
+		memcpy(&rctx->ccm_rfc4309_iv[1], ctx->ccm4309_salt, QCE_CCM4309_SALT_SIZE);
+		memcpy(&rctx->ccm_rfc4309_iv[4], req->iv, 8);
+		rctx->iv = rctx->ccm_rfc4309_iv;
+		rctx->ivsize = AES_BLOCK_SIZE;
+	} else {
+		rctx->iv = req->iv;
+		rctx->ivsize = crypto_aead_ivsize(tfm);
+	}
+	if (IS_CCM_RFC4309(rctx->flags))
+		rctx->assoclen = req->assoclen - 8;
+	else
+		rctx->assoclen = req->assoclen;
+
+	totallen = rctx->cryptlen + rctx->assoclen;
+
+	diff_dst = (req->src != req->dst) ? true : false;
+	dir_src = diff_dst ? DMA_TO_DEVICE : DMA_BIDIRECTIONAL;
+	dir_dst = diff_dst ? DMA_FROM_DEVICE : DMA_BIDIRECTIONAL;
+
+	if (IS_CCM(rctx->flags)) {
+		ret = qce_aead_create_ccm_nonce(rctx, ctx);
+		if (ret)
+			return ret;
+	}
+	if (IS_CCM(rctx->flags))
+		ret = qce_aead_ccm_prepare_buf(req);
+	else
+		ret = qce_aead_prepare_buf(req);
+
+	if (ret)
+		return ret;
+	ret = dma_map_sg(qce->dev, rctx->dst_sg, rctx->dst_nents, dir_dst);
+	if (ret < 0)
+		goto error_free;
+
+	if (diff_dst) {
+		ret = dma_map_sg(qce->dev, rctx->src_sg, rctx->src_nents, dir_src);
+		if (ret < 0)
+			goto error_unmap_dst;
+	}
+
+	ret = qce_dma_prep_sgs(&qce->dma, rctx->src_sg, rctx->src_nents,
+			       rctx->dst_sg, rctx->dst_nents,
+			       qce_aead_done, async_req);
+	if (ret)
+		goto error_unmap_src;
+
+	qce_dma_issue_pending(&qce->dma);
+
+	ret = qce_start(async_req, tmpl->crypto_alg_type);
+	if (ret)
+		goto error_terminate;
+
+	return 0;
+
+error_terminate:
+	qce_dma_terminate_all(&qce->dma);
+error_unmap_src:
+	if (diff_dst)
+		dma_unmap_sg(qce->dev, req->src, rctx->src_nents, dir_src);
+error_unmap_dst:
+	dma_unmap_sg(qce->dev, rctx->dst_sg, rctx->dst_nents, dir_dst);
+error_free:
+	if (IS_CCM(rctx->flags) && rctx->assoclen) {
+		sg_free_table(&rctx->src_tbl);
+		if (diff_dst)
+			sg_free_table(&rctx->dst_tbl);
+	} else {
+		sg_free_table(&rctx->dst_tbl);
+	}
+	return ret;
+}
+
+static int qce_aead_crypt(struct aead_request *req, int encrypt)
+{
+	struct crypto_aead *tfm = crypto_aead_reqtfm(req);
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+	struct qce_alg_template *tmpl = to_aead_tmpl(tfm);
+	unsigned int blocksize = crypto_aead_blocksize(tfm);
+
+	rctx->flags  = tmpl->alg_flags;
+	rctx->flags |= encrypt ? QCE_ENCRYPT : QCE_DECRYPT;
+
+	if (encrypt)
+		rctx->cryptlen = req->cryptlen;
+	else
+		rctx->cryptlen = req->cryptlen - ctx->authsize;
+
+	/* CE does not handle 0 length messages */
+	if (!rctx->cryptlen) {
+		if (!(IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags)))
+			return -EINVAL;
+	}
+
+	/*
+	 * CBC algorithms require message lengths to be
+	 * multiples of block size.
+	 */
+	if (IS_CBC(rctx->flags) && !IS_ALIGNED(rctx->cryptlen, blocksize))
+		return -EINVAL;
+
+	/* RFC4309 supported AAD size 16 bytes/20 bytes */
+	if (IS_CCM_RFC4309(rctx->flags))
+		if (crypto_ipsec_check_assoclen(req->assoclen))
+			return -EINVAL;
+
+	return tmpl->qce->async_req_enqueue(tmpl->qce, &req->base);
+}
+
+static int qce_aead_encrypt(struct aead_request *req)
+{
+	return qce_aead_crypt(req, 1);
+}
+
+static int qce_aead_decrypt(struct aead_request *req)
+{
+	return qce_aead_crypt(req, 0);
+}
+
+static int qce_aead_ccm_setkey(struct crypto_aead *tfm, const u8 *key,
+			       unsigned int keylen)
+{
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+	unsigned long flags = to_aead_tmpl(tfm)->alg_flags;
+
+	if (IS_CCM_RFC4309(flags)) {
+		if (keylen < QCE_CCM4309_SALT_SIZE)
+			return -EINVAL;
+		keylen -= QCE_CCM4309_SALT_SIZE;
+		memcpy(ctx->ccm4309_salt, key + keylen, QCE_CCM4309_SALT_SIZE);
+	}
+
+	if (keylen != AES_KEYSIZE_128 && keylen != AES_KEYSIZE_256)
+		return -EINVAL;
+
+	ctx->enc_keylen = keylen;
+	ctx->auth_keylen = keylen;
+
+	memcpy(ctx->enc_key, key, keylen);
+	memcpy(ctx->auth_key, key, keylen);
+
+	return 0;
+}
+
+static int qce_aead_setkey(struct crypto_aead *tfm, const u8 *key, unsigned int keylen)
+{
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+	struct crypto_authenc_keys authenc_keys;
+	unsigned long flags = to_aead_tmpl(tfm)->alg_flags;
+	u32 _key[6];
+	int err;
+
+	err = crypto_authenc_extractkeys(&authenc_keys, key, keylen);
+	if (err)
+		return err;
+
+	if (authenc_keys.enckeylen > QCE_MAX_KEY_SIZE ||
+	    authenc_keys.authkeylen > QCE_MAX_KEY_SIZE)
+		return -EINVAL;
+
+	if (IS_DES(flags)) {
+		err = verify_aead_des_key(tfm, authenc_keys.enckey, authenc_keys.enckeylen);
+		if (err)
+			return err;
+	} else if (IS_3DES(flags)) {
+		err = verify_aead_des3_key(tfm, authenc_keys.enckey, authenc_keys.enckeylen);
+		if (err)
+			return err;
+		/*
+		 * The crypto engine does not support any two keys
+		 * being the same for triple des algorithms. The
+		 * verify_skcipher_des3_key does not check for all the
+		 * below conditions. Return -EINVAL in case any two keys
+		 * are the same. Revisit to see if a fallback cipher
+		 * is needed to handle this condition.
+		 */
+		memcpy(_key, authenc_keys.enckey, DES3_EDE_KEY_SIZE);
+		if (!((_key[0] ^ _key[2]) | (_key[1] ^ _key[3])) ||
+		    !((_key[2] ^ _key[4]) | (_key[3] ^ _key[5])) ||
+		    !((_key[0] ^ _key[4]) | (_key[1] ^ _key[5])))
+			return -EINVAL;
+	} else if (IS_AES(flags)) {
+		/* No random key sizes */
+		if (authenc_keys.enckeylen != AES_KEYSIZE_128 &&
+		    authenc_keys.enckeylen != AES_KEYSIZE_256)
+			return -EINVAL;
+	}
+
+	ctx->enc_keylen = authenc_keys.enckeylen;
+	ctx->auth_keylen = authenc_keys.authkeylen;
+
+	memcpy(ctx->enc_key, authenc_keys.enckey, authenc_keys.enckeylen);
+
+	memset(ctx->auth_key, 0, sizeof(ctx->auth_key));
+	memcpy(ctx->auth_key, authenc_keys.authkey, authenc_keys.authkeylen);
+
+	return 0;
+}
+
+static int qce_aead_setauthsize(struct crypto_aead *tfm, unsigned int authsize)
+{
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+	unsigned long flags = to_aead_tmpl(tfm)->alg_flags;
+
+	if (IS_CCM(flags)) {
+		if (authsize < 4 || authsize > 16 || authsize % 2)
+			return -EINVAL;
+		if (IS_CCM_RFC4309(flags) && (authsize < 8 || authsize % 4))
+			return -EINVAL;
+	}
+	ctx->authsize = authsize;
+	return 0;
+}
+
+static int qce_aead_init(struct crypto_aead *tfm)
+{
+	crypto_aead_set_reqsize(tfm, sizeof(struct qce_aead_reqctx));
+	return 0;
+}
+
+struct qce_aead_def {
+	unsigned long flags;
+	const char *name;
+	const char *drv_name;
+	unsigned int blocksize;
+	unsigned int chunksize;
+	unsigned int ivsize;
+	unsigned int maxauthsize;
+};
+
+static const struct qce_aead_def aead_def[] = {
+	{
+		.flags          = QCE_ALG_DES | QCE_MODE_CBC | QCE_HASH_SHA1_HMAC,
+		.name           = "authenc(hmac(sha1),cbc(des))",
+		.drv_name       = "authenc-hmac-sha1-cbc-des-qce",
+		.blocksize      = DES_BLOCK_SIZE,
+		.ivsize         = DES_BLOCK_SIZE,
+		.maxauthsize	= SHA1_DIGEST_SIZE,
+	},
+	{
+		.flags          = QCE_ALG_3DES | QCE_MODE_CBC | QCE_HASH_SHA1_HMAC,
+		.name           = "authenc(hmac(sha1),cbc(des3_ede))",
+		.drv_name       = "authenc-hmac-sha1-cbc-3des-qce",
+		.blocksize      = DES3_EDE_BLOCK_SIZE,
+		.ivsize         = DES3_EDE_BLOCK_SIZE,
+		.maxauthsize	= SHA1_DIGEST_SIZE,
+	},
+	{
+		.flags          = QCE_ALG_DES | QCE_MODE_CBC | QCE_HASH_SHA256_HMAC,
+		.name           = "authenc(hmac(sha256),cbc(des))",
+		.drv_name       = "authenc-hmac-sha256-cbc-des-qce",
+		.blocksize      = DES_BLOCK_SIZE,
+		.ivsize         = DES_BLOCK_SIZE,
+		.maxauthsize	= SHA256_DIGEST_SIZE,
+	},
+	{
+		.flags          = QCE_ALG_3DES | QCE_MODE_CBC | QCE_HASH_SHA256_HMAC,
+		.name           = "authenc(hmac(sha256),cbc(des3_ede))",
+		.drv_name       = "authenc-hmac-sha256-cbc-3des-qce",
+		.blocksize      = DES3_EDE_BLOCK_SIZE,
+		.ivsize         = DES3_EDE_BLOCK_SIZE,
+		.maxauthsize	= SHA256_DIGEST_SIZE,
+	},
+	{
+		.flags          =  QCE_ALG_AES | QCE_MODE_CBC | QCE_HASH_SHA256_HMAC,
+		.name           = "authenc(hmac(sha256),cbc(aes))",
+		.drv_name       = "authenc-hmac-sha256-cbc-aes-qce",
+		.blocksize      = AES_BLOCK_SIZE,
+		.ivsize         = AES_BLOCK_SIZE,
+		.maxauthsize	= SHA256_DIGEST_SIZE,
+	},
+	{
+		.flags          =  QCE_ALG_AES | QCE_MODE_CCM,
+		.name           = "ccm(aes)",
+		.drv_name       = "ccm-aes-qce",
+		.blocksize	= 1,
+		.ivsize         = AES_BLOCK_SIZE,
+		.maxauthsize	= AES_BLOCK_SIZE,
+	},
+	{
+		.flags          =  QCE_ALG_AES | QCE_MODE_CCM | QCE_MODE_CCM_RFC4309,
+		.name           = "rfc4309(ccm(aes))",
+		.drv_name       = "rfc4309-ccm-aes-qce",
+		.blocksize	= 1,
+		.ivsize         = 8,
+		.maxauthsize	= AES_BLOCK_SIZE,
+	},
+};
+
+static int qce_aead_register_one(const struct qce_aead_def *def, struct qce_device *qce)
+{
+	struct qce_alg_template *tmpl;
+	struct aead_alg *alg;
+	int ret;
+
+	tmpl = kzalloc(sizeof(*tmpl), GFP_KERNEL);
+	if (!tmpl)
+		return -ENOMEM;
+
+	alg = &tmpl->alg.aead;
+
+	snprintf(alg->base.cra_name, CRYPTO_MAX_ALG_NAME, "%s", def->name);
+	snprintf(alg->base.cra_driver_name, CRYPTO_MAX_ALG_NAME, "%s",
+		 def->drv_name);
+
+	alg->base.cra_blocksize		= def->blocksize;
+	alg->chunksize			= def->chunksize;
+	alg->ivsize			= def->ivsize;
+	alg->maxauthsize		= def->maxauthsize;
+	if (IS_CCM(def->flags))
+		alg->setkey		= qce_aead_ccm_setkey;
+	else
+		alg->setkey		= qce_aead_setkey;
+	alg->setauthsize		= qce_aead_setauthsize;
+	alg->encrypt			= qce_aead_encrypt;
+	alg->decrypt			= qce_aead_decrypt;
+	alg->init			= qce_aead_init;
+
+	alg->base.cra_priority		= 300;
+	alg->base.cra_flags		= CRYPTO_ALG_ASYNC |
+					  CRYPTO_ALG_ALLOCATES_MEMORY |
+					  CRYPTO_ALG_KERN_DRIVER_ONLY;
+	alg->base.cra_ctxsize		= sizeof(struct qce_aead_ctx);
+	alg->base.cra_alignmask		= 0;
+	alg->base.cra_module		= THIS_MODULE;
+
+	INIT_LIST_HEAD(&tmpl->entry);
+	tmpl->crypto_alg_type = CRYPTO_ALG_TYPE_AEAD;
+	tmpl->alg_flags = def->flags;
+	tmpl->qce = qce;
+
+	ret = crypto_register_aead(alg);
+	if (ret) {
+		kfree(tmpl);
+		dev_err(qce->dev, "%s registration failed\n", alg->base.cra_name);
+		return ret;
+	}
+
+	list_add_tail(&tmpl->entry, &aead_algs);
+	dev_dbg(qce->dev, "%s is registered\n", alg->base.cra_name);
+	return 0;
+}
+
+static void qce_aead_unregister(struct qce_device *qce)
+{
+	struct qce_alg_template *tmpl, *n;
+
+	list_for_each_entry_safe(tmpl, n, &aead_algs, entry) {
+		crypto_unregister_aead(&tmpl->alg.aead);
+		list_del(&tmpl->entry);
+		kfree(tmpl);
+	}
+}
+
+static int qce_aead_register(struct qce_device *qce)
+{
+	int ret, i;
+
+	for (i = 0; i < ARRAY_SIZE(aead_def); i++) {
+		ret = qce_aead_register_one(&aead_def[i], qce);
+		if (ret)
+			goto err;
+	}
+
+	return 0;
+err:
+	qce_aead_unregister(qce);
+	return ret;
+}
+
+const struct qce_algo_ops aead_ops = {
+	.type = CRYPTO_ALG_TYPE_AEAD,
+	.register_algs = qce_aead_register,
+	.unregister_algs = qce_aead_unregister,
+	.async_req_handle = qce_aead_async_req_handle,
+};
diff --git a/drivers/crypto/qce/aead.h b/drivers/crypto/qce/aead.h
new file mode 100644
index 000000000000..3d1f2039930b
--- /dev/null
+++ b/drivers/crypto/qce/aead.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2021, Linaro Limited. All rights reserved.
+ */
+
+#ifndef _AEAD_H_
+#define _AEAD_H_
+
+#include "common.h"
+#include "core.h"
+
+#define QCE_MAX_KEY_SIZE		64
+#define QCE_CCM4309_SALT_SIZE		3
+
+struct qce_aead_ctx {
+	u8 enc_key[QCE_MAX_KEY_SIZE];
+	u8 auth_key[QCE_MAX_KEY_SIZE];
+	u8 ccm4309_salt[QCE_CCM4309_SALT_SIZE];
+	unsigned int enc_keylen;
+	unsigned int auth_keylen;
+	unsigned int authsize;
+};
+
+struct qce_aead_reqctx {
+	unsigned long flags;
+	u8 *iv;
+	unsigned int ivsize;
+	int src_nents;
+	int dst_nents;
+	struct scatterlist result_sg;
+	struct scatterlist adata_sg;
+	struct sg_table dst_tbl;
+	struct sg_table src_tbl;
+	struct scatterlist *dst_sg;
+	struct scatterlist *src_sg;
+	unsigned int cryptlen;
+	unsigned int assoclen;
+	unsigned char *adata;
+	u8 ccm_nonce[QCE_MAX_NONCE];
+	u8 ccmresult_buf[QCE_BAM_BURST_SIZE];
+	u8 ccm_rfc4309_iv[QCE_MAX_IV_SIZE];
+};
+
+static inline struct qce_alg_template *to_aead_tmpl(struct crypto_aead *tfm)
+{
+	struct aead_alg *alg = crypto_aead_alg(tfm);
+
+	return container_of(alg, struct qce_alg_template, alg.aead);
+}
+
+extern const struct qce_algo_ops aead_ops;
+
+#endif /* _AEAD_H_ */
diff --git a/drivers/crypto/qce/common.h b/drivers/crypto/qce/common.h
index 3ffe719b79e4..ba7ff6b68825 100644
--- a/drivers/crypto/qce/common.h
+++ b/drivers/crypto/qce/common.h
@@ -11,6 +11,7 @@
 #include <crypto/aes.h>
 #include <crypto/hash.h>
 #include <crypto/internal/skcipher.h>
+#include <crypto/internal/aead.h>
 
 /* xts du size */
 #define QCE_SECTOR_SIZE			512
@@ -88,6 +89,7 @@ struct qce_alg_template {
 	union {
 		struct skcipher_alg skcipher;
 		struct ahash_alg ahash;
+		struct aead_alg aead;
 	} alg;
 	struct qce_device *qce;
 	const u8 *hash_zero;
diff --git a/drivers/crypto/qce/core.c b/drivers/crypto/qce/core.c
index 80b75085c265..d3780be44a76 100644
--- a/drivers/crypto/qce/core.c
+++ b/drivers/crypto/qce/core.c
@@ -17,6 +17,7 @@
 #include "core.h"
 #include "cipher.h"
 #include "sha.h"
+#include "aead.h"
 
 #define QCE_MAJOR_VERSION5	0x05
 #define QCE_QUEUE_LENGTH	1
@@ -28,6 +29,9 @@ static const struct qce_algo_ops *qce_ops[] = {
 #ifdef CONFIG_CRYPTO_DEV_QCE_SHA
 	&ahash_ops,
 #endif
+#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
+	&aead_ops,
+#endif
 };
 
 static void qce_unregister_algs(struct qce_device *qce)
-- 
2.25.1

Re: [PATCH 4/7] crypto: qce: Add support for AEAD algorithms

[Herbert Xu]

On Thu, Feb 25, 2021 at 01:27:13PM -0500, Thara Gopinath wrote:
>
> +static int
> +qce_aead_async_req_handle(struct crypto_async_request *async_req)
> +{
> +	struct aead_request *req = aead_request_cast(async_req);
> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
> +	struct crypto_aead *tfm = crypto_aead_reqtfm(req);
> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
> +	struct qce_device *qce = tmpl->qce;
> +	enum dma_data_direction dir_src, dir_dst;
> +	unsigned int totallen;
> +	bool diff_dst;
> +	int ret;
> +
> +	if (IS_CCM_RFC4309(rctx->flags)) {
> +		memset(rctx->ccm_rfc4309_iv, 0, QCE_MAX_IV_SIZE);
> +		rctx->ccm_rfc4309_iv[0] = 3;
> +		memcpy(&rctx->ccm_rfc4309_iv[1], ctx->ccm4309_salt, QCE_CCM4309_SALT_SIZE);
> +		memcpy(&rctx->ccm_rfc4309_iv[4], req->iv, 8);
> +		rctx->iv = rctx->ccm_rfc4309_iv;
> +		rctx->ivsize = AES_BLOCK_SIZE;
> +	} else {
> +		rctx->iv = req->iv;
> +		rctx->ivsize = crypto_aead_ivsize(tfm);
> +	}
> +	if (IS_CCM_RFC4309(rctx->flags))
> +		rctx->assoclen = req->assoclen - 8;
> +	else
> +		rctx->assoclen = req->assoclen;
> +
> +	totallen = rctx->cryptlen + rctx->assoclen;

This triggers a warning on totallen not being used.  Please fix.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH 4/7] crypto: qce: Add support for AEAD algorithms

[Thara Gopinath]

On 3/12/21 8:01 AM, Herbert Xu wrote:
> On Thu, Feb 25, 2021 at 01:27:13PM -0500, Thara Gopinath wrote:
>>
>> +static int
>> +qce_aead_async_req_handle(struct crypto_async_request *async_req)
>> +{
>> +	struct aead_request *req = aead_request_cast(async_req);
>> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
>> +	struct crypto_aead *tfm = crypto_aead_reqtfm(req);
>> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
>> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
>> +	struct qce_device *qce = tmpl->qce;
>> +	enum dma_data_direction dir_src, dir_dst;
>> +	unsigned int totallen;
>> +	bool diff_dst;
>> +	int ret;
>> +
>> +	if (IS_CCM_RFC4309(rctx->flags)) {
>> +		memset(rctx->ccm_rfc4309_iv, 0, QCE_MAX_IV_SIZE);
>> +		rctx->ccm_rfc4309_iv[0] = 3;
>> +		memcpy(&rctx->ccm_rfc4309_iv[1], ctx->ccm4309_salt, QCE_CCM4309_SALT_SIZE);
>> +		memcpy(&rctx->ccm_rfc4309_iv[4], req->iv, 8);
>> +		rctx->iv = rctx->ccm_rfc4309_iv;
>> +		rctx->ivsize = AES_BLOCK_SIZE;
>> +	} else {
>> +		rctx->iv = req->iv;
>> +		rctx->ivsize = crypto_aead_ivsize(tfm);
>> +	}
>> +	if (IS_CCM_RFC4309(rctx->flags))
>> +		rctx->assoclen = req->assoclen - 8;
>> +	else
>> +		rctx->assoclen = req->assoclen;
>> +
>> +	totallen = rctx->cryptlen + rctx->assoclen;
> 
> This triggers a warning on totallen not being used.  Please fix.

hmm.. this is strange. I could swear that I checked for warnings before 
sending this out. But I will fix this. I will wait for a couple of more 
days for any other comments and then spin a v2.

> 
> Thanks,
> 

-- 
Warm Regards
Thara

[PATCH 5/7] crypto: qce: common: Clean up qce_auth_cfg

[Thara Gopinath]

Remove various redundant checks in qce_auth_cfg. Also allow qce_auth_cfg
to take auth_size as a parameter which is a required setting for ccm(aes)
algorithms

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/qce/common.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
index 2485aa371d83..05a71c5ecf61 100644
--- a/drivers/crypto/qce/common.c
+++ b/drivers/crypto/qce/common.c
@@ -97,11 +97,11 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
 }
 
 #ifdef CONFIG_CRYPTO_DEV_QCE_SHA
-static u32 qce_auth_cfg(unsigned long flags, u32 key_size)
+static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
 {
 	u32 cfg = 0;
 
-	if (IS_AES(flags) && (IS_CCM(flags) || IS_CMAC(flags)))
+	if (IS_CCM(flags) || IS_CMAC(flags))
 		cfg |= AUTH_ALG_AES << AUTH_ALG_SHIFT;
 	else
 		cfg |= AUTH_ALG_SHA << AUTH_ALG_SHIFT;
@@ -119,15 +119,16 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size)
 		cfg |= AUTH_SIZE_SHA256 << AUTH_SIZE_SHIFT;
 	else if (IS_CMAC(flags))
 		cfg |= AUTH_SIZE_ENUM_16_BYTES << AUTH_SIZE_SHIFT;
+	else if (IS_CCM(flags))
+		cfg |= (auth_size - 1) << AUTH_SIZE_SHIFT;
 
 	if (IS_SHA1(flags) || IS_SHA256(flags))
 		cfg |= AUTH_MODE_HASH << AUTH_MODE_SHIFT;
-	else if (IS_SHA1_HMAC(flags) || IS_SHA256_HMAC(flags) ||
-		 IS_CBC(flags) || IS_CTR(flags))
+	else if (IS_SHA1_HMAC(flags) || IS_SHA256_HMAC(flags))
 		cfg |= AUTH_MODE_HMAC << AUTH_MODE_SHIFT;
-	else if (IS_AES(flags) && IS_CCM(flags))
+	else if (IS_CCM(flags))
 		cfg |= AUTH_MODE_CCM << AUTH_MODE_SHIFT;
-	else if (IS_AES(flags) && IS_CMAC(flags))
+	else if (IS_CMAC(flags))
 		cfg |= AUTH_MODE_CMAC << AUTH_MODE_SHIFT;
 
 	if (IS_SHA(flags) || IS_SHA_HMAC(flags))
@@ -136,10 +137,6 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size)
 	if (IS_CCM(flags))
 		cfg |= QCE_MAX_NONCE_WORDS << AUTH_NONCE_NUM_WORDS_SHIFT;
 
-	if (IS_CBC(flags) || IS_CTR(flags) || IS_CCM(flags) ||
-	    IS_CMAC(flags))
-		cfg |= BIT(AUTH_LAST_SHIFT) | BIT(AUTH_FIRST_SHIFT);
-
 	return cfg;
 }
 
@@ -171,7 +168,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
 		qce_clear_array(qce, REG_AUTH_KEY0, 16);
 		qce_clear_array(qce, REG_AUTH_BYTECNT0, 4);
 
-		auth_cfg = qce_auth_cfg(rctx->flags, rctx->authklen);
+		auth_cfg = qce_auth_cfg(rctx->flags, rctx->authklen, digestsize);
 	}
 
 	if (IS_SHA_HMAC(rctx->flags) || IS_CMAC(rctx->flags)) {
@@ -199,7 +196,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
 		qce_write_array(qce, REG_AUTH_BYTECNT0,
 				(u32 *)rctx->byte_count, 2);
 
-	auth_cfg = qce_auth_cfg(rctx->flags, 0);
+	auth_cfg = qce_auth_cfg(rctx->flags, 0, digestsize);
 
 	if (rctx->last_blk)
 		auth_cfg |= BIT(AUTH_LAST_SHIFT);
-- 
2.25.1

[PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Thara Gopinath]

Add register programming sequence for enabling AEAD
algorithms on the Qualcomm crypto engine.

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
 1 file changed, 153 insertions(+), 2 deletions(-)

diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
index 05a71c5ecf61..54d209cb0525 100644
--- a/drivers/crypto/qce/common.c
+++ b/drivers/crypto/qce/common.c
@@ -15,6 +15,16 @@
 #include "core.h"
 #include "regs-v5.h"
 #include "sha.h"
+#include "aead.h"
+
+static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
+	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
+};
+
+static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
+	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
+	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
+};
 
 static inline u32 qce_read(struct qce_device *qce, u32 offset)
 {
@@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
 		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
 }
 
-#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
+#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
 static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
 {
 	u32 cfg = 0;
@@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
 
 	return cfg;
 }
+#endif
 
+#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
 static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
 {
 	struct ahash_request *req = ahash_request_cast(async_req);
@@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
 }
 #endif
 
-#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
+#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
 static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
 {
 	u32 cfg = 0;
@@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
 
 	return cfg;
 }
+#endif
 
+#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
 static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
 {
 	u8 swap[QCE_AES_IV_LENGTH];
@@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
 }
 #endif
 
+#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
+static int qce_setup_regs_aead(struct crypto_async_request *async_req)
+{
+	struct aead_request *req = aead_request_cast(async_req);
+	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
+	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
+	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
+	struct qce_device *qce = tmpl->qce;
+	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
+	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
+	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
+	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
+	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
+	unsigned int enc_keylen = ctx->enc_keylen;
+	unsigned int auth_keylen = ctx->auth_keylen;
+	unsigned int enc_ivsize = rctx->ivsize;
+	unsigned int auth_ivsize;
+	unsigned int enckey_words, enciv_words;
+	unsigned int authkey_words, authiv_words, authnonce_words;
+	unsigned long flags = rctx->flags;
+	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
+	u32 *iv_last_word;
+
+	qce_setup_config(qce);
+
+	/* Write encryption key */
+	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
+	enckey_words = enc_keylen / sizeof(u32);
+	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
+
+	/* Write encryption iv */
+	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
+	enciv_words = enc_ivsize / sizeof(u32);
+	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
+
+	if (IS_CCM(rctx->flags)) {
+		iv_last_word = (u32 *)&enciv[enciv_words - 1];
+//		qce_write(qce, REG_CNTR3_IV3, enciv[enciv_words - 1] + 1);
+		qce_write(qce, REG_CNTR3_IV3, (*iv_last_word) + 1);
+		qce_write_array(qce, REG_ENCR_CCM_INT_CNTR0, (u32 *)enciv, enciv_words);
+		qce_write(qce, REG_CNTR_MASK, ~0);
+		qce_write(qce, REG_CNTR_MASK0, ~0);
+		qce_write(qce, REG_CNTR_MASK1, ~0);
+		qce_write(qce, REG_CNTR_MASK2, ~0);
+	}
+
+	/* Clear authentication IV and KEY registers of previous values */
+	qce_clear_array(qce, REG_AUTH_IV0, 16);
+	qce_clear_array(qce, REG_AUTH_KEY0, 16);
+
+	/* Clear byte count */
+	qce_clear_array(qce, REG_AUTH_BYTECNT0, 4);
+
+	/* Write authentication key */
+	qce_cpu_to_be32p_array(authkey, ctx->auth_key, auth_keylen);
+	authkey_words = DIV_ROUND_UP(auth_keylen, sizeof(u32));
+	qce_write_array(qce, REG_AUTH_KEY0, (u32 *)authkey, authkey_words);
+
+	if (IS_SHA_HMAC(rctx->flags)) {
+		/* Write default authentication iv */
+		if (IS_SHA1_HMAC(rctx->flags)) {
+			auth_ivsize = SHA1_DIGEST_SIZE;
+			memcpy(authiv, std_iv_sha1, auth_ivsize);
+		} else if (IS_SHA256_HMAC(rctx->flags)) {
+			auth_ivsize = SHA256_DIGEST_SIZE;
+			memcpy(authiv, std_iv_sha256, auth_ivsize);
+		}
+		authiv_words = auth_ivsize / sizeof(u32);
+		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);
+	}
+
+	if (IS_CCM(rctx->flags)) {
+		qce_cpu_to_be32p_array(authnonce, rctx->ccm_nonce, QCE_MAX_NONCE);
+		authnonce_words = QCE_MAX_NONCE / sizeof(u32);
+		qce_write_array(qce, REG_AUTH_INFO_NONCE0, (u32 *)authnonce, authnonce_words);
+	}
+
+	/* Set up ENCR_SEG_CFG */
+	encr_cfg = qce_encr_cfg(flags, enc_keylen);
+	if (IS_ENCRYPT(flags))
+		encr_cfg |= BIT(ENCODE_SHIFT);
+	qce_write(qce, REG_ENCR_SEG_CFG, encr_cfg);
+
+	/* Set up AUTH_SEG_CFG */
+	auth_cfg = qce_auth_cfg(rctx->flags, auth_keylen, ctx->authsize);
+	auth_cfg |= BIT(AUTH_LAST_SHIFT);
+	auth_cfg |= BIT(AUTH_FIRST_SHIFT);
+	if (IS_ENCRYPT(flags)) {
+		if (IS_CCM(rctx->flags))
+			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
+		else
+			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
+	} else {
+		if (IS_CCM(rctx->flags))
+			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
+		else
+			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
+	}
+	qce_write(qce, REG_AUTH_SEG_CFG, auth_cfg);
+
+	totallen = rctx->cryptlen + rctx->assoclen;
+
+	/* Set the encryption size and start offset */
+	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
+		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen + ctx->authsize);
+	else
+		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen);
+	qce_write(qce, REG_ENCR_SEG_START, rctx->assoclen & 0xffff);
+
+	/* Set the authentication size and start offset */
+	qce_write(qce, REG_AUTH_SEG_SIZE, totallen);
+	qce_write(qce, REG_AUTH_SEG_START, 0);
+
+	/* Write total length */
+	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
+		qce_write(qce, REG_SEG_SIZE, totallen + ctx->authsize);
+	else
+		qce_write(qce, REG_SEG_SIZE, totallen);
+
+	/* get little endianness */
+	config = qce_config_reg(qce, 1);
+	qce_write(qce, REG_CONFIG, config);
+
+	/* Start the process */
+	if (IS_CCM(flags))
+		qce_crypto_go(qce, 0);
+	else
+		qce_crypto_go(qce, 1);
+
+	return 0;
+}
+#endif
+
 int qce_start(struct crypto_async_request *async_req, u32 type)
 {
 	switch (type) {
@@ -396,6 +543,10 @@ int qce_start(struct crypto_async_request *async_req, u32 type)
 #ifdef CONFIG_CRYPTO_DEV_QCE_SHA
 	case CRYPTO_ALG_TYPE_AHASH:
 		return qce_setup_regs_ahash(async_req);
+#endif
+#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
+	case CRYPTO_ALG_TYPE_AEAD:
+		return qce_setup_regs_aead(async_req);
 #endif
 	default:
 		return -EINVAL;
-- 
2.25.1

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Bjorn Andersson]

On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:

> Add register programming sequence for enabling AEAD
> algorithms on the Qualcomm crypto engine.
> 
> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> ---
>  drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
>  1 file changed, 153 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
> index 05a71c5ecf61..54d209cb0525 100644
> --- a/drivers/crypto/qce/common.c
> +++ b/drivers/crypto/qce/common.c
> @@ -15,6 +15,16 @@
>  #include "core.h"
>  #include "regs-v5.h"
>  #include "sha.h"
> +#include "aead.h"
> +
> +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
> +};
> +
> +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
> +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
> +};
>  
>  static inline u32 qce_read(struct qce_device *qce, u32 offset)
>  {
> @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
>  		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
>  }
>  
> -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>  static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>  {
>  	u32 cfg = 0;
> @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>  
>  	return cfg;
>  }
> +#endif
>  
> +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>  static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>  {
>  	struct ahash_request *req = ahash_request_cast(async_req);
> @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>  }
>  #endif
>  
> -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>  static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>  {
>  	u32 cfg = 0;
> @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>  
>  	return cfg;
>  }
> +#endif
>  
> +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>  static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
>  {
>  	u8 swap[QCE_AES_IV_LENGTH];
> @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
>  }
>  #endif
>  
> +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
> +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
> +{
> +	struct aead_request *req = aead_request_cast(async_req);
> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
> +	struct qce_device *qce = tmpl->qce;
> +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
> +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
> +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
> +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
> +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
> +	unsigned int enc_keylen = ctx->enc_keylen;
> +	unsigned int auth_keylen = ctx->auth_keylen;
> +	unsigned int enc_ivsize = rctx->ivsize;
> +	unsigned int auth_ivsize;
> +	unsigned int enckey_words, enciv_words;
> +	unsigned int authkey_words, authiv_words, authnonce_words;
> +	unsigned long flags = rctx->flags;
> +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;

I don't see any reason to initialize encr_cfg or auth_cfg.

> +	u32 *iv_last_word;
> +
> +	qce_setup_config(qce);
> +
> +	/* Write encryption key */
> +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
> +	enckey_words = enc_keylen / sizeof(u32);
> +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);

Afaict all "array registers" in this function are affected by the
CRYPTO_SETUP little endian bit, but you set this bit before launching
the operation dependent on IS_CCM(). So is this really working for the
!IS_CCM() case?

> +
> +	/* Write encryption iv */
> +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
> +	enciv_words = enc_ivsize / sizeof(u32);
> +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);

It would be nice if this snippet was extracted to a helper function.

> +
> +	if (IS_CCM(rctx->flags)) {
> +		iv_last_word = (u32 *)&enciv[enciv_words - 1];
> +//		qce_write(qce, REG_CNTR3_IV3, enciv[enciv_words - 1] + 1);

I believe this is a remnant of the two surrounding lines.

> +		qce_write(qce, REG_CNTR3_IV3, (*iv_last_word) + 1);

enciv is an array of big endian 32-bit integers, which you tell the
compiler to treat as cpu-native endian, and then you do math on it.
Afaict from the documentation the value of REG_CNTR3_IVn should be set
to rctx->iv + 1, but if the hardware expects these in big endian then I
think you added 16777216.

Perhaps I'm missing something here though?

PS. Based on how the documentation is written, shouldn't you write out
REG_CNTR_IV[012] as well?

> +		qce_write_array(qce, REG_ENCR_CCM_INT_CNTR0, (u32 *)enciv, enciv_words);
> +		qce_write(qce, REG_CNTR_MASK, ~0);
> +		qce_write(qce, REG_CNTR_MASK0, ~0);
> +		qce_write(qce, REG_CNTR_MASK1, ~0);
> +		qce_write(qce, REG_CNTR_MASK2, ~0);
> +	}
> +
> +	/* Clear authentication IV and KEY registers of previous values */
> +	qce_clear_array(qce, REG_AUTH_IV0, 16);
> +	qce_clear_array(qce, REG_AUTH_KEY0, 16);
> +
> +	/* Clear byte count */
> +	qce_clear_array(qce, REG_AUTH_BYTECNT0, 4);
> +
> +	/* Write authentication key */
> +	qce_cpu_to_be32p_array(authkey, ctx->auth_key, auth_keylen);
> +	authkey_words = DIV_ROUND_UP(auth_keylen, sizeof(u32));
> +	qce_write_array(qce, REG_AUTH_KEY0, (u32 *)authkey, authkey_words);
> +
> +	if (IS_SHA_HMAC(rctx->flags)) {
> +		/* Write default authentication iv */
> +		if (IS_SHA1_HMAC(rctx->flags)) {
> +			auth_ivsize = SHA1_DIGEST_SIZE;
> +			memcpy(authiv, std_iv_sha1, auth_ivsize);
> +		} else if (IS_SHA256_HMAC(rctx->flags)) {
> +			auth_ivsize = SHA256_DIGEST_SIZE;
> +			memcpy(authiv, std_iv_sha256, auth_ivsize);
> +		}
> +		authiv_words = auth_ivsize / sizeof(u32);
> +		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);

AUTH_IV0 is affected by the little endian configuration, does this imply
that IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags? If so
I think it would be nice if you grouped the conditionals in a way that
made that obvious when reading the function.

> +	}
> +
> +	if (IS_CCM(rctx->flags)) {
> +		qce_cpu_to_be32p_array(authnonce, rctx->ccm_nonce, QCE_MAX_NONCE);
> +		authnonce_words = QCE_MAX_NONCE / sizeof(u32);
> +		qce_write_array(qce, REG_AUTH_INFO_NONCE0, (u32 *)authnonce, authnonce_words);
> +	}
> +
> +	/* Set up ENCR_SEG_CFG */
> +	encr_cfg = qce_encr_cfg(flags, enc_keylen);
> +	if (IS_ENCRYPT(flags))
> +		encr_cfg |= BIT(ENCODE_SHIFT);
> +	qce_write(qce, REG_ENCR_SEG_CFG, encr_cfg);
> +
> +	/* Set up AUTH_SEG_CFG */
> +	auth_cfg = qce_auth_cfg(rctx->flags, auth_keylen, ctx->authsize);
> +	auth_cfg |= BIT(AUTH_LAST_SHIFT);
> +	auth_cfg |= BIT(AUTH_FIRST_SHIFT);
> +	if (IS_ENCRYPT(flags)) {
> +		if (IS_CCM(rctx->flags))
> +			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
> +		else
> +			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
> +	} else {
> +		if (IS_CCM(rctx->flags))
> +			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
> +		else
> +			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
> +	}
> +	qce_write(qce, REG_AUTH_SEG_CFG, auth_cfg);
> +
> +	totallen = rctx->cryptlen + rctx->assoclen;
> +
> +	/* Set the encryption size and start offset */
> +	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
> +		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen + ctx->authsize);
> +	else
> +		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen);
> +	qce_write(qce, REG_ENCR_SEG_START, rctx->assoclen & 0xffff);
> +
> +	/* Set the authentication size and start offset */
> +	qce_write(qce, REG_AUTH_SEG_SIZE, totallen);
> +	qce_write(qce, REG_AUTH_SEG_START, 0);
> +
> +	/* Write total length */
> +	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
> +		qce_write(qce, REG_SEG_SIZE, totallen + ctx->authsize);
> +	else
> +		qce_write(qce, REG_SEG_SIZE, totallen);
> +
> +	/* get little endianness */
> +	config = qce_config_reg(qce, 1);
> +	qce_write(qce, REG_CONFIG, config);
> +
> +	/* Start the process */
> +	if (IS_CCM(flags))
> +		qce_crypto_go(qce, 0);

Second parameter is defined as "bool", please use "false" here (and true
below). Or

	qce_crypto_go(qce, !IS_CCM(flags));

Regards,
Bjorn

> +	else
> +		qce_crypto_go(qce, 1);
> +
> +	return 0;
> +}
> +#endif
> +
>  int qce_start(struct crypto_async_request *async_req, u32 type)
>  {
>  	switch (type) {
> @@ -396,6 +543,10 @@ int qce_start(struct crypto_async_request *async_req, u32 type)
>  #ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>  	case CRYPTO_ALG_TYPE_AHASH:
>  		return qce_setup_regs_ahash(async_req);
> +#endif
> +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
> +	case CRYPTO_ALG_TYPE_AEAD:
> +		return qce_setup_regs_aead(async_req);
>  #endif
>  	default:
>  		return -EINVAL;
> -- 
> 2.25.1
> 

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Thara Gopinath]

Hi Bjorn,

On 4/5/21 6:18 PM, Bjorn Andersson wrote:
> On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> 
>> Add register programming sequence for enabling AEAD
>> algorithms on the Qualcomm crypto engine.
>>
>> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
>> ---
>>   drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
>>   1 file changed, 153 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
>> index 05a71c5ecf61..54d209cb0525 100644
>> --- a/drivers/crypto/qce/common.c
>> +++ b/drivers/crypto/qce/common.c
>> @@ -15,6 +15,16 @@
>>   #include "core.h"
>>   #include "regs-v5.h"
>>   #include "sha.h"
>> +#include "aead.h"
>> +
>> +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>> +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
>> +};
>> +
>> +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>> +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
>> +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
>> +};
>>   
>>   static inline u32 qce_read(struct qce_device *qce, u32 offset)
>>   {
>> @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
>>   		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
>>   }
>>   
>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>   static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>   {
>>   	u32 cfg = 0;
>> @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>   
>>   	return cfg;
>>   }
>> +#endif
>>   
>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>>   static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>   {
>>   	struct ahash_request *req = ahash_request_cast(async_req);
>> @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>   }
>>   #endif
>>   
>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>   static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>   {
>>   	u32 cfg = 0;
>> @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>   
>>   	return cfg;
>>   }
>> +#endif
>>   
>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>>   static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
>>   {
>>   	u8 swap[QCE_AES_IV_LENGTH];
>> @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
>>   }
>>   #endif
>>   
>> +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
>> +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
>> +{
>> +	struct aead_request *req = aead_request_cast(async_req);
>> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
>> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
>> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
>> +	struct qce_device *qce = tmpl->qce;
>> +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
>> +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
>> +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
>> +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
>> +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
>> +	unsigned int enc_keylen = ctx->enc_keylen;
>> +	unsigned int auth_keylen = ctx->auth_keylen;
>> +	unsigned int enc_ivsize = rctx->ivsize;
>> +	unsigned int auth_ivsize;
>> +	unsigned int enckey_words, enciv_words;
>> +	unsigned int authkey_words, authiv_words, authnonce_words;
>> +	unsigned long flags = rctx->flags;
>> +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
> 
> I don't see any reason to initialize encr_cfg or auth_cfg.

right.. I will remove it

> 
>> +	u32 *iv_last_word;
>> +
>> +	qce_setup_config(qce);
>> +
>> +	/* Write encryption key */
>> +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
>> +	enckey_words = enc_keylen / sizeof(u32);
>> +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
> 
> Afaict all "array registers" in this function are affected by the
> CRYPTO_SETUP little endian bit, but you set this bit before launching
> the operation dependent on IS_CCM(). So is this really working for the
> !IS_CCM() case?

I am not sure I understand you. Below ,
	/* get little endianness */
	config = qce_config_reg(qce, 1);
	qce_write(qce, REG_CONFIG, config);

is outside of any checks..

> 
>> +
>> +	/* Write encryption iv */
>> +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
>> +	enciv_words = enc_ivsize / sizeof(u32);
>> +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
> 
> It would be nice if this snippet was extracted to a helper function.
> 
>> +
>> +	if (IS_CCM(rctx->flags)) {
>> +		iv_last_word = (u32 *)&enciv[enciv_words - 1];
>> +//		qce_write(qce, REG_CNTR3_IV3, enciv[enciv_words - 1] + 1);
> 
> I believe this is a remnant of the two surrounding lines.

It indeed is.. I will remove it.

> 
>> +		qce_write(qce, REG_CNTR3_IV3, (*iv_last_word) + 1);
> 
> enciv is an array of big endian 32-bit integers, which you tell the
> compiler to treat as cpu-native endian, and then you do math on it.
> Afaict from the documentation the value of REG_CNTR3_IVn should be set
> to rctx->iv + 1, but if the hardware expects these in big endian then I
> think you added 16777216.

So, the crypto engine documentation talks of writing to these registers 
in little endian mode. The byte stream that you get for iv from the user
is in big endian mode as in the MSB is byte 0. So we kind of invert this 
and write  to these registers. This is what happens with declaring the 
__be32 array and copying words to it from the byte stream. So now byte 0 
is the LSB and a +1 will just add a 1 to it.

I suspect from what I read in the documentation we could get away by 
removing this and writing the big endian byte stream directly and never 
setting the little endian in config register. Though I am not sure if 
this has ever been tested out. If we change it, it will be across 
algorithms and as a separate effort.

> 
> Perhaps I'm missing something here though?
> 
> PS. Based on how the documentation is written, shouldn't you write out
> REG_CNTR_IV[012] as well?

It is done on top, right ?
qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);

> 
>> +		qce_write_array(qce, REG_ENCR_CCM_INT_CNTR0, (u32 *)enciv, enciv_words);
>> +		qce_write(qce, REG_CNTR_MASK, ~0);
>> +		qce_write(qce, REG_CNTR_MASK0, ~0);
>> +		qce_write(qce, REG_CNTR_MASK1, ~0);
>> +		qce_write(qce, REG_CNTR_MASK2, ~0);
>> +	}
>> +
>> +	/* Clear authentication IV and KEY registers of previous values */
>> +	qce_clear_array(qce, REG_AUTH_IV0, 16);
>> +	qce_clear_array(qce, REG_AUTH_KEY0, 16);
>> +
>> +	/* Clear byte count */
>> +	qce_clear_array(qce, REG_AUTH_BYTECNT0, 4);
>> +
>> +	/* Write authentication key */
>> +	qce_cpu_to_be32p_array(authkey, ctx->auth_key, auth_keylen);
>> +	authkey_words = DIV_ROUND_UP(auth_keylen, sizeof(u32));
>> +	qce_write_array(qce, REG_AUTH_KEY0, (u32 *)authkey, authkey_words);
>> +
>> +	if (IS_SHA_HMAC(rctx->flags)) {
>> +		/* Write default authentication iv */
>> +		if (IS_SHA1_HMAC(rctx->flags)) {
>> +			auth_ivsize = SHA1_DIGEST_SIZE;
>> +			memcpy(authiv, std_iv_sha1, auth_ivsize);
>> +		} else if (IS_SHA256_HMAC(rctx->flags)) {
>> +			auth_ivsize = SHA256_DIGEST_SIZE;
>> +			memcpy(authiv, std_iv_sha256, auth_ivsize);
>> +		}
>> +		authiv_words = auth_ivsize / sizeof(u32);
>> +		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);
> 
> AUTH_IV0 is affected by the little endian configuration, does this imply
> that IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags? If so
> I think it would be nice if you grouped the conditionals in a way that
> made that obvious when reading the function.

So yes IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags. 
AUTH_IVn is 0 for ccm and has initial value for HMAC algorithms. I don't 
understand the confusion here.

> 
>> +	}
>> +
>> +	if (IS_CCM(rctx->flags)) {
>> +		qce_cpu_to_be32p_array(authnonce, rctx->ccm_nonce, QCE_MAX_NONCE);
>> +		authnonce_words = QCE_MAX_NONCE / sizeof(u32);
>> +		qce_write_array(qce, REG_AUTH_INFO_NONCE0, (u32 *)authnonce, authnonce_words);
>> +	}
>> +
>> +	/* Set up ENCR_SEG_CFG */
>> +	encr_cfg = qce_encr_cfg(flags, enc_keylen);
>> +	if (IS_ENCRYPT(flags))
>> +		encr_cfg |= BIT(ENCODE_SHIFT);
>> +	qce_write(qce, REG_ENCR_SEG_CFG, encr_cfg);
>> +
>> +	/* Set up AUTH_SEG_CFG */
>> +	auth_cfg = qce_auth_cfg(rctx->flags, auth_keylen, ctx->authsize);
>> +	auth_cfg |= BIT(AUTH_LAST_SHIFT);
>> +	auth_cfg |= BIT(AUTH_FIRST_SHIFT);
>> +	if (IS_ENCRYPT(flags)) {
>> +		if (IS_CCM(rctx->flags))
>> +			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
>> +		else
>> +			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
>> +	} else {
>> +		if (IS_CCM(rctx->flags))
>> +			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
>> +		else
>> +			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
>> +	}
>> +	qce_write(qce, REG_AUTH_SEG_CFG, auth_cfg);
>> +
>> +	totallen = rctx->cryptlen + rctx->assoclen;
>> +
>> +	/* Set the encryption size and start offset */
>> +	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
>> +		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen + ctx->authsize);
>> +	else
>> +		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen);
>> +	qce_write(qce, REG_ENCR_SEG_START, rctx->assoclen & 0xffff);
>> +
>> +	/* Set the authentication size and start offset */
>> +	qce_write(qce, REG_AUTH_SEG_SIZE, totallen);
>> +	qce_write(qce, REG_AUTH_SEG_START, 0);
>> +
>> +	/* Write total length */
>> +	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
>> +		qce_write(qce, REG_SEG_SIZE, totallen + ctx->authsize);
>> +	else
>> +		qce_write(qce, REG_SEG_SIZE, totallen);
>> +
>> +	/* get little endianness */
>> +	config = qce_config_reg(qce, 1);
>> +	qce_write(qce, REG_CONFIG, config);
>> +
>> +	/* Start the process */
>> +	if (IS_CCM(flags))
>> +		qce_crypto_go(qce, 0);
> 
> Second parameter is defined as "bool", please use "false" here (and true
> below). Or
> 
> 	qce_crypto_go(qce, !IS_CCM(flags));

will do... I like the one liner better.


-- 
Warm Regards
Thara

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Bjorn Andersson]

On Tue 13 Apr 16:31 CDT 2021, Thara Gopinath wrote:

> 
> Hi Bjorn,
> 
> On 4/5/21 6:18 PM, Bjorn Andersson wrote:
> > On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> > 
> > > Add register programming sequence for enabling AEAD
> > > algorithms on the Qualcomm crypto engine.
> > > 
> > > Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> > > ---
> > >   drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
> > >   1 file changed, 153 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
> > > index 05a71c5ecf61..54d209cb0525 100644
> > > --- a/drivers/crypto/qce/common.c
> > > +++ b/drivers/crypto/qce/common.c
> > > @@ -15,6 +15,16 @@
> > >   #include "core.h"
> > >   #include "regs-v5.h"
> > >   #include "sha.h"
> > > +#include "aead.h"
> > > +
> > > +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> > > +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
> > > +};
> > > +
> > > +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> > > +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
> > > +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
> > > +};
> > >   static inline u32 qce_read(struct qce_device *qce, u32 offset)
> > >   {
> > > @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
> > >   		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
> > >   }
> > > -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> > > +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
> > >   static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
> > >   {
> > >   	u32 cfg = 0;
> > > @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
> > >   	return cfg;
> > >   }
> > > +#endif
> > > +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> > >   static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
> > >   {
> > >   	struct ahash_request *req = ahash_request_cast(async_req);
> > > @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
> > >   }
> > >   #endif
> > > -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> > > +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
> > >   static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
> > >   {
> > >   	u32 cfg = 0;
> > > @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
> > >   	return cfg;
> > >   }
> > > +#endif
> > > +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> > >   static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
> > >   {
> > >   	u8 swap[QCE_AES_IV_LENGTH];
> > > @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
> > >   }
> > >   #endif
> > > +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
> > > +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
> > > +{
> > > +	struct aead_request *req = aead_request_cast(async_req);
> > > +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
> > > +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
> > > +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
> > > +	struct qce_device *qce = tmpl->qce;
> > > +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
> > > +	unsigned int enc_keylen = ctx->enc_keylen;
> > > +	unsigned int auth_keylen = ctx->auth_keylen;
> > > +	unsigned int enc_ivsize = rctx->ivsize;
> > > +	unsigned int auth_ivsize;
> > > +	unsigned int enckey_words, enciv_words;
> > > +	unsigned int authkey_words, authiv_words, authnonce_words;
> > > +	unsigned long flags = rctx->flags;
> > > +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
> > 
> > I don't see any reason to initialize encr_cfg or auth_cfg.
> 
> right.. I will remove it
> 
> > 
> > > +	u32 *iv_last_word;
> > > +
> > > +	qce_setup_config(qce);
> > > +
> > > +	/* Write encryption key */
> > > +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
> > > +	enckey_words = enc_keylen / sizeof(u32);
> > > +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
> > 
> > Afaict all "array registers" in this function are affected by the
> > CRYPTO_SETUP little endian bit, but you set this bit before launching
> > the operation dependent on IS_CCM(). So is this really working for the
> > !IS_CCM() case?
> 
> I am not sure I understand you. Below ,
> 	/* get little endianness */
> 	config = qce_config_reg(qce, 1);
> 	qce_write(qce, REG_CONFIG, config);
> 
> is outside of any checks..
> 

You're right, I misread that snippet as I was jumping through the
function. So we're unconditionally running the hardware in little endian
mode.

> > 
> > > +
> > > +	/* Write encryption iv */
> > > +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
> > > +	enciv_words = enc_ivsize / sizeof(u32);
> > > +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
> > 
> > It would be nice if this snippet was extracted to a helper function.
> > 
> > > +
> > > +	if (IS_CCM(rctx->flags)) {
> > > +		iv_last_word = (u32 *)&enciv[enciv_words - 1];
> > > +//		qce_write(qce, REG_CNTR3_IV3, enciv[enciv_words - 1] + 1);
> > 
> > I believe this is a remnant of the two surrounding lines.
> 
> It indeed is.. I will remove it.
> 
> > 
> > > +		qce_write(qce, REG_CNTR3_IV3, (*iv_last_word) + 1);
> > 
> > enciv is an array of big endian 32-bit integers, which you tell the
> > compiler to treat as cpu-native endian, and then you do math on it.
> > Afaict from the documentation the value of REG_CNTR3_IVn should be set
> > to rctx->iv + 1, but if the hardware expects these in big endian then I
> > think you added 16777216.
> 
> So, the crypto engine documentation talks of writing to these registers in
> little endian mode. The byte stream that you get for iv from the user
> is in big endian mode as in the MSB is byte 0. So we kind of invert this and
> write  to these registers. This is what happens with declaring the __be32
> array and copying words to it from the byte stream. So now byte 0 is the LSB
> and a +1 will just add a 1 to it.
> 

But if the data come in big endian and after qce_cpu_to_be32p_array()
you're able to do math on them with expected result and you're finally
passing the data to writel() then I think that qce_cpu_to_be32p_array()
is actually be32_to_cpu() and after the conversion you should carry the
results in CPU-native u32 arrays - and thereby skip the typecasting.

> I suspect from what I read in the documentation we could get away by
> removing this and writing the big endian byte stream directly and never
> setting the little endian in config register. Though I am not sure if this
> has ever been tested out. If we change it, it will be across algorithms and
> as a separate effort.

writel() will, at least on arm64, convert the CPU native value to little
endian before writing it out, so I think the current setting make sense.

> 
> > 
> > Perhaps I'm missing something here though?
> > 
> > PS. Based on how the documentation is written, shouldn't you write out
> > REG_CNTR_IV[012] as well?
> 
> It is done on top, right ?
> qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
> 

You're right, depending on enciv_words you write the 4 registers, then
increment the last word and write that out again.

> > 
> > > +		qce_write_array(qce, REG_ENCR_CCM_INT_CNTR0, (u32 *)enciv, enciv_words);
> > > +		qce_write(qce, REG_CNTR_MASK, ~0);
> > > +		qce_write(qce, REG_CNTR_MASK0, ~0);
> > > +		qce_write(qce, REG_CNTR_MASK1, ~0);
> > > +		qce_write(qce, REG_CNTR_MASK2, ~0);
> > > +	}
> > > +
> > > +	/* Clear authentication IV and KEY registers of previous values */
> > > +	qce_clear_array(qce, REG_AUTH_IV0, 16);
> > > +	qce_clear_array(qce, REG_AUTH_KEY0, 16);
> > > +
> > > +	/* Clear byte count */
> > > +	qce_clear_array(qce, REG_AUTH_BYTECNT0, 4);
> > > +
> > > +	/* Write authentication key */
> > > +	qce_cpu_to_be32p_array(authkey, ctx->auth_key, auth_keylen);
> > > +	authkey_words = DIV_ROUND_UP(auth_keylen, sizeof(u32));
> > > +	qce_write_array(qce, REG_AUTH_KEY0, (u32 *)authkey, authkey_words);
> > > +
> > > +	if (IS_SHA_HMAC(rctx->flags)) {
> > > +		/* Write default authentication iv */
> > > +		if (IS_SHA1_HMAC(rctx->flags)) {
> > > +			auth_ivsize = SHA1_DIGEST_SIZE;
> > > +			memcpy(authiv, std_iv_sha1, auth_ivsize);
> > > +		} else if (IS_SHA256_HMAC(rctx->flags)) {
> > > +			auth_ivsize = SHA256_DIGEST_SIZE;
> > > +			memcpy(authiv, std_iv_sha256, auth_ivsize);
> > > +		}
> > > +		authiv_words = auth_ivsize / sizeof(u32);
> > > +		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);
> > 
> > AUTH_IV0 is affected by the little endian configuration, does this imply
> > that IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags? If so
> > I think it would be nice if you grouped the conditionals in a way that
> > made that obvious when reading the function.
> 
> So yes IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags.
> AUTH_IVn is 0 for ccm and has initial value for HMAC algorithms. I don't
> understand the confusion here.
> 

I'm just saying that writing is as below would have made it obvious to
me that IS_SHA_HMAC() and IS_CCM() are exclusive:

if (IS_SHA_HMAC(flags)) {
  ...
} else if (IS_CCM(flags)) {
  ....
}

Regards,
Bjorn

> > 
> > > +	}
> > > +
> > > +	if (IS_CCM(rctx->flags)) {
> > > +		qce_cpu_to_be32p_array(authnonce, rctx->ccm_nonce, QCE_MAX_NONCE);
> > > +		authnonce_words = QCE_MAX_NONCE / sizeof(u32);
> > > +		qce_write_array(qce, REG_AUTH_INFO_NONCE0, (u32 *)authnonce, authnonce_words);
> > > +	}
> > > +
> > > +	/* Set up ENCR_SEG_CFG */
> > > +	encr_cfg = qce_encr_cfg(flags, enc_keylen);
> > > +	if (IS_ENCRYPT(flags))
> > > +		encr_cfg |= BIT(ENCODE_SHIFT);
> > > +	qce_write(qce, REG_ENCR_SEG_CFG, encr_cfg);
> > > +
> > > +	/* Set up AUTH_SEG_CFG */
> > > +	auth_cfg = qce_auth_cfg(rctx->flags, auth_keylen, ctx->authsize);
> > > +	auth_cfg |= BIT(AUTH_LAST_SHIFT);
> > > +	auth_cfg |= BIT(AUTH_FIRST_SHIFT);
> > > +	if (IS_ENCRYPT(flags)) {
> > > +		if (IS_CCM(rctx->flags))
> > > +			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
> > > +		else
> > > +			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
> > > +	} else {
> > > +		if (IS_CCM(rctx->flags))
> > > +			auth_cfg |= AUTH_POS_AFTER << AUTH_POS_SHIFT;
> > > +		else
> > > +			auth_cfg |= AUTH_POS_BEFORE << AUTH_POS_SHIFT;
> > > +	}
> > > +	qce_write(qce, REG_AUTH_SEG_CFG, auth_cfg);
> > > +
> > > +	totallen = rctx->cryptlen + rctx->assoclen;
> > > +
> > > +	/* Set the encryption size and start offset */
> > > +	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
> > > +		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen + ctx->authsize);
> > > +	else
> > > +		qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen);
> > > +	qce_write(qce, REG_ENCR_SEG_START, rctx->assoclen & 0xffff);
> > > +
> > > +	/* Set the authentication size and start offset */
> > > +	qce_write(qce, REG_AUTH_SEG_SIZE, totallen);
> > > +	qce_write(qce, REG_AUTH_SEG_START, 0);
> > > +
> > > +	/* Write total length */
> > > +	if (IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags))
> > > +		qce_write(qce, REG_SEG_SIZE, totallen + ctx->authsize);
> > > +	else
> > > +		qce_write(qce, REG_SEG_SIZE, totallen);
> > > +
> > > +	/* get little endianness */
> > > +	config = qce_config_reg(qce, 1);
> > > +	qce_write(qce, REG_CONFIG, config);
> > > +
> > > +	/* Start the process */
> > > +	if (IS_CCM(flags))
> > > +		qce_crypto_go(qce, 0);
> > 
> > Second parameter is defined as "bool", please use "false" here (and true
> > below). Or
> > 
> > 	qce_crypto_go(qce, !IS_CCM(flags));
> 
> will do... I like the one liner better.
> 
> 
> -- 
> Warm Regards
> Thara

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Thara Gopinath]

On 4/13/21 6:20 PM, Bjorn Andersson wrote:
> On Tue 13 Apr 16:31 CDT 2021, Thara Gopinath wrote:
> 
>>
>> Hi Bjorn,
>>
>> On 4/5/21 6:18 PM, Bjorn Andersson wrote:
>>> On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
>>>
>>>> Add register programming sequence for enabling AEAD
>>>> algorithms on the Qualcomm crypto engine.
>>>>
>>>> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
>>>> ---
>>>>    drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
>>>>    1 file changed, 153 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
>>>> index 05a71c5ecf61..54d209cb0525 100644
>>>> --- a/drivers/crypto/qce/common.c
>>>> +++ b/drivers/crypto/qce/common.c
>>>> @@ -15,6 +15,16 @@
>>>>    #include "core.h"
>>>>    #include "regs-v5.h"
>>>>    #include "sha.h"
>>>> +#include "aead.h"
>>>> +
>>>> +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>>>> +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
>>>> +};
>>>> +
>>>> +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>>>> +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
>>>> +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
>>>> +};
>>>>    static inline u32 qce_read(struct qce_device *qce, u32 offset)
>>>>    {
>>>> @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
>>>>    		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
>>>>    }
>>>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>>>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>>>    static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>>>    {
>>>>    	u32 cfg = 0;
>>>> @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>>>    	return cfg;
>>>>    }
>>>> +#endif
>>>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>>>>    static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>>>    {
>>>>    	struct ahash_request *req = ahash_request_cast(async_req);
>>>> @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>>>    }
>>>>    #endif
>>>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>>>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>>>    static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>>>    {
>>>>    	u32 cfg = 0;
>>>> @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>>>    	return cfg;
>>>>    }
>>>> +#endif
>>>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>>>>    static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
>>>>    {
>>>>    	u8 swap[QCE_AES_IV_LENGTH];
>>>> @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
>>>>    }
>>>>    #endif
>>>> +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
>>>> +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
>>>> +{
>>>> +	struct aead_request *req = aead_request_cast(async_req);
>>>> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
>>>> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
>>>> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
>>>> +	struct qce_device *qce = tmpl->qce;
>>>> +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
>>>> +	unsigned int enc_keylen = ctx->enc_keylen;
>>>> +	unsigned int auth_keylen = ctx->auth_keylen;
>>>> +	unsigned int enc_ivsize = rctx->ivsize;
>>>> +	unsigned int auth_ivsize;
>>>> +	unsigned int enckey_words, enciv_words;
>>>> +	unsigned int authkey_words, authiv_words, authnonce_words;
>>>> +	unsigned long flags = rctx->flags;
>>>> +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
>>>
>>> I don't see any reason to initialize encr_cfg or auth_cfg.
>>
>> right.. I will remove it
>>
>>>
>>>> +	u32 *iv_last_word;
>>>> +
>>>> +	qce_setup_config(qce);
>>>> +
>>>> +	/* Write encryption key */
>>>> +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
>>>> +	enckey_words = enc_keylen / sizeof(u32);
>>>> +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
>>>
>>> Afaict all "array registers" in this function are affected by the
>>> CRYPTO_SETUP little endian bit, but you set this bit before launching
>>> the operation dependent on IS_CCM(). So is this really working for the
>>> !IS_CCM() case?
>>
>> I am not sure I understand you. Below ,
>> 	/* get little endianness */
>> 	config = qce_config_reg(qce, 1);
>> 	qce_write(qce, REG_CONFIG, config);
>>
>> is outside of any checks..
>>
> 
> You're right, I misread that snippet as I was jumping through the
> function. So we're unconditionally running the hardware in little endian
> mode.
> 
>>>
>>>> +
>>>> +	/* Write encryption iv */
>>>> +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
>>>> +	enciv_words = enc_ivsize / sizeof(u32);
>>>> +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
>>>
>>> It would be nice if this snippet was extracted to a helper function.
>>>
>>>> +
>>>> +	if (IS_CCM(rctx->flags)) {
>>>> +		iv_last_word = (u32 *)&enciv[enciv_words - 1];
>>>> +//		qce_write(qce, REG_CNTR3_IV3, enciv[enciv_words - 1] + 1);
>>>
>>> I believe this is a remnant of the two surrounding lines.
>>
>> It indeed is.. I will remove it.
>>
>>>
>>>> +		qce_write(qce, REG_CNTR3_IV3, (*iv_last_word) + 1);
>>>
>>> enciv is an array of big endian 32-bit integers, which you tell the
>>> compiler to treat as cpu-native endian, and then you do math on it.
>>> Afaict from the documentation the value of REG_CNTR3_IVn should be set
>>> to rctx->iv + 1, but if the hardware expects these in big endian then I
>>> think you added 16777216.
>>
>> So, the crypto engine documentation talks of writing to these registers in
>> little endian mode. The byte stream that you get for iv from the user
>> is in big endian mode as in the MSB is byte 0. So we kind of invert this and
>> write  to these registers. This is what happens with declaring the __be32
>> array and copying words to it from the byte stream. So now byte 0 is the LSB
>> and a +1 will just add a 1 to it.
>>
> 
> But if the data come in big endian and after qce_cpu_to_be32p_array()
> you're able to do math on them with expected result and you're finally
> passing the data to writel() then I think that qce_cpu_to_be32p_array()
> is actually be32_to_cpu() and after the conversion you should carry the
> results in CPU-native u32 arrays - and thereby skip the typecasting.

you mean I can replace __be32 arrays with u32 arrays ?? yes I probably 
can. I will try this out and if it works do the change.

> 
>> I suspect from what I read in the documentation we could get away by
>> removing this and writing the big endian byte stream directly and never
>> setting the little endian in config register. Though I am not sure if this
>> has ever been tested out. If we change it, it will be across algorithms and
>> as a separate effort.
> 
> writel() will, at least on arm64, convert the CPU native value to little
> endian before writing it out, so I think the current setting make sense.

hmm.. ok.

> 
>>
>>>
>>> Perhaps I'm missing something here though?
>>>
>>> PS. Based on how the documentation is written, shouldn't you write out
>>> REG_CNTR_IV[012] as well?
>>
>> It is done on top, right ?
>> qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
>>
> 
> You're right, depending on enciv_words you write the 4 registers, then
> increment the last word and write that out again.
> 
>>>
>>>> +		qce_write_array(qce, REG_ENCR_CCM_INT_CNTR0, (u32 *)enciv, enciv_words);
>>>> +		qce_write(qce, REG_CNTR_MASK, ~0);
>>>> +		qce_write(qce, REG_CNTR_MASK0, ~0);
>>>> +		qce_write(qce, REG_CNTR_MASK1, ~0);
>>>> +		qce_write(qce, REG_CNTR_MASK2, ~0);
>>>> +	}
>>>> +
>>>> +	/* Clear authentication IV and KEY registers of previous values */
>>>> +	qce_clear_array(qce, REG_AUTH_IV0, 16);
>>>> +	qce_clear_array(qce, REG_AUTH_KEY0, 16);
>>>> +
>>>> +	/* Clear byte count */
>>>> +	qce_clear_array(qce, REG_AUTH_BYTECNT0, 4);
>>>> +
>>>> +	/* Write authentication key */
>>>> +	qce_cpu_to_be32p_array(authkey, ctx->auth_key, auth_keylen);
>>>> +	authkey_words = DIV_ROUND_UP(auth_keylen, sizeof(u32));
>>>> +	qce_write_array(qce, REG_AUTH_KEY0, (u32 *)authkey, authkey_words);
>>>> +
>>>> +	if (IS_SHA_HMAC(rctx->flags)) {
>>>> +		/* Write default authentication iv */
>>>> +		if (IS_SHA1_HMAC(rctx->flags)) {
>>>> +			auth_ivsize = SHA1_DIGEST_SIZE;
>>>> +			memcpy(authiv, std_iv_sha1, auth_ivsize);
>>>> +		} else if (IS_SHA256_HMAC(rctx->flags)) {
>>>> +			auth_ivsize = SHA256_DIGEST_SIZE;
>>>> +			memcpy(authiv, std_iv_sha256, auth_ivsize);
>>>> +		}
>>>> +		authiv_words = auth_ivsize / sizeof(u32);
>>>> +		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);
>>>
>>> AUTH_IV0 is affected by the little endian configuration, does this imply
>>> that IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags? If so
>>> I think it would be nice if you grouped the conditionals in a way that
>>> made that obvious when reading the function.
>>
>> So yes IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags.
>> AUTH_IVn is 0 for ccm and has initial value for HMAC algorithms. I don't
>> understand the confusion here.
>>
> 
> I'm just saying that writing is as below would have made it obvious to
> me that IS_SHA_HMAC() and IS_CCM() are exclusive:

So regardless of the mode, it is a good idea  to clear the IV  registers 
which happens above in

	qce_clear_array(qce, REG_AUTH_IV0, 16);


This is important becasue the size of IV varies between HMAC(SHA1) and 
HMAC(SHA256) and we don't want any previous bits sticking around.
For CCM after the above step we don't do anything with AUTH_IV where as 
for SHA_HMAC we have to go and program the registers. I can split it into
if (IS_SHA_HMAC(flags)) {
	...
} else {
	...
}

but both snippets will have the above line code clearing the IV register 
and the if part will have more stuff actually programming these 
registers.. Is that what you are looking for ?


-- 
Warm Regards
Thara

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Bjorn Andersson]

On Tue 13 Apr 17:44 CDT 2021, Thara Gopinath wrote:

> 
> 
> On 4/13/21 6:20 PM, Bjorn Andersson wrote:
> > On Tue 13 Apr 16:31 CDT 2021, Thara Gopinath wrote:
> > 
> > > 
> > > Hi Bjorn,
> > > 
> > > On 4/5/21 6:18 PM, Bjorn Andersson wrote:
> > > > On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> > > > 
> > > > > Add register programming sequence for enabling AEAD
> > > > > algorithms on the Qualcomm crypto engine.
> > > > > 
> > > > > Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> > > > > ---
> > > > >    drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
> > > > >    1 file changed, 153 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
> > > > > index 05a71c5ecf61..54d209cb0525 100644
> > > > > --- a/drivers/crypto/qce/common.c
> > > > > +++ b/drivers/crypto/qce/common.c
> > > > > @@ -15,6 +15,16 @@
> > > > >    #include "core.h"
> > > > >    #include "regs-v5.h"
> > > > >    #include "sha.h"
> > > > > +#include "aead.h"
> > > > > +
> > > > > +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> > > > > +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
> > > > > +};
> > > > > +
> > > > > +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> > > > > +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
> > > > > +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
> > > > > +};
> > > > >    static inline u32 qce_read(struct qce_device *qce, u32 offset)
> > > > >    {
> > > > > @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
> > > > >    		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
> > > > >    }
> > > > > -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> > > > > +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
> > > > >    static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
> > > > >    {
> > > > >    	u32 cfg = 0;
> > > > > @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
> > > > >    	return cfg;
> > > > >    }
> > > > > +#endif
> > > > > +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> > > > >    static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
> > > > >    {
> > > > >    	struct ahash_request *req = ahash_request_cast(async_req);
> > > > > @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
> > > > >    }
> > > > >    #endif
> > > > > -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> > > > > +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
> > > > >    static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
> > > > >    {
> > > > >    	u32 cfg = 0;
> > > > > @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
> > > > >    	return cfg;
> > > > >    }
> > > > > +#endif
> > > > > +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> > > > >    static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
> > > > >    {
> > > > >    	u8 swap[QCE_AES_IV_LENGTH];
> > > > > @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
> > > > >    }
> > > > >    #endif
> > > > > +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
> > > > > +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
> > > > > +{
> > > > > +	struct aead_request *req = aead_request_cast(async_req);
> > > > > +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
> > > > > +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
> > > > > +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
> > > > > +	struct qce_device *qce = tmpl->qce;
> > > > > +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
> > > > > +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
> > > > > +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
> > > > > +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
> > > > > +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
> > > > > +	unsigned int enc_keylen = ctx->enc_keylen;
> > > > > +	unsigned int auth_keylen = ctx->auth_keylen;
> > > > > +	unsigned int enc_ivsize = rctx->ivsize;
> > > > > +	unsigned int auth_ivsize;
> > > > > +	unsigned int enckey_words, enciv_words;
> > > > > +	unsigned int authkey_words, authiv_words, authnonce_words;
> > > > > +	unsigned long flags = rctx->flags;
> > > > > +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
> > > > 
> > > > I don't see any reason to initialize encr_cfg or auth_cfg.
> > > 
> > > right.. I will remove it
> > > 
> > > > 
> > > > > +	u32 *iv_last_word;
> > > > > +
> > > > > +	qce_setup_config(qce);
> > > > > +
> > > > > +	/* Write encryption key */
> > > > > +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
> > > > > +	enckey_words = enc_keylen / sizeof(u32);
> > > > > +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
> > > > 
> > > > Afaict all "array registers" in this function are affected by the
> > > > CRYPTO_SETUP little endian bit, but you set this bit before launching
> > > > the operation dependent on IS_CCM(). So is this really working for the
> > > > !IS_CCM() case?
> > > 
> > > I am not sure I understand you. Below ,
> > > 	/* get little endianness */
> > > 	config = qce_config_reg(qce, 1);
> > > 	qce_write(qce, REG_CONFIG, config);
> > > 
> > > is outside of any checks..
> > > 
> > 
> > You're right, I misread that snippet as I was jumping through the
> > function. So we're unconditionally running the hardware in little endian
> > mode.
> > 
> > > > 
> > > > > +
> > > > > +	/* Write encryption iv */
> > > > > +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
> > > > > +	enciv_words = enc_ivsize / sizeof(u32);
> > > > > +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
> > > > 
> > > > It would be nice if this snippet was extracted to a helper function.
> > > > 
> > > > > +
> > > > > +	if (IS_CCM(rctx->flags)) {
> > > > > +		iv_last_word = (u32 *)&enciv[enciv_words - 1];
> > > > > +//		qce_write(qce, REG_CNTR3_IV3, enciv[enciv_words - 1] + 1);
> > > > 
> > > > I believe this is a remnant of the two surrounding lines.
> > > 
> > > It indeed is.. I will remove it.
> > > 
> > > > 
> > > > > +		qce_write(qce, REG_CNTR3_IV3, (*iv_last_word) + 1);
> > > > 
> > > > enciv is an array of big endian 32-bit integers, which you tell the
> > > > compiler to treat as cpu-native endian, and then you do math on it.
> > > > Afaict from the documentation the value of REG_CNTR3_IVn should be set
> > > > to rctx->iv + 1, but if the hardware expects these in big endian then I
> > > > think you added 16777216.
> > > 
> > > So, the crypto engine documentation talks of writing to these registers in
> > > little endian mode. The byte stream that you get for iv from the user
> > > is in big endian mode as in the MSB is byte 0. So we kind of invert this and
> > > write  to these registers. This is what happens with declaring the __be32
> > > array and copying words to it from the byte stream. So now byte 0 is the LSB
> > > and a +1 will just add a 1 to it.
> > > 
> > 
> > But if the data come in big endian and after qce_cpu_to_be32p_array()
> > you're able to do math on them with expected result and you're finally
> > passing the data to writel() then I think that qce_cpu_to_be32p_array()
> > is actually be32_to_cpu() and after the conversion you should carry the
> > results in CPU-native u32 arrays - and thereby skip the typecasting.
> 
> you mean I can replace __be32 arrays with u32 arrays ?? yes I probably can.
> I will try this out and if it works do the change.
> 

Yes, given that you just typecast things as you do it should just work
to move the typecast to the qce_cpu_to_be32p_array().

But as I said, this would indicate that what is cpu_to_be32() should
have been be32_to_cpu() (both performs the same swap, it's just a matter
of what type goes in and what type goes out).

Looking at the other uses of qce_cpu_to_be32p_array() I suspect that
it's the same situation there, so perhaps introduce a new function
qce_be32_to_cpu() in this patchset (that returns number of words
converted) and then look into the existing users after that?

[..]
> > > > > +	if (IS_SHA_HMAC(rctx->flags)) {
> > > > > +		/* Write default authentication iv */
> > > > > +		if (IS_SHA1_HMAC(rctx->flags)) {
> > > > > +			auth_ivsize = SHA1_DIGEST_SIZE;
> > > > > +			memcpy(authiv, std_iv_sha1, auth_ivsize);
> > > > > +		} else if (IS_SHA256_HMAC(rctx->flags)) {
> > > > > +			auth_ivsize = SHA256_DIGEST_SIZE;
> > > > > +			memcpy(authiv, std_iv_sha256, auth_ivsize);
> > > > > +		}
> > > > > +		authiv_words = auth_ivsize / sizeof(u32);
> > > > > +		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);
> > > > 
> > > > AUTH_IV0 is affected by the little endian configuration, does this imply
> > > > that IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags? If so
> > > > I think it would be nice if you grouped the conditionals in a way that
> > > > made that obvious when reading the function.
> > > 
> > > So yes IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags.
> > > AUTH_IVn is 0 for ccm and has initial value for HMAC algorithms. I don't
> > > understand the confusion here.
> > > 
> > 
> > I'm just saying that writing is as below would have made it obvious to
> > me that IS_SHA_HMAC() and IS_CCM() are exclusive:
> 
> So regardless of the mode, it is a good idea  to clear the IV  registers
> which happens above in
> 
> 	qce_clear_array(qce, REG_AUTH_IV0, 16);
> 
> 
> This is important becasue the size of IV varies between HMAC(SHA1) and
> HMAC(SHA256) and we don't want any previous bits sticking around.
> For CCM after the above step we don't do anything with AUTH_IV where as for
> SHA_HMAC we have to go and program the registers. I can split it into
> if (IS_SHA_HMAC(flags)) {
> 	...
> } else {
> 	...
> }
> 
> but both snippets will have the above line code clearing the IV register and
> the if part will have more stuff actually programming these registers.. Is
> that what you are looking for ?

I didn't find an answer quickly to the question if the two where
mutually exclusive and couldn't determine from the code flow either. But
my comment seems to stem from my misunderstanding that the little endian
bit was dependent on IS_CCM().

That said, if the logic really is "do this for IS_SHA_HMAC(), otherwise
do that", then if else makes sense.

Regards,
Bjorn

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Thara Gopinath]

On 4/13/21 7:09 PM, Bjorn Andersson wrote:
> On Tue 13 Apr 17:44 CDT 2021, Thara Gopinath wrote:

[..]

> 
> Yes, given that you just typecast things as you do it should just work
> to move the typecast to the qce_cpu_to_be32p_array().
> 
> But as I said, this would indicate that what is cpu_to_be32() should
> have been be32_to_cpu() (both performs the same swap, it's just a matter
> of what type goes in and what type goes out).
> 
> Looking at the other uses of qce_cpu_to_be32p_array() I suspect that
> it's the same situation there, so perhaps introduce a new function
> qce_be32_to_cpu() in this patchset (that returns number of words
> converted) and then look into the existing users after that?

Hi!

I have sent out the v2 with the new function. To me, it does look 
cleaner. So thanks!

> 
> [..]
>>>>>> +	if (IS_SHA_HMAC(rctx->flags)) {
>>>>>> +		/* Write default authentication iv */
>>>>>> +		if (IS_SHA1_HMAC(rctx->flags)) {
>>>>>> +			auth_ivsize = SHA1_DIGEST_SIZE;
>>>>>> +			memcpy(authiv, std_iv_sha1, auth_ivsize);
>>>>>> +		} else if (IS_SHA256_HMAC(rctx->flags)) {
>>>>>> +			auth_ivsize = SHA256_DIGEST_SIZE;
>>>>>> +			memcpy(authiv, std_iv_sha256, auth_ivsize);
>>>>>> +		}
>>>>>> +		authiv_words = auth_ivsize / sizeof(u32);
>>>>>> +		qce_write_array(qce, REG_AUTH_IV0, (u32 *)authiv, authiv_words);
>>>>>
>>>>> AUTH_IV0 is affected by the little endian configuration, does this imply
>>>>> that IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags? If so
>>>>> I think it would be nice if you grouped the conditionals in a way that
>>>>> made that obvious when reading the function.
>>>>
>>>> So yes IS_SHA_HMAC() and IS_CCM() are exclusive bits of rctx->flags.
>>>> AUTH_IVn is 0 for ccm and has initial value for HMAC algorithms. I don't
>>>> understand the confusion here.
>>>>
>>>
>>> I'm just saying that writing is as below would have made it obvious to
>>> me that IS_SHA_HMAC() and IS_CCM() are exclusive:
>>
>> So regardless of the mode, it is a good idea  to clear the IV  registers
>> which happens above in
>>
>> 	qce_clear_array(qce, REG_AUTH_IV0, 16);
>>
>>
>> This is important becasue the size of IV varies between HMAC(SHA1) and
>> HMAC(SHA256) and we don't want any previous bits sticking around.
>> For CCM after the above step we don't do anything with AUTH_IV where as for
>> SHA_HMAC we have to go and program the registers. I can split it into
>> if (IS_SHA_HMAC(flags)) {
>> 	...
>> } else {
>> 	...
>> }
>>
>> but both snippets will have the above line code clearing the IV register and
>> the if part will have more stuff actually programming these registers.. Is
>> that what you are looking for ?
> 
> I didn't find an answer quickly to the question if the two where
> mutually exclusive and couldn't determine from the code flow either. But
> my comment seems to stem from my misunderstanding that the little endian
> bit was dependent on IS_CCM().
> 
> That said, if the logic really is "do this for IS_SHA_HMAC(), otherwise
> do that", then if else makes sense.

So, the logic is really. do "clearing of IV" in all cases. Do setting of 
initial IV only for HMAC. I tried the if..else like you said. It did not 
look correct to duplicate the code clearing the IV. So I have added 
comments around this section hopefully making it clearer. Do take a look 
and let me know. I can rework it further if you think so.

> 
> Regards,
> Bjorn
> 

-- 
Warm Regards
Thara

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Thara Gopinath]

On 4/5/21 6:18 PM, Bjorn Andersson wrote:
> On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> 
>> Add register programming sequence for enabling AEAD
>> algorithms on the Qualcomm crypto engine.
>>
>> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
>> ---
>>   drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
>>   1 file changed, 153 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
>> index 05a71c5ecf61..54d209cb0525 100644
>> --- a/drivers/crypto/qce/common.c
>> +++ b/drivers/crypto/qce/common.c
>> @@ -15,6 +15,16 @@
>>   #include "core.h"
>>   #include "regs-v5.h"
>>   #include "sha.h"
>> +#include "aead.h"
>> +
>> +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>> +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
>> +};
>> +
>> +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>> +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
>> +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
>> +};
>>   
>>   static inline u32 qce_read(struct qce_device *qce, u32 offset)
>>   {
>> @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
>>   		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
>>   }
>>   
>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>   static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>   {
>>   	u32 cfg = 0;
>> @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>   
>>   	return cfg;
>>   }
>> +#endif
>>   
>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>>   static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>   {
>>   	struct ahash_request *req = ahash_request_cast(async_req);
>> @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>   }
>>   #endif
>>   
>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>   static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>   {
>>   	u32 cfg = 0;
>> @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>   
>>   	return cfg;
>>   }
>> +#endif
>>   
>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>>   static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
>>   {
>>   	u8 swap[QCE_AES_IV_LENGTH];
>> @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
>>   }
>>   #endif
>>   
>> +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
>> +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
>> +{
>> +	struct aead_request *req = aead_request_cast(async_req);
>> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
>> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
>> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
>> +	struct qce_device *qce = tmpl->qce;
>> +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
>> +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
>> +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
>> +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
>> +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
>> +	unsigned int enc_keylen = ctx->enc_keylen;
>> +	unsigned int auth_keylen = ctx->auth_keylen;
>> +	unsigned int enc_ivsize = rctx->ivsize;
>> +	unsigned int auth_ivsize;
>> +	unsigned int enckey_words, enciv_words;
>> +	unsigned int authkey_words, authiv_words, authnonce_words;
>> +	unsigned long flags = rctx->flags;
>> +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
> 
> I don't see any reason to initialize encr_cfg or auth_cfg.
> 
>> +	u32 *iv_last_word;
>> +
>> +	qce_setup_config(qce);
>> +
>> +	/* Write encryption key */
>> +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
>> +	enckey_words = enc_keylen / sizeof(u32);
>> +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
> 
> Afaict all "array registers" in this function are affected by the
> CRYPTO_SETUP little endian bit, but you set this bit before launching
> the operation dependent on IS_CCM(). So is this really working for the
> !IS_CCM() case?
> 
>> +
>> +	/* Write encryption iv */
>> +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
>> +	enciv_words = enc_ivsize / sizeof(u32);
>> +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
> 
> It would be nice if this snippet was extracted to a helper function.

I kind of forgot to type this earlier. So yes I agree in principle.
It is more elegant to have something like qce_convert_be32_and_write
and in the function do the above three steps. This snippet is prevalent 
in this driver code across other alogs as well (skcipher and hash).
Take it up as a separate clean up activity  ?

-- 
Warm Regards
Thara

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Bjorn Andersson]

On Tue 13 Apr 17:27 CDT 2021, Thara Gopinath wrote:

> 
> 
> On 4/5/21 6:18 PM, Bjorn Andersson wrote:
> > On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
> > 
> > > Add register programming sequence for enabling AEAD
> > > algorithms on the Qualcomm crypto engine.
> > > 
> > > Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
> > > ---
> > >   drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
> > >   1 file changed, 153 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
> > > index 05a71c5ecf61..54d209cb0525 100644
> > > --- a/drivers/crypto/qce/common.c
> > > +++ b/drivers/crypto/qce/common.c
> > > @@ -15,6 +15,16 @@
> > >   #include "core.h"
> > >   #include "regs-v5.h"
> > >   #include "sha.h"
> > > +#include "aead.h"
> > > +
> > > +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> > > +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
> > > +};
> > > +
> > > +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
> > > +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
> > > +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
> > > +};
> > >   static inline u32 qce_read(struct qce_device *qce, u32 offset)
> > >   {
> > > @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
> > >   		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
> > >   }
> > > -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> > > +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
> > >   static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
> > >   {
> > >   	u32 cfg = 0;
> > > @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
> > >   	return cfg;
> > >   }
> > > +#endif
> > > +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
> > >   static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
> > >   {
> > >   	struct ahash_request *req = ahash_request_cast(async_req);
> > > @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
> > >   }
> > >   #endif
> > > -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> > > +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
> > >   static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
> > >   {
> > >   	u32 cfg = 0;
> > > @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
> > >   	return cfg;
> > >   }
> > > +#endif
> > > +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
> > >   static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
> > >   {
> > >   	u8 swap[QCE_AES_IV_LENGTH];
> > > @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
> > >   }
> > >   #endif
> > > +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
> > > +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
> > > +{
> > > +	struct aead_request *req = aead_request_cast(async_req);
> > > +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
> > > +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
> > > +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
> > > +	struct qce_device *qce = tmpl->qce;
> > > +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
> > > +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
> > > +	unsigned int enc_keylen = ctx->enc_keylen;
> > > +	unsigned int auth_keylen = ctx->auth_keylen;
> > > +	unsigned int enc_ivsize = rctx->ivsize;
> > > +	unsigned int auth_ivsize;
> > > +	unsigned int enckey_words, enciv_words;
> > > +	unsigned int authkey_words, authiv_words, authnonce_words;
> > > +	unsigned long flags = rctx->flags;
> > > +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
> > 
> > I don't see any reason to initialize encr_cfg or auth_cfg.
> > 
> > > +	u32 *iv_last_word;
> > > +
> > > +	qce_setup_config(qce);
> > > +
> > > +	/* Write encryption key */
> > > +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
> > > +	enckey_words = enc_keylen / sizeof(u32);
> > > +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
> > 
> > Afaict all "array registers" in this function are affected by the
> > CRYPTO_SETUP little endian bit, but you set this bit before launching
> > the operation dependent on IS_CCM(). So is this really working for the
> > !IS_CCM() case?
> > 
> > > +
> > > +	/* Write encryption iv */
> > > +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
> > > +	enciv_words = enc_ivsize / sizeof(u32);
> > > +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
> > 
> > It would be nice if this snippet was extracted to a helper function.
> 
> I kind of forgot to type this earlier. So yes I agree in principle.
> It is more elegant to have something like qce_convert_be32_and_write
> and in the function do the above three steps. This snippet is prevalent in
> this driver code across other alogs as well (skcipher and hash).

Perhaps make qce_cpu_to_be32p_array() (or qce_be32_to_cpu() per the
other reply), could return the number of words written - that way you
remove the ugly middle line at least - and can still poke at the data
when needed.

> Take it up as a separate clean up activity  ?
> 

Yes, that seems like a good idea.

Regards,
Bjorn

Re: [PATCH 6/7] crypto: qce: common: Add support for AEAD algorithms

[Thara Gopinath]

On 4/13/21 6:33 PM, Bjorn Andersson wrote:
> On Tue 13 Apr 17:27 CDT 2021, Thara Gopinath wrote:
> 
>>
>>
>> On 4/5/21 6:18 PM, Bjorn Andersson wrote:
>>> On Thu 25 Feb 12:27 CST 2021, Thara Gopinath wrote:
>>>
>>>> Add register programming sequence for enabling AEAD
>>>> algorithms on the Qualcomm crypto engine.
>>>>
>>>> Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
>>>> ---
>>>>    drivers/crypto/qce/common.c | 155 +++++++++++++++++++++++++++++++++++-
>>>>    1 file changed, 153 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
>>>> index 05a71c5ecf61..54d209cb0525 100644
>>>> --- a/drivers/crypto/qce/common.c
>>>> +++ b/drivers/crypto/qce/common.c
>>>> @@ -15,6 +15,16 @@
>>>>    #include "core.h"
>>>>    #include "regs-v5.h"
>>>>    #include "sha.h"
>>>> +#include "aead.h"
>>>> +
>>>> +static const u32 std_iv_sha1[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>>>> +	SHA1_H0, SHA1_H1, SHA1_H2, SHA1_H3, SHA1_H4, 0, 0, 0
>>>> +};
>>>> +
>>>> +static const u32 std_iv_sha256[SHA256_DIGEST_SIZE / sizeof(u32)] = {
>>>> +	SHA256_H0, SHA256_H1, SHA256_H2, SHA256_H3,
>>>> +	SHA256_H4, SHA256_H5, SHA256_H6, SHA256_H7
>>>> +};
>>>>    static inline u32 qce_read(struct qce_device *qce, u32 offset)
>>>>    {
>>>> @@ -96,7 +106,7 @@ static inline void qce_crypto_go(struct qce_device *qce, bool result_dump)
>>>>    		qce_write(qce, REG_GOPROC, BIT(GO_SHIFT));
>>>>    }
>>>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>>>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SHA) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>>>    static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>>>    {
>>>>    	u32 cfg = 0;
>>>> @@ -139,7 +149,9 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size, u32 auth_size)
>>>>    	return cfg;
>>>>    }
>>>> +#endif
>>>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
>>>>    static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>>>    {
>>>>    	struct ahash_request *req = ahash_request_cast(async_req);
>>>> @@ -225,7 +237,7 @@ static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
>>>>    }
>>>>    #endif
>>>> -#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>>>> +#if defined(CONFIG_CRYPTO_DEV_QCE_SKCIPHER) || defined(CONFIG_CRYPTO_DEV_QCE_AEAD)
>>>>    static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>>>    {
>>>>    	u32 cfg = 0;
>>>> @@ -271,7 +283,9 @@ static u32 qce_encr_cfg(unsigned long flags, u32 aes_key_size)
>>>>    	return cfg;
>>>>    }
>>>> +#endif
>>>> +#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
>>>>    static void qce_xts_swapiv(__be32 *dst, const u8 *src, unsigned int ivsize)
>>>>    {
>>>>    	u8 swap[QCE_AES_IV_LENGTH];
>>>> @@ -386,6 +400,139 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
>>>>    }
>>>>    #endif
>>>> +#ifdef CONFIG_CRYPTO_DEV_QCE_AEAD
>>>> +static int qce_setup_regs_aead(struct crypto_async_request *async_req)
>>>> +{
>>>> +	struct aead_request *req = aead_request_cast(async_req);
>>>> +	struct qce_aead_reqctx *rctx = aead_request_ctx(req);
>>>> +	struct qce_aead_ctx *ctx = crypto_tfm_ctx(async_req->tfm);
>>>> +	struct qce_alg_template *tmpl = to_aead_tmpl(crypto_aead_reqtfm(req));
>>>> +	struct qce_device *qce = tmpl->qce;
>>>> +	__be32 enckey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 enciv[QCE_MAX_IV_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 authkey[QCE_SHA_HMAC_KEY_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 authiv[SHA256_DIGEST_SIZE / sizeof(__be32)] = {0};
>>>> +	__be32 authnonce[QCE_MAX_NONCE / sizeof(__be32)] = {0};
>>>> +	unsigned int enc_keylen = ctx->enc_keylen;
>>>> +	unsigned int auth_keylen = ctx->auth_keylen;
>>>> +	unsigned int enc_ivsize = rctx->ivsize;
>>>> +	unsigned int auth_ivsize;
>>>> +	unsigned int enckey_words, enciv_words;
>>>> +	unsigned int authkey_words, authiv_words, authnonce_words;
>>>> +	unsigned long flags = rctx->flags;
>>>> +	u32 encr_cfg = 0, auth_cfg = 0, config, totallen;
>>>
>>> I don't see any reason to initialize encr_cfg or auth_cfg.
>>>
>>>> +	u32 *iv_last_word;
>>>> +
>>>> +	qce_setup_config(qce);
>>>> +
>>>> +	/* Write encryption key */
>>>> +	qce_cpu_to_be32p_array(enckey, ctx->enc_key, enc_keylen);
>>>> +	enckey_words = enc_keylen / sizeof(u32);
>>>> +	qce_write_array(qce, REG_ENCR_KEY0, (u32 *)enckey, enckey_words);
>>>
>>> Afaict all "array registers" in this function are affected by the
>>> CRYPTO_SETUP little endian bit, but you set this bit before launching
>>> the operation dependent on IS_CCM(). So is this really working for the
>>> !IS_CCM() case?
>>>
>>>> +
>>>> +	/* Write encryption iv */
>>>> +	qce_cpu_to_be32p_array(enciv, rctx->iv, enc_ivsize);
>>>> +	enciv_words = enc_ivsize / sizeof(u32);
>>>> +	qce_write_array(qce, REG_CNTR0_IV0, (u32 *)enciv, enciv_words);
>>>
>>> It would be nice if this snippet was extracted to a helper function.
>>
>> I kind of forgot to type this earlier. So yes I agree in principle.
>> It is more elegant to have something like qce_convert_be32_and_write
>> and in the function do the above three steps. This snippet is prevalent in
>> this driver code across other alogs as well (skcipher and hash).
> 
> Perhaps make qce_cpu_to_be32p_array() (or qce_be32_to_cpu() per the
> other reply), could return the number of words written - that way you
> remove the ugly middle line at least - and can still poke at the data
> when needed.

Right. I will note this down for clean up as it touches other stuff as 
well.

> 
>> Take it up as a separate clean up activity  ?
>>
> 
> Yes, that seems like a good idea.
> 
> Regards,
> Bjorn
> 

-- 
Warm Regards
Thara

[PATCH 7/7] crypto: qce: aead: Schedule fallback algorithm

[Thara Gopinath]

Qualcomm crypto engine does not handle the following scenarios and
will issue an abort. In such cases, pass on the transformation to
a fallback algorithm.

- DES3 algorithms with all three keys same.
- AES192 algorithms.
- 0 length messages.

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
---
 drivers/crypto/qce/aead.c | 58 ++++++++++++++++++++++++++++++++-------
 drivers/crypto/qce/aead.h |  3 ++
 2 files changed, 51 insertions(+), 10 deletions(-)

diff --git a/drivers/crypto/qce/aead.c b/drivers/crypto/qce/aead.c
index b594c4bb2640..4c2d024e5296 100644
--- a/drivers/crypto/qce/aead.c
+++ b/drivers/crypto/qce/aead.c
@@ -492,7 +492,20 @@ static int qce_aead_crypt(struct aead_request *req, int encrypt)
 	/* CE does not handle 0 length messages */
 	if (!rctx->cryptlen) {
 		if (!(IS_CCM(rctx->flags) && IS_DECRYPT(rctx->flags)))
-			return -EINVAL;
+			ctx->need_fallback = true;
+	}
+
+	/* If fallback is needed, schedule and exit */
+	if (ctx->need_fallback) {
+		aead_request_set_tfm(&rctx->fallback_req, ctx->fallback);
+		aead_request_set_callback(&rctx->fallback_req, req->base.flags,
+					  req->base.complete, req->base.data);
+		aead_request_set_crypt(&rctx->fallback_req, req->src,
+				       req->dst, req->cryptlen, req->iv);
+		aead_request_set_ad(&rctx->fallback_req, req->assoclen);
+
+		return encrypt ? crypto_aead_encrypt(&rctx->fallback_req) :
+				 crypto_aead_decrypt(&rctx->fallback_req);
 	}
 
 	/*
@@ -533,7 +546,7 @@ static int qce_aead_ccm_setkey(struct crypto_aead *tfm, const u8 *key,
 		memcpy(ctx->ccm4309_salt, key + keylen, QCE_CCM4309_SALT_SIZE);
 	}
 
-	if (keylen != AES_KEYSIZE_128 && keylen != AES_KEYSIZE_256)
+	if (keylen != AES_KEYSIZE_128 && keylen != AES_KEYSIZE_256 && keylen != AES_KEYSIZE_192)
 		return -EINVAL;
 
 	ctx->enc_keylen = keylen;
@@ -542,7 +555,12 @@ static int qce_aead_ccm_setkey(struct crypto_aead *tfm, const u8 *key,
 	memcpy(ctx->enc_key, key, keylen);
 	memcpy(ctx->auth_key, key, keylen);
 
-	return 0;
+	if (keylen == AES_KEYSIZE_192)
+		ctx->need_fallback = true;
+
+	return IS_CCM_RFC4309(flags) ?
+		crypto_aead_setkey(ctx->fallback, key, keylen + QCE_CCM4309_SALT_SIZE) :
+		crypto_aead_setkey(ctx->fallback, key, keylen);
 }
 
 static int qce_aead_setkey(struct crypto_aead *tfm, const u8 *key, unsigned int keylen)
@@ -573,20 +591,21 @@ static int qce_aead_setkey(struct crypto_aead *tfm, const u8 *key, unsigned int
 		 * The crypto engine does not support any two keys
 		 * being the same for triple des algorithms. The
 		 * verify_skcipher_des3_key does not check for all the
-		 * below conditions. Return -EINVAL in case any two keys
-		 * are the same. Revisit to see if a fallback cipher
-		 * is needed to handle this condition.
+		 * below conditions. Schedule fallback in this case.
 		 */
 		memcpy(_key, authenc_keys.enckey, DES3_EDE_KEY_SIZE);
 		if (!((_key[0] ^ _key[2]) | (_key[1] ^ _key[3])) ||
 		    !((_key[2] ^ _key[4]) | (_key[3] ^ _key[5])) ||
 		    !((_key[0] ^ _key[4]) | (_key[1] ^ _key[5])))
-			return -EINVAL;
+			ctx->need_fallback = true;
 	} else if (IS_AES(flags)) {
 		/* No random key sizes */
 		if (authenc_keys.enckeylen != AES_KEYSIZE_128 &&
+		    authenc_keys.enckeylen != AES_KEYSIZE_192 &&
 		    authenc_keys.enckeylen != AES_KEYSIZE_256)
 			return -EINVAL;
+		if (authenc_keys.enckeylen == AES_KEYSIZE_192)
+			ctx->need_fallback = true;
 	}
 
 	ctx->enc_keylen = authenc_keys.enckeylen;
@@ -597,7 +616,7 @@ static int qce_aead_setkey(struct crypto_aead *tfm, const u8 *key, unsigned int
 	memset(ctx->auth_key, 0, sizeof(ctx->auth_key));
 	memcpy(ctx->auth_key, authenc_keys.authkey, authenc_keys.authkeylen);
 
-	return 0;
+	return crypto_aead_setkey(ctx->fallback, key, keylen);
 }
 
 static int qce_aead_setauthsize(struct crypto_aead *tfm, unsigned int authsize)
@@ -612,15 +631,32 @@ static int qce_aead_setauthsize(struct crypto_aead *tfm, unsigned int authsize)
 			return -EINVAL;
 	}
 	ctx->authsize = authsize;
-	return 0;
+
+	return crypto_aead_setauthsize(ctx->fallback, authsize);
 }
 
 static int qce_aead_init(struct crypto_aead *tfm)
 {
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+
+	ctx->need_fallback = false;
+	ctx->fallback = crypto_alloc_aead(crypto_tfm_alg_name(&tfm->base),
+					  0, CRYPTO_ALG_NEED_FALLBACK);
+
+	if (IS_ERR(ctx->fallback))
+		return PTR_ERR(ctx->fallback);
+
 	crypto_aead_set_reqsize(tfm, sizeof(struct qce_aead_reqctx));
 	return 0;
 }
 
+static void qce_aead_exit(struct crypto_aead *tfm)
+{
+	struct qce_aead_ctx *ctx = crypto_aead_ctx(tfm);
+
+	crypto_free_aead(ctx->fallback);
+}
+
 struct qce_aead_def {
 	unsigned long flags;
 	const char *name;
@@ -718,11 +754,13 @@ static int qce_aead_register_one(const struct qce_aead_def *def, struct qce_devi
 	alg->encrypt			= qce_aead_encrypt;
 	alg->decrypt			= qce_aead_decrypt;
 	alg->init			= qce_aead_init;
+	alg->exit			= qce_aead_exit;
 
 	alg->base.cra_priority		= 300;
 	alg->base.cra_flags		= CRYPTO_ALG_ASYNC |
 					  CRYPTO_ALG_ALLOCATES_MEMORY |
-					  CRYPTO_ALG_KERN_DRIVER_ONLY;
+					  CRYPTO_ALG_KERN_DRIVER_ONLY |
+					  CRYPTO_ALG_NEED_FALLBACK;
 	alg->base.cra_ctxsize		= sizeof(struct qce_aead_ctx);
 	alg->base.cra_alignmask		= 0;
 	alg->base.cra_module		= THIS_MODULE;
diff --git a/drivers/crypto/qce/aead.h b/drivers/crypto/qce/aead.h
index 3d1f2039930b..efb8477cc088 100644
--- a/drivers/crypto/qce/aead.h
+++ b/drivers/crypto/qce/aead.h
@@ -19,6 +19,8 @@ struct qce_aead_ctx {
 	unsigned int enc_keylen;
 	unsigned int auth_keylen;
 	unsigned int authsize;
+	bool need_fallback;
+	struct crypto_aead *fallback;
 };
 
 struct qce_aead_reqctx {
@@ -39,6 +41,7 @@ struct qce_aead_reqctx {
 	u8 ccm_nonce[QCE_MAX_NONCE];
 	u8 ccmresult_buf[QCE_BAM_BURST_SIZE];
 	u8 ccm_rfc4309_iv[QCE_MAX_IV_SIZE];
+	struct aead_request fallback_req;
 };
 
 static inline struct qce_alg_template *to_aead_tmpl(struct crypto_aead *tfm)
-- 
2.25.1

Re: [PATCH 0/7] Add support for AEAD algorithms in Qualcomm Crypto Engine driver

[Herbert Xu]

On Thu, Feb 25, 2021 at 01:27:09PM -0500, Thara Gopinath wrote:
> Enable support for AEAD algorithms in Qualcomm CE driver.  The first three
> patches in this series are cleanups and add a few missing pieces required
> to add support for AEAD algorithms.  Patch 4 introduces supported AEAD
> transformations on Qualcomm CE.  Patches 5 and 6 implements the h/w
> infrastructure needed to enable and run the AEAD transformations on
> Qualcomm CE.  Patch 7 adds support to queue fallback algorithms in case of
> unsupported special inputs.
> 
> This series is dependant on https://lkml.org/lkml/2021/2/11/1052.

Did this patch series pass the fuzz tests?

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH 0/7] Add support for AEAD algorithms in Qualcomm Crypto Engine driver

[Thara Gopinath]

On Thu, 4 Mar 2021 at 00:30, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Thu, Feb 25, 2021 at 01:27:09PM -0500, Thara Gopinath wrote:
> > Enable support for AEAD algorithms in Qualcomm CE driver.  The first three
> > patches in this series are cleanups and add a few missing pieces required
> > to add support for AEAD algorithms.  Patch 4 introduces supported AEAD
> > transformations on Qualcomm CE.  Patches 5 and 6 implements the h/w
> > infrastructure needed to enable and run the AEAD transformations on
> > Qualcomm CE.  Patch 7 adds support to queue fallback algorithms in case of
> > unsupported special inputs.
> >
> > This series is dependant on https://lkml.org/lkml/2021/2/11/1052.
>
> Did this patch series pass the fuzz tests?

Hi Herbert,

Yes it did. The last patch adds fallback for unsupported cases and
this will make it pass the fuzz tests.

>
> Thanks,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt



-- 
Warm Regards
Thara

Re: [PATCH 0/7] Add support for AEAD algorithms in Qualcomm Crypto Engine driver

[Herbert Xu]

On Thu, Mar 04, 2021 at 01:41:15PM -0500, Thara Gopinath wrote:
>
> Yes it did. The last patch adds fallback for unsupported cases and
> this will make it pass the fuzz tests.

Please include this information in the next round.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH 0/7] Add support for AEAD algorithms in Qualcomm Crypto Engine driver

[Thara Gopinath]

On 3/12/21 8:02 AM, Herbert Xu wrote:
> On Thu, Mar 04, 2021 at 01:41:15PM -0500, Thara Gopinath wrote:
>>
>> Yes it did. The last patch adds fallback for unsupported cases and
>> this will make it pass the fuzz tests.
> 
> Please include this information in the next round.

I will. Thanks!
> 
> Thanks,
> 

-- 
Warm Regards
Thara