crypto apis in cifs module allocating storage for character array during run-time vs. dynamic allocation

crypto apis in cifs module allocating storage for character array during run-time vs. dynamic allocation

[Shirish Pargaonkar]

Instead of determining and allocating a char array for use during usage of
crypto_shash_* calls, would like to instead dynamically
allocate (and free) storage for the duration of crypto calculation
(crypto_shash_init,
crypto_shash_update, and crypto_shash_final)
But everytime I try, it results in some sort of oops in the cifs module.
The motivation is to avoid sparse warnings like this

  CHECK   fs/cifs/cifsencrypt.c
fs/cifs/cifsencrypt.c:51:33: error: bad constant expression
fs/cifs/cifsencrypt.c:121:33: error: bad constant expression
fs/cifs/cifsencrypt.c:318:33: error: bad constant expression
fs/cifs/cifsencrypt.c:447:33: error: bad constant expression
fs/cifs/cifsencrypt.c:485:33: error: bad constant expression

that are generated because the size is ctx array is undetermined at
compile time.



+struct sdesc {
+       struct shash_desc shash;
+       char *ctx;
+};


@@ -46,21 +46,25 @@ static int cifs_calculate_signature(const struct
smb_hdr *cifs_pdu,
                        struct TCP_Server_Info *server, char *signature)
 {
        int rc = 0;
-       struct {
-               struct shash_desc shash;
-               char ctx[crypto_shash_descsize(server->ntlmssp.md5)];
-       } sdesc;
+       struct sdesc sdesc;

        if (cifs_pdu == NULL || server == NULL || signature == NULL)
                return -EINVAL;

+       sdesc.ctx = kmalloc(crypto_shash_descsize(server->ntlmssp.md5),
+                                                               GFP_KERNEL);
+       if (!sdesc.ctx) {
+               cERROR(1, "cifs_calculate_signature: could not
initialize crypto hmacmd5\n");
+               return -ENOMEM;
+       }
+
        sdesc.shash.tfm = server->ntlmssp.md5;
        sdesc.shash.flags = 0x0;

        rc = crypto_shash_init(&sdesc.shash);
        if (rc) {
                cERROR(1, "could not initialize master crypto API hmacmd5\n");
-               return rc;
+               goto calc_sig_ret;
        }

Re: crypto apis in cifs module allocating storage for character array during run-time vs. dynamic allocation

[Miloslav Trmac]

----- "Shirish Pargaonkar" <shirishpargaonkar-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Instead of determining and allocating a char array for use during usage of
> crypto_shash_* calls, would like to instead dynamically
> allocate (and free) storage for the duration of crypto calculation
> (crypto_shash_init,
> crypto_shash_update, and crypto_shash_final)
> But everytime I try, it results in some sort of oops in the cifs module.
Let me just suggest something, without trying it...

> +struct sdesc {
> +       struct shash_desc shash;
> +       char *ctx;
        char ctx[];
would be correct here.
> +};
And you need to allocate both shash_desc and "ctx" together as a single piece of memory - exactly mirror the memory layout of the original "sdesc" variable.
    Mirek

Re: crypto apis in cifs module allocating storage for character array during run-time vs. dynamic allocation

[Shirish Pargaonkar]

On Mon, Aug 23, 2010 at 5:39 PM, Miloslav Trmac <mitr@redhat.com> wrote:
> ----- "Shirish Pargaonkar" <shirishpargaonkar@gmail.com> wrote:
>> Instead of determining and allocating a char array for use during usage of
>> crypto_shash_* calls, would like to instead dynamically
>> allocate (and free) storage for the duration of crypto calculation
>> (crypto_shash_init,
>> crypto_shash_update, and crypto_shash_final)
>> But everytime I try, it results in some sort of oops in the cifs module.
> Let me just suggest something, without trying it...
>
>> +struct sdesc {
>> +       struct shash_desc shash;
>> +       char *ctx;
>        char ctx[];
> would be correct here.
>> +};
> And you need to allocate both shash_desc and "ctx" together as a single piece of memory - exactly mirror the memory layout of the original "sdesc" variable.
>    Mirek
>

Mirek,

Thanks, that worked.