How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

[Coiby Xu]

Hi,

Recently, I notice cryptsetup itself consumes significant amount of
memory (~256M) when estimating the memory requirement for dumping vmcore
to a LUKS-encrypted disk,

$ time -v cryptsetup luksOpen encrypted.img volume --key-file mykey.keyfile | grep "Maximum resident set size"
         Maximum resident set size (kbytes): 1309828
$ cryptsetup luksDump encrypted.img      
...
Keyslots:
   0: luks2
         PBKDF:      argon2id
         Memory:     1048576
         ...


So is there a way to estimate the upper bound of the peak memory
consumption of cryptsetup itself without running cryptsetup? 

Thanks!

-- 
Best regards,
Coiby

Re: How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

[Milan Broz]

On 16/06/2022 06:43, Coiby Xu wrote:
> Hi,
> 
> Recently, I notice cryptsetup itself consumes significant amount of
> memory (~256M) when estimating the memory requirement for dumping vmcore
> to a LUKS-encrypted disk,
> 
> $ time -v cryptsetup luksOpen encrypted.img volume --key-file mykey.keyfile | grep "Maximum resident set size"
>           Maximum resident set size (kbytes): 1309828
> $ cryptsetup luksDump encrypted.img
> ...
> Keyslots:
>     0: luks2
>           PBKDF:      argon2id
>           Memory:     1048576
>           ...
> 
> 
> So is there a way to estimate the upper bound of the peak memory
> consumption of cryptsetup itself without running cryptsetup?

As you already found, the major memory consumption is by memory-hard KDF.
But this memory is used only while calculating keyslot encryption key,
it is released immediately after the Argon call is finished.
I do not think we have better estimation here.

(Another story is locking all memory, including big areas used by libc,
but that should not be problem here, I hope.)

Milan

Re: How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

[Coiby Xu]

On Sat, Jun 18, 2022 at 05:12:56PM +0200, Milan Broz wrote:
>On 16/06/2022 06:43, Coiby Xu wrote:
>>Hi,
>>
>>Recently, I notice cryptsetup itself consumes significant amount of
>>memory (~256M) when estimating the memory requirement for dumping vmcore
>>to a LUKS-encrypted disk,
>>
>>$ time -v cryptsetup luksOpen encrypted.img volume --key-file mykey.keyfile | grep "Maximum resident set size"
>>          Maximum resident set size (kbytes): 1309828
>>$ cryptsetup luksDump encrypted.img
>>...
>>Keyslots:
>>    0: luks2
>>          PBKDF:      argon2id
>>          Memory:     1048576
>>          ...
>>
>>
>>So is there a way to estimate the upper bound of the peak memory
>>consumption of cryptsetup itself without running cryptsetup?
>
>As you already found, the major memory consumption is by memory-hard KDF.
>But this memory is used only while calculating keyslot encryption key,
>it is released immediately after the Argon call is finished.
>I do not think we have better estimation here.

Thanks for the reply! Sorry I meant the way to estimate the overhead of
crypsetup itself i.e. ~256M in the above example. Previously I only take
the memory consumption by memory-hard KDF into consideration and
neglected the memory consumption of cryptsetup itself. This obviously
leads to an underestimation of the memory requirement of cryptsetup. I
need to overestimate the memory requirement a bit to make sure OOM won't
happen that's why I am asking if there is a way to estimate the
upper bound of memory requirement of cryptsetup itself. 

>
>(Another story is locking all memory, including big areas used by libc,
>but that should not be problem here, I hope.)

Do you mean locking all memory first in order to know the memory
requirement?

>
>Milan
>

-- 
Best regards,
Coiby

Re: How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

[Milan Broz]

On 20/06/2022 02:19, Coiby Xu wrote:
> On Sat, Jun 18, 2022 at 05:12:56PM +0200, Milan Broz wrote:
>> On 16/06/2022 06:43, Coiby Xu wrote:
>>> Hi,
>>>
>>> Recently, I notice cryptsetup itself consumes significant amount of
>>> memory (~256M) when estimating the memory requirement for dumping vmcore
>>> to a LUKS-encrypted disk,
>>>
>>> $ time -v cryptsetup luksOpen encrypted.img volume --key-file mykey.keyfile | grep "Maximum resident set size"
>>>           Maximum resident set size (kbytes): 1309828
>>> $ cryptsetup luksDump encrypted.img
>>> ...
>>> Keyslots:
>>>     0: luks2
>>>           PBKDF:      argon2id
>>>           Memory:     1048576
>>>           ...
>>>
>>>
>>> So is there a way to estimate the upper bound of the peak memory
>>> consumption of cryptsetup itself without running cryptsetup?
>>
>> As you already found, the major memory consumption is by memory-hard KDF.
>> But this memory is used only while calculating keyslot encryption key,
>> it is released immediately after the Argon call is finished.
>> I do not think we have better estimation here.
> 
> Thanks for the reply! Sorry I meant the way to estimate the overhead of
> crypsetup itself i.e. ~256M in the above example. Previously I only take
> the memory consumption by memory-hard KDF into consideration and
> neglected the memory consumption of cryptsetup itself. This obviously
> leads to an underestimation of the memory requirement of cryptsetup. I
> need to overestimate the memory requirement a bit to make sure OOM won't
> happen that's why I am asking if there is a way to estimate the
> upper bound of memory requirement of cryptsetup itself.

There is no generic way to get a number - it depends on configuration
of the distro, libc, translations, everything that is locked including
shared libraries.

If it is about RHEL, you can perhaps know exact configuration - please
ask people in Red Hat.

>> (Another story is locking all memory, including big areas used by libc,
>> but that should not be problem here, I hope.)
> 
> Do you mean locking all memory first in order to know the memory
> requirement?

We use (mlockall(MCL_CURRENT | MCL_FUTURE) so it locks all used memory
+ all future allocated memory.

Today, it is not the best option and we will probably lock only specific
region with stored keys in the future.

Milan

Re: How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

[Coiby Xu]

On Fri, Jun 24, 2022 at 11:14:37AM +0200, Milan Broz wrote:
>On 20/06/2022 02:19, Coiby Xu wrote:
>>On Sat, Jun 18, 2022 at 05:12:56PM +0200, Milan Broz wrote:
>>>On 16/06/2022 06:43, Coiby Xu wrote:
>>>>Hi,
>>>>
>>>>Recently, I notice cryptsetup itself consumes significant amount of
>>>>memory (~256M) when estimating the memory requirement for dumping vmcore
>>>>to a LUKS-encrypted disk,
>>>>
>>>>$ time -v cryptsetup luksOpen encrypted.img volume --key-file mykey.keyfile | grep "Maximum resident set size"
>>>>          Maximum resident set size (kbytes): 1309828
>>>>$ cryptsetup luksDump encrypted.img
>>>>...
>>>>Keyslots:
>>>>    0: luks2
>>>>          PBKDF:      argon2id
>>>>          Memory:     1048576
>>>>          ...
>>>>
>>>>
>>>>So is there a way to estimate the upper bound of the peak memory
>>>>consumption of cryptsetup itself without running cryptsetup?
>>>
>>>As you already found, the major memory consumption is by memory-hard KDF.
>>>But this memory is used only while calculating keyslot encryption key,
>>>it is released immediately after the Argon call is finished.
>>>I do not think we have better estimation here.
>>
>>Thanks for the reply! Sorry I meant the way to estimate the overhead of
>>crypsetup itself i.e. ~256M in the above example. Previously I only take
>>the memory consumption by memory-hard KDF into consideration and
>>neglected the memory consumption of cryptsetup itself. This obviously
>>leads to an underestimation of the memory requirement of cryptsetup. I
>>need to overestimate the memory requirement a bit to make sure OOM won't
>>happen that's why I am asking if there is a way to estimate the
>>upper bound of memory requirement of cryptsetup itself.
>
>There is no generic way to get a number - it depends on configuration
>of the distro, libc, translations, everything that is locked including
>shared libraries.
>
>If it is about RHEL, you can perhaps know exact configuration - please
>ask people in Red Hat.

Provided the configuration, is there an golden algorithm or a formula to
get the number? If it doesn't exist and I need to do some tests to get
an empirical number, are there any big factors I need to be aware of?

>
>>>(Another story is locking all memory, including big areas used by libc,
>>>but that should not be problem here, I hope.)
>>
>>Do you mean locking all memory first in order to know the memory
>>requirement?
>
>We use (mlockall(MCL_CURRENT | MCL_FUTURE) so it locks all used memory
>+ all future allocated memory.
>
>Today, it is not the best option and we will probably lock only specific
>region with stored keys in the future.

Thanks for the explanation! 

>
>Milan
>

-- 
Best regards,
Coiby

Re: How to estimate the upper bound of the peak memory consumption of cryptsetup itself?

[Milan Broz]

On 24/06/2022 12:49, Coiby Xu wrote:
>>> Thanks for the reply! Sorry I meant the way to estimate the overhead of
>>> crypsetup itself i.e. ~256M in the above example. Previously I only take
>>> the memory consumption by memory-hard KDF into consideration and
>>> neglected the memory consumption of cryptsetup itself. This obviously
>>> leads to an underestimation of the memory requirement of cryptsetup. I
>>> need to overestimate the memory requirement a bit to make sure OOM won't
>>> happen that's why I am asking if there is a way to estimate the
>>> upper bound of memory requirement of cryptsetup itself.
>>
>> There is no generic way to get a number - it depends on configuration
>> of the distro, libc, translations, everything that is locked including
>> shared libraries.
>>
>> If it is about RHEL, you can perhaps know exact configuration - please
>> ask people in Red Hat.
> 
> Provided the configuration, is there an golden algorithm or a formula to
> get the number? If it doesn't exist and I need to do some tests to get
> an empirical number, are there any big factors I need to be aware of?

Think about current locales, OpenSSL policy and many other things.
(The major I remember was locking all libc loaded translations in memory.)

So really, the only answer here would be ... 42 :)

You can try to measure it. Not sure if it is reliable.

Usually people want to know this for some weird scenarios like kdump
or so - this is definitely not something upstream can help with.

Milan