Re: Read before you deploy btrfs + zstd

[Nick Terrell <terrelln@fb.com>]

> On Nov 21, 2017, at 8:22 AM, David Sterba <dsterba@suse.cz> wrote:
> 
> On Wed, Nov 15, 2017 at 08:09:15PM +0000, Nick Terrell wrote:
>> On 11/15/17, 6:41 AM, "David Sterba" <dsterba@suse.cz> wrote:
>>> The branch is now in a state that can be tested. Turns out the memory
>>> requirements are too much for grub, so the boot fails with "not enough
>>> memory". The calculated value
>>> 
>>> ZSTD_BTRFS_MAX_INPUT: 131072
>>> ZSTD_DStreamWorkspaceBound with ZSTD_BTRFS_MAX_INPUT: 549424
>>> 
>>> This is not something I could fix easily, we'd probalby need a tuned
>>> version of ZSTD for grub constraints. Adding Nick to CC.
>> 
>> If I understand the grub code correctly, we only need to read, and we have
>> the entire input and output buffer in one segment. In that case you can use
>> ZSTD_initDCtx(), and ZSTD_decompressDCtx(). ZSTD_DCtxWorkspaceBound() is
>> only 155984. See decompress_single() in
>> https://patchwork.kernel.org/patch/9997909/ for an example.
> 
> Does not help, still ENOMEM.

It looks like XZ had the same issue, and they make the decompression
context a static object (grep for GRUB_EMBED_DECOMPRESSOR). We could
potentially do the same and statically allocate the workspace:

```
/* Could also be size_t */
#define BTRFS_ZSTD_WORKSPACE_SIZE_U64 (155984 / sizeof(uint64_t))
static uint64_t workspace[BTRFS_ZSTD_WORKSPACE_SIZE_U64];

/* ... */

assert(sizeof(workspace) >= ZSTD_DCtxWorkspaceBound());
```