Re: Coverity: HUF_buildCTableFromTree(): Memory - corruptions

[Nick Terrell <terrelln@meta.com>]

> On Oct 27, 2022, at 6:43 AM, Jann Horn <jannh@google.com> wrote:
> 
> !-------------------------------------------------------------------|
>  This Message Is From an External Sender
> 
> |-------------------------------------------------------------------!
> 
> On Thu, Oct 27, 2022 at 2:06 AM coverity-bot <keescook@chromium.org> wrote:
>> 
>> Hello!
>> 
>> This is an experimental semi-automated report about issues detected by
>> Coverity from a scan of next-20221026 as part of the linux-next scan project:
>> https://scan.coverity.com/projects/linux-next-weekly-scan  
>> 
>> You're getting this email because you were associated with the identified
>> lines of code (noted below) that were touched by commits:
>> 
>>  Mon Oct 24 12:12:32 2022 -0700
>>    2aa14b1ab2c4 ("zstd: import usptream v1.5.2")
>> 
>> Coverity reported the following:
>> 
>> *** CID 1525550:  Memory - corruptions  (OVERRUN)
>> /lib/zstd/compress/huf_compress.c: 673 in HUF_buildCTableFromTree()
>> 667                 min += nbPerRank[n];
>> 668                 min >>= 1;
>> 669         }   }
>> 670         for (n=0; n<alphabetSize; n++)
>> 671             HUF_setNbBits(ct + huffNode[n].byte, huffNode[n].nbBits);   /* push nbBits per symbol, symbol order */
>> 672         for (n=0; n<alphabetSize; n++)
>> vvv     CID 1525550:  Memory - corruptions  (OVERRUN)
>> vvv     Overrunning array "valPerRank" of 13 2-byte elements at element index 255 (byte offset 511) using index "HUF_getNbBits(ct[n])" (which evaluates to 255).
>> 673             HUF_setValue(ct + n, valPerRank[HUF_getNbBits(ct[n])]++);   /* assign value within rank, symbol order */
>> 674         CTable[0] = maxNbBits;
>> 675     }
>> 676
>> 677     size_t HUF_buildCTable_wksp (HUF_CElt* CTable, const unsigned* count, U32 maxSymbolValue, U32 maxNbBits, void* workSpace, size_t wkspSize)
>> 678     {
> 
> I haven't looked at the other warnings, but from a glance this code
> looks fine to me. Coverity is claiming that some symbols can have 255
> bits, but we just went through HUF_setMaxHeight(), which enforced that
> no symbols have more bits than HUF_TABLELOG_MAX.
> 
> Heuristic checks like this are likely to generate lots of false
> positives in compression code, I think.

Yeah, the warnings in huf_compress.c are definitely false positives.

I'm checking on the others.