Re: [PATCH 2/2] strbuf: allow to use preallocated memory

[Mike Hommey <mh@glandium.org>]

On Tue, May 31, 2016 at 12:46:23AM +0200, William Duclot wrote:
> Mike Hommey <mh@glandium.org> writes:
> >>  struct strbuf {
> >> +	unsigned int flags;
> >>  	size_t alloc;
> >>  	size_t len;
> >>  	char *buf;
> >>  };
> > 
> > Depending whether the size of strbuf matters, it /might/ be worth
> > considering some packing here. malloc() usually returns buffers that can
> > contain more data than what is requested. Which means allocation sizes
> > could be rounded and that wouldn't change the amount of allocated
> > memory. On glibc malloc_usable_size(malloc(1)) apparently returns 24.
> > On jemalloc, it's 4 or 8. It's in the same ballbark with many
> > allocators.
> > 
> > So, it would be possible to round alloc such that it's always a multiple
> > of, say, 4, and stick flags in the low, unused bits.
> 
> If I'm not mistaken, the memory allocated is not necessarily linear with
> the size asked, depending on the algorithm used by the allocator and/or
> the kernel. The system for exemple use powers of two, if the user asks
> for exactly 2^x bytes, adding the space for the flags would lead to an
> allocation of 2^(x+1) bytes.

No, it would not. If you requested 129 bytes, you'd request 136 instead,
which the allocator would round to the same power of two. If you
requested 128, you'd still request 128. It's not about adding space in
the allocated buffer for the flags, it's about needing less bits in
`alloc` because those bits are effectively useless because of how
allocators work.

> Way worse than storing an unsigned.
> If the allocator use a fibonnaci system, we can't even rely on multiples
> of 4 (or 2).
> I'm not sure the fibonnaci system is actually used by any allocator, but
> my point is that I'm not sure it is a good thing to rely on such 
> low-level implementations.

Allocators have constraints related to word sizes and alignment, so they
are pretty much guaranteed to align things to powers of two.

Mike