Question about cache_size in rte_mempool_create

Question about cache_size in rte_mempool_create

[Roy Shterman]

Hi,

In the documentation it says that:

 * @param cache_size
 *   If cache_size is non-zero, the rte_mempool library will try to
 *   limit the accesses to the common lockless pool, by maintaining a
 *   per-lcore object cache. This argument must be lower or equal to
 *   CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.* It is advised to
choose*
* *   cache_size to have "n modulo cache_size == 0": if this is*
* *   not the case, some elements will always stay in the pool and will*
* *   never be used.* The access to the per-lcore table is of course
 *   faster than the multi-producer/consumer pool. The cache can be
 *   disabled if the cache_size argument is set to 0; it can be useful to
 *   avoid losing objects in cache.

I wonder if someone can please explain the high-lightened sentence, how the
cache size affects the objects inside the ring. And also does it mean that
if I'm sharing pool between different cores can it be that a core sees the
pool as empty although it has objects in it?

Thanks,
Roy

Re: Question about cache_size in rte_mempool_create

[Bruce Richardson]

On Thu, Nov 23, 2017 at 11:05:11PM +0200, Roy Shterman wrote:
> Hi,
> 
> In the documentation it says that:
> 
>  * @param cache_size
>  *   If cache_size is non-zero, the rte_mempool library will try to
>  *   limit the accesses to the common lockless pool, by maintaining a
>  *   per-lcore object cache. This argument must be lower or equal to
>  *   CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.* It is advised to
> choose*
> * *   cache_size to have "n modulo cache_size == 0": if this is*
> * *   not the case, some elements will always stay in the pool and will*
> * *   never be used.* The access to the per-lcore table is of course
>  *   faster than the multi-producer/consumer pool. The cache can be
>  *   disabled if the cache_size argument is set to 0; it can be useful to
>  *   avoid losing objects in cache.
> 
> I wonder if someone can please explain the high-lightened sentence, how the
> cache size affects the objects inside the ring.

It has no effects upon the objects themselves. Having a cache is
strongly recommended for performance reasons. Accessing a shared ring
for a mempool is very slow compared to pulling packets from a per-core
cache. To test this you can run testpmd using different --mbcache
parameters. 

> And also does it mean that
> if I'm sharing pool between different cores can it be that a core sees the
> pool as empty although it has objects in it?
> 
Yes, that can occur. You need to dimension the pool to take account of
your cache usage.

/Bruce

Re: Question about cache_size in rte_mempool_create

[roy]

Thanks for your answer, but I cannot understand the dimension of the 
ring and it is affected by the cache size.

On 24/11/17 11:30, Bruce Richardson wrote:
> On Thu, Nov 23, 2017 at 11:05:11PM +0200, Roy Shterman wrote:
>> Hi,
>>
>> In the documentation it says that:
>>
>>   * @param cache_size
>>   *   If cache_size is non-zero, the rte_mempool library will try to
>>   *   limit the accesses to the common lockless pool, by maintaining a
>>   *   per-lcore object cache. This argument must be lower or equal to
>>   *   CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.* It is advised to
>> choose*
>> * *   cache_size to have "n modulo cache_size == 0": if this is*
>> * *   not the case, some elements will always stay in the pool and will*
>> * *   never be used.* The access to the per-lcore table is of course
>>   *   faster than the multi-producer/consumer pool. The cache can be
>>   *   disabled if the cache_size argument is set to 0; it can be useful to
>>   *   avoid losing objects in cache.
>>
>> I wonder if someone can please explain the high-lightened sentence, how the
>> cache size affects the objects inside the ring.
> It has no effects upon the objects themselves. Having a cache is
> strongly recommended for performance reasons. Accessing a shared ring
> for a mempool is very slow compared to pulling packets from a per-core
> cache. To test this you can run testpmd using different --mbcache
> parameters.
Still, I didn't understand the sentence from above:

*It is advised to choose cache_size to have "n modulo cache_size == 0": 
if this is* not the case, some elements will always stay in the pool and 
will* never be used.*

>
>> And also does it mean that
>> if I'm sharing pool between different cores can it be that a core sees the
>> pool as empty although it has objects in it?
>>
> Yes, that can occur. You need to dimension the pool to take account of
> your cache usage.

can you please elaborate more on this issue? I'm working with 
multi-consumer multi-producer pools, in my understanding object can or 
in lcore X cache or in ring.
Each core when looking for objects in pool (ring) is looking at 
prod/cons head/tail so how can it be that the cache of different cores 
affects this?

>
> /Bruce

Re: Question about cache_size in rte_mempool_create

[Bruce Richardson]

On Fri, Nov 24, 2017 at 11:39:54AM +0200, roy wrote:
> Thanks for your answer, but I cannot understand the dimension of the ring
> and it is affected by the cache size.
> 
> On 24/11/17 11:30, Bruce Richardson wrote:
> > On Thu, Nov 23, 2017 at 11:05:11PM +0200, Roy Shterman wrote:
> > > Hi,
> > > 
> > > In the documentation it says that:
> > > 
> > >   * @param cache_size
> > >   *   If cache_size is non-zero, the rte_mempool library will try to
> > >   *   limit the accesses to the common lockless pool, by maintaining a
> > >   *   per-lcore object cache. This argument must be lower or equal to
> > >   *   CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.* It is advised to
> > > choose*
> > > * *   cache_size to have "n modulo cache_size == 0": if this is*
> > > * *   not the case, some elements will always stay in the pool and will*
> > > * *   never be used.* The access to the per-lcore table is of course
> > >   *   faster than the multi-producer/consumer pool. The cache can be
> > >   *   disabled if the cache_size argument is set to 0; it can be useful to
> > >   *   avoid losing objects in cache.
> > > 
> > > I wonder if someone can please explain the high-lightened sentence, how the
> > > cache size affects the objects inside the ring.
> > It has no effects upon the objects themselves. Having a cache is
> > strongly recommended for performance reasons. Accessing a shared ring
> > for a mempool is very slow compared to pulling packets from a per-core
> > cache. To test this you can run testpmd using different --mbcache
> > parameters.
> Still, I didn't understand the sentence from above:
> 
> *It is advised to choose cache_size to have "n modulo cache_size == 0": if
> this is* not the case, some elements will always stay in the pool and will*
> never be used.*
>

This would be an artifact of the way in which the elements are taken
from the pool ring. If a full cache-size burst of elements is not
available in the ring, no elements from the ring are put in the cache.
It just means that the pool can never fully be emptied. However, in most
cases, even having the pool nearly empty indicates a problem, so
practically I wouldn't be worried about this.

> > 
> > > And also does it mean that
> > > if I'm sharing pool between different cores can it be that a core sees the
> > > pool as empty although it has objects in it?
> > > 
> > Yes, that can occur. You need to dimension the pool to take account of
> > your cache usage.
> 
> can you please elaborate more on this issue? I'm working with multi-consumer
> multi-producer pools, in my understanding object can or in lcore X cache or
> in ring.
> Each core when looking for objects in pool (ring) is looking at prod/cons
> head/tail so how can it be that the cache of different cores affects this?
> 

Mempool elements in the caches are free elements that are available for
use. However, they are inaccessible to any core except the core which
freed them to that cache. For a slightly simplified example, consider a
pool with 256 elements, and a cache size of 64. If 4 cores all request 1
buffer each, those four cores will each fill their caches and then take
1 element from those caches. This means that the ring will be empty even
though only 4 buffers are actually in use - the other 63*4 buffers are
in per-core caches. A 5th core which comes along and requests a buffer
will be told that the pool is empty, despite there being 252 free
elements.

Therefore you need to take account of the possibilities of buffers being
stored in caches when dimensioning your mempool.

/Bruce

Re: Question about cache_size in rte_mempool_create

[Roy Shterman]

נשלח מה-iPhone שלי

‫ב-24 בנוב׳ 2017, בשעה 12:03, ‏‏Bruce Richardson ‏<bruce.richardson@intel.com> כתב/ה:‬

>> On Fri, Nov 24, 2017 at 11:39:54AM +0200, roy wrote:
>> Thanks for your answer, but I cannot understand the dimension of the ring
>> and it is affected by the cache size.
>> 
>>> On 24/11/17 11:30, Bruce Richardson wrote:
>>>> On Thu, Nov 23, 2017 at 11:05:11PM +0200, Roy Shterman wrote:
>>>> Hi,
>>>> 
>>>> In the documentation it says that:
>>>> 
>>>>  * @param cache_size
>>>>  *   If cache_size is non-zero, the rte_mempool library will try to
>>>>  *   limit the accesses to the common lockless pool, by maintaining a
>>>>  *   per-lcore object cache. This argument must be lower or equal to
>>>>  *   CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.* It is advised to
>>>> choose*
>>>> * *   cache_size to have "n modulo cache_size == 0": if this is*
>>>> * *   not the case, some elements will always stay in the pool and will*
>>>> * *   never be used.* The access to the per-lcore table is of course
>>>>  *   faster than the multi-producer/consumer pool. The cache can be
>>>>  *   disabled if the cache_size argument is set to 0; it can be useful to
>>>>  *   avoid losing objects in cache.
>>>> 
>>>> I wonder if someone can please explain the high-lightened sentence, how the
>>>> cache size affects the objects inside the ring.
>>> It has no effects upon the objects themselves. Having a cache is
>>> strongly recommended for performance reasons. Accessing a shared ring
>>> for a mempool is very slow compared to pulling packets from a per-core
>>> cache. To test this you can run testpmd using different --mbcache
>>> parameters.
>> Still, I didn't understand the sentence from above:
>> 
>> *It is advised to choose cache_size to have "n modulo cache_size == 0": if
>> this is* not the case, some elements will always stay in the pool and will*
>> never be used.*
>> 
> 
> This would be an artifact of the way in which the elements are taken
> from the pool ring. If a full cache-size burst of elements is not
> available in the ring, no elements from the ring are put in the cache.
> It just means that the pool can never fully be emptied. However, in most
> cases, even having the pool nearly empty indicates a problem, so
> practically I wouldn't be worried about this.
> 

But in case we tried to get cache size from pool and failed we will try to get the number on objects defined in rte_mempool_get_bulk, so in case rte_mempool_get() usage we will try to get one object out of the pool (ring) so also if there isn't cache size in the ring itself each core can get 1 object in each rte_memoool_get until the pool is empty, am I wrong?

>>> 
>>>> And also does it mean that
>>>> if I'm sharing pool between different cores can it be that a core sees the
>>>> pool as empty although it has objects in it?
>>>> 
>>> Yes, that can occur. You need to dimension the pool to take account of
>>> your cache usage.
>> 
>> can you please elaborate more on this issue? I'm working with multi-consumer
>> multi-producer pools, in my understanding object can or in lcore X cache or
>> in ring.
>> Each core when looking for objects in pool (ring) is looking at prod/cons
>> head/tail so how can it be that the cache of different cores affects this?
>> 
> 
> Mempool elements in the caches are free elements that are available for
> use. However, they are inaccessible to any core except the core which
> freed them to that cache. For a slightly simplified example, consider a
> pool with 256 elements, and a cache size of 64. If 4 cores all request 1
> buffer each, those four cores will each fill their caches and then take
> 1 element from those caches. This means that the ring will be empty even
> though only 4 buffers are actually in use - the other 63*4 buffers are
> in per-core caches. A 5th core which comes along and requests a buffer
> will be told that the pool is empty, despite there being 252 free
> elements.
> 
> Therefore you need to take account of the possibilities of buffers being
> stored in caches when dimensioning your mempool.
> 
> /Bruce

Re: Question about cache_size in rte_mempool_create

[Bruce Richardson]

On Fri, Nov 24, 2017 at 01:01:08PM +0200, Roy Shterman wrote:
> 
> 
> נשלח מה-iPhone שלי
> 
> ‫ב-24 בנוב׳ 2017, בשעה 12:03, ‏‏Bruce Richardson ‏<bruce.richardson@intel.com> כתב/ה:‬
> 
> >> On Fri, Nov 24, 2017 at 11:39:54AM +0200, roy wrote:
> >> Thanks for your answer, but I cannot understand the dimension of the ring
> >> and it is affected by the cache size.
> >> 
> >>> On 24/11/17 11:30, Bruce Richardson wrote:
> >>>> On Thu, Nov 23, 2017 at 11:05:11PM +0200, Roy Shterman wrote:
> >>>> Hi,
> >>>> 
> >>>> In the documentation it says that:
> >>>> 
> >>>>  * @param cache_size
> >>>>  *   If cache_size is non-zero, the rte_mempool library will try to
> >>>>  *   limit the accesses to the common lockless pool, by maintaining a
> >>>>  *   per-lcore object cache. This argument must be lower or equal to
> >>>>  *   CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE and n / 1.5.* It is advised to
> >>>> choose*
> >>>> * *   cache_size to have "n modulo cache_size == 0": if this is*
> >>>> * *   not the case, some elements will always stay in the pool and will*
> >>>> * *   never be used.* The access to the per-lcore table is of course
> >>>>  *   faster than the multi-producer/consumer pool. The cache can be
> >>>>  *   disabled if the cache_size argument is set to 0; it can be useful to
> >>>>  *   avoid losing objects in cache.
> >>>> 
> >>>> I wonder if someone can please explain the high-lightened sentence, how the
> >>>> cache size affects the objects inside the ring.
> >>> It has no effects upon the objects themselves. Having a cache is
> >>> strongly recommended for performance reasons. Accessing a shared ring
> >>> for a mempool is very slow compared to pulling packets from a per-core
> >>> cache. To test this you can run testpmd using different --mbcache
> >>> parameters.
> >> Still, I didn't understand the sentence from above:
> >> 
> >> *It is advised to choose cache_size to have "n modulo cache_size == 0": if
> >> this is* not the case, some elements will always stay in the pool and will*
> >> never be used.*
> >> 
> > 
> > This would be an artifact of the way in which the elements are taken
> > from the pool ring. If a full cache-size burst of elements is not
> > available in the ring, no elements from the ring are put in the cache.
> > It just means that the pool can never fully be emptied. However, in most
> > cases, even having the pool nearly empty indicates a problem, so
> > practically I wouldn't be worried about this.
> > 
> 
> But in case we tried to get cache size from pool and failed we will try to get the number on objects defined in rte_mempool_get_bulk, so in case rte_mempool_get() usage we will try to get one object out of the pool (ring) so also if there isn't cache size in the ring itself each core can get 1 object in each rte_memoool_get until the pool is empty, am I wrong?
> 

If there is no cache, you can always get all elements out of the ring by
getting one at a time, yes.