RTE Ring removing

RTE Ring removing

[Igor Ryzhov]

Hello.

For what reason RTE Rings can not be removed once created?
In my application I want to use many rings with different names so I 
think there may be a problem with memory because of many ring that 
already not in use, but allocated.
Or DPDK has a mechanism of reusing memory if rings are not in use?

Best regards,
Igor Ryzhov

Re: RTE Ring removing

[Igor Ryzhov]

Hello again.

I did some investigation on the code.
I learned that RTE Ring creation function uses functions related to RTE 
Memzone to reserve memory (rte_memzone_reserve).
Documentation states that once reserved memzone can not be unreserved. I 
decided to find out why it is so.

I noticed that in Memzone realization there is a special global variable 
"free_memseg" containing pointers on free memory segments.
An memzone reserve function just finst the best segment for allocation 
from this "free_memseg" variable.

So I think there is a possibility to unreserve already reserved memory 
back to "free_memseg", and impossibility of unreserving memory is just 
because there is no function for that, not because it is impossible in 
principle.
Am I right? Or there are any restrictions?

Best regards,
Igor Ryzhov

06.05.2014 13:05, Igor Ryzhov пишет:
> Hello.
>
> For what reason RTE Rings can not be removed once created?
> In my application I want to use many rings with different names so I 
> think there may be a problem with memory because of many ring that 
> already not in use, but allocated.
> Or DPDK has a mechanism of reusing memory if rings are not in use?
>
> Best regards,
> Igor Ryzhov

Re: RTE Ring removing

[Olivier MATZ]

Hi Igor,

On 05/07/2014 09:54 AM, Igor Ryzhov wrote:
> I noticed that in Memzone realization there is a special global variable
> "free_memseg" containing pointers on free memory segments.
> An memzone reserve function just finst the best segment for allocation
> from this "free_memseg" variable.
>
> So I think there is a possibility to unreserve already reserved memory
> back to "free_memseg", and impossibility of unreserving memory is just
> because there is no function for that, not because it is impossible in
> principle.
> Am I right? Or there are any restrictions?

I think that implementing a freeing of memory segment is feasible, but
it would require some work to properly merge freed zones to avoid memory
fragmentation.

Another solution is to allocate/free rings in standard memory (malloc
for instance) instead of rte_memzones. Let me know if the patches I've
just sent on the mailing list solves your issue.

By the way, I plan to do the same thing for mempools in the coming
weeks but there is much more work.

Regards,
Olivier

Re: RTE Ring removing

[Igor Ryzhov]

It seems to be a good idea, thank you, Olivier!

But a few questions:
1. Will this changes affect performance?
2. In PATCH 2/2 you have a small bug:

In file rte_ring.h, in comments describing rte_ring_init function you have:

+ * @param name
+ *   The size of the ring.

But it is name of the ring, not size.

Best regards,
Igor Ryzhov

07.05.2014 15:39, Olivier MATZ пишет:
> Hi Igor,
>
> On 05/07/2014 09:54 AM, Igor Ryzhov wrote:
>> I noticed that in Memzone realization there is a special global variable
>> "free_memseg" containing pointers on free memory segments.
>> An memzone reserve function just finst the best segment for allocation
>> from this "free_memseg" variable.
>>
>> So I think there is a possibility to unreserve already reserved memory
>> back to "free_memseg", and impossibility of unreserving memory is just
>> because there is no function for that, not because it is impossible in
>> principle.
>> Am I right? Or there are any restrictions?
>
> I think that implementing a freeing of memory segment is feasible, but
> it would require some work to properly merge freed zones to avoid memory
> fragmentation.
>
> Another solution is to allocate/free rings in standard memory (malloc
> for instance) instead of rte_memzones. Let me know if the patches I've
> just sent on the mailing list solves your issue.
>
> By the way, I plan to do the same thing for mempools in the coming
> weeks but there is much more work.
>
> Regards,
> Olivier
>

Re: RTE Ring removing

[Olivier MATZ]

Hi Igor,

On 05/07/2014 02:42 PM, Igor Ryzhov wrote:
> But a few questions:
> 1. Will this changes affect performance?

It should not. If you have many rings, you may allocate them
in huge pages to avoid TLB misses.

> 2. In PATCH 2/2 you have a small bug:
>
> In file rte_ring.h, in comments describing rte_ring_init function you have:
>
> + * @param name
> + *   The size of the ring.
>
> But it is name of the ring, not size.

Thank you for this comment, I'll fix it in the v2.

Regards,
Olivier

Re: RTE Ring removing

[Venkatesan, Venky]

Olivier, 

We should look at how to make the memseg capable of doing alloc/free (including re-assembly of fragments) after the 1.7 release. Is that something you are considering doing (or are there any other DPDKers considering this), or should I look at putting together a patch for that? 

Regards, 
-Venky

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier MATZ
Sent: Wednesday, May 07, 2014 4:39 AM
To: Igor Ryzhov; dev@dpdk.org
Subject: Re: [dpdk-dev] RTE Ring removing

Hi Igor,

On 05/07/2014 09:54 AM, Igor Ryzhov wrote:
> I noticed that in Memzone realization there is a special global 
> variable "free_memseg" containing pointers on free memory segments.
> An memzone reserve function just finst the best segment for allocation 
> from this "free_memseg" variable.
>
> So I think there is a possibility to unreserve already reserved memory 
> back to "free_memseg", and impossibility of unreserving memory is just 
> because there is no function for that, not because it is impossible in 
> principle.
> Am I right? Or there are any restrictions?

I think that implementing a freeing of memory segment is feasible, but it would require some work to properly merge freed zones to avoid memory fragmentation.

Another solution is to allocate/free rings in standard memory (malloc for instance) instead of rte_memzones. Let me know if the patches I've just sent on the mailing list solves your issue.

By the way, I plan to do the same thing for mempools in the coming weeks but there is much more work.

Regards,
Olivier

Re: RTE Ring removing

[Olivier MATZ]

Hi Venky,

On 05/07/2014 04:01 PM, Venkatesan, Venky wrote:
> We should look at how to make the memseg capable of doing alloc/free
> (including re-assembly of fragments) after the 1.7 release. Is that
> something you are considering doing (or are there any other DPDKers
> considering this), or should I look at putting together a patch for
> that?

No, that's not something I'm working on today.

On this topic, I have some work in progress in the rte_mempool code.
I'll submit it here as soon as it is ready, I'm not sure it will be
finished before the end of the 1.7.0 integration window.

Regards,
Olivier

Re: RTE Ring removing

[Ananyev, Konstantin]

Hi Oliver,
Just to clarify about mempool - I suppose you are talking about ability to place internal ring and mempool metadata inside externally allocated memory?
It is already possible to keep mempool elements inside externally allocated memory (rte_mempool_xmem_create()). 
Konstantin

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier MATZ
Sent: Wednesday, May 07, 2014 4:10 PM
To: Venkatesan, Venky; Igor Ryzhov; dev@dpdk.org
Subject: Re: [dpdk-dev] RTE Ring removing

Hi Venky,

On 05/07/2014 04:01 PM, Venkatesan, Venky wrote:
> We should look at how to make the memseg capable of doing alloc/free
> (including re-assembly of fragments) after the 1.7 release. Is that
> something you are considering doing (or are there any other DPDKers
> considering this), or should I look at putting together a patch for
> that?

No, that's not something I'm working on today.

On this topic, I have some work in progress in the rte_mempool code.
I'll submit it here as soon as it is ready, I'm not sure it will be
finished before the end of the 1.7.0 integration window.

Regards,
Olivier

Re: RTE Ring removing

[Rogers, Gerald]

Venky,

This also applies to mbuf pools.  Inside of the openvswitch.org patches we
allocate mbuf pools for a port, but we are unable to free them back when
the port is removed.

One other request (maybe it is there, and I¹m unaware), is the ability to
dynamically add / remove a physical port to DPDK.  Basically we should be
able to reassign on the fly a port from the kernel to DPDK, and vice versa
(of course with the caveat that all structures be released in both
environments and a port reinitialized).

Gerald

On 5/7/14, 7:01 AM, "Venkatesan, Venky" <venky.venkatesan-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:

>Olivier, 
>
>We should look at how to make the memseg capable of doing alloc/free
>(including re-assembly of fragments) after the 1.7 release. Is that
>something you are considering doing (or are there any other DPDKers
>considering this), or should I look at putting together a patch for that?
>
>Regards, 
>-Venky
>
>-----Original Message-----
>From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Olivier MATZ
>Sent: Wednesday, May 07, 2014 4:39 AM
>To: Igor Ryzhov; dev-VfR2kkLFssw@public.gmane.org
>Subject: Re: [dpdk-dev] RTE Ring removing
>
>Hi Igor,
>
>On 05/07/2014 09:54 AM, Igor Ryzhov wrote:
>> I noticed that in Memzone realization there is a special global
>> variable "free_memseg" containing pointers on free memory segments.
>> An memzone reserve function just finst the best segment for allocation
>> from this "free_memseg" variable.
>>
>> So I think there is a possibility to unreserve already reserved memory
>> back to "free_memseg", and impossibility of unreserving memory is just
>> because there is no function for that, not because it is impossible in
>> principle.
>> Am I right? Or there are any restrictions?
>
>I think that implementing a freeing of memory segment is feasible, but it
>would require some work to properly merge freed zones to avoid memory
>fragmentation.
>
>Another solution is to allocate/free rings in standard memory (malloc for
>instance) instead of rte_memzones. Let me know if the patches I've just
>sent on the mailing list solves your issue.
>
>By the way, I plan to do the same thing for mempools in the coming weeks
>but there is much more work.
>
>Regards,
>Olivier
>

Re: RTE Ring removing

[Olivier MATZ]

Hi Konstantin,

On 05/07/2014 05:19 PM, Ananyev, Konstantin wrote:
> Just to clarify about mempool - I suppose you are talking about
> ability to place internal ring and mempool metadata inside externally
> allocated memory?

Yes, exactly.

> It is already possible to keep mempool elements inside externally
> allocated memory (rte_mempool_xmem_create()).

You are right, but I think the current API is a bit too complex.
For instance the function rte_mempool_xmem_create() has 15 arguments
which is probably too much.

Anyway, as soon as I'll have a patch to show, I'll send it to the
mailing list so we can discuss it.

Regards,
Olivier