How to signal kernel that shared library is not in use by any process anymore ?

How to signal kernel that shared library is not in use by any process anymore ?

[Lev Olshvang]

Hi All,

I have  an executable (C++) which is the exclusive user of the some shared library that it uses only during  the initialization phase.

I would like to free memory used by this shared library, because I am running on embedded system.

How can I achieve this?

I know that dlopen() will load shared library, and hope that following dlclose() will free this lib memory. 

1. Still  I do not know what method should be used to make dynamic linker look again into executable and resolve symbols of the newly appeared symbols ?
2. And  how to tell the dynamic linker ld-linux.so to postpone the symbol resolution until dlopen()  will load  the library?
3. Whether to compile and link executable with this library or leave unresolved symbols?

Waiting for your advises,
Lev.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to signal kernel that shared library is not in use by any process anymore ?

[Greg KH]

On Fri, Dec 21, 2018 at 05:20:36PM +0300, Lev Olshvang wrote:
> 
> Hi All,
> 
> I have  an executable (C++) which is the exclusive user of the some shared library that it uses only during  the initialization phase.
> 
> I would like to free memory used by this shared library, because I am running on embedded system.
> 
> How can I achieve this?
> 
> I know that dlopen() will load shared library, and hope that following dlclose() will free this lib memory. 

That right there is how you "achieve" this, call dlclose() and all will
be fine.  If your system needs the memory that was being used, it will
reclaim it from the memory that was previously being used by the library
at this point in time.

Nothing else needs to be done.

Have you tested this and found it not to work properly?

> 1. Still  I do not know what method should be used to make dynamic linker look again into executable and resolve symbols of the newly appeared symbols ?

What "newly appeared symbols"?

If you need to load the library again, call dlopen().

> 2. And  how to tell the dynamic linker ld-linux.so to postpone the symbol resolution until dlopen()  will load  the library?

It will not happen until you tell it to, right?

> 3. Whether to compile and link executable with this library or leave unresolved symbols?

It sounds like you don't really know what type of problem you are trying
to solve here.

Back up, what is the real issue you are having with the kernel at this
point in time?

thanks,

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to signal kernel that shared library is not in use by any process anymore ?

[Lev Olshvang]

Hello Greg,

Thanks for you your reply.
It help me to better express my question

From the application I can access /proc/self/maps and see which memory is  mapped  for my library I do not intend to use after application passes init phase.
I would like to unmap this memory region, but since I do not have file descriptor for this so I can not do this.
Only kernel can unmap this memory for me,
I am willing to pass library name or address to system cal, but IMHO such system call does not exist.
Should I rely on memory manager which will use reuse this pages because they will never page faulted back ?

I am afraid that since this is C++ lib, some pages are modified and will not be considered clean ?


Thanks again for your precious time.
Perhaps somebody from the list can respond too.

Thank you all
Lev

21.12.2018, 18:31, "Greg KH" <greg@kroah.com>:
> On Fri, Dec 21, 2018 at 05:20:36PM +0300, Lev Olshvang wrote:
>>  Hi All,
>>
>>  I have an executable (C++) which is the exclusive user of the some shared library that it uses only during the initialization phase.
>>
>>  I would like to free memory used by this shared library, because I am running on embedded system.
>>
>>  How can I achieve this?
>>
>>  I know that dlopen() will load shared library, and hope that following dlclose() will free this lib memory.
>
> That right there is how you "achieve" this, call dlclose() and all will
> be fine. If your system needs the memory that was being used, it will
> reclaim it from the memory that was previously being used by the library
> at this point in time.
>
> Nothing else needs to be done.
>
> Have you tested this and found it not to work properly?
>
>>  1. Still I do not know what method should be used to make dynamic linker look again into executable and resolve symbols of the newly appeared symbols ?
>
> What "newly appeared symbols"?
>
> If you need to load the library again, call dlopen().
>
>>  2. And how to tell the dynamic linker ld-linux.so to postpone the symbol resolution until dlopen() will load the library?
>
> It will not happen until you tell it to, right?
>
>>  3. Whether to compile and link executable with this library or leave unresolved symbols?
>
> It sounds like you don't really know what type of problem you are trying
> to solve here.
>
> Back up, what is the real issue you are having with the kernel at this
> point in time?
>
> thanks,
>
> greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to signal kernel that shared library is not in use by any process anymore ?

[Greg KH]

On Thu, Dec 27, 2018 at 10:47:42AM +0300, Lev Olshvang wrote:
> Hello Greg,
> 
> Thanks for you your reply.
> It help me to better express my question
> 
> >From the application I can access /proc/self/maps and see which memory is  mapped  for my library I do not intend to use after application passes init phase.
> I would like to unmap this memory region, but since I do not have file descriptor for this so I can not do this.

Why do you care?  The loader should handle all of this for you
automatically.

> Only kernel can unmap this memory for me,
> I am willing to pass library name or address to system cal, but IMHO such system call does not exist.
> Should I rely on memory manager which will use reuse this pages because they will never page faulted back ?

Yes you should.

> I am afraid that since this is C++ lib, some pages are modified and will not be considered clean ?

That depends on your code, if it is still in use, yes, it will not be
freed, you have control over that as this is your userspace code.  There
is nothing in the kernel you can do about this, sorry, please just fix
up your application.

good luck!

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to signal kernel that shared library is not in use by any process anymore ?

[Lior Okman]

[-- Attachment #1.1: Type: text/plain, Size: 3708 bytes --]

On Fri, Dec 21, 2018 at 4:21 PM Lev Olshvang <levonshe@yandex.com> wrote:

>
> Hi All,
>
> I have  an executable (C++) which is the exclusive user of the some shared
> library that it uses only during  the initialization phase.
>
> I would like to free memory used by this shared library, because I am
> running on embedded system.
>
> How can I achieve this?
>
> I know that dlopen() will load shared library, and hope that following
> dlclose() will free this lib memory.
>

According to the dlclose (2) man page:
      "The  function  dlclose() decrements the reference count on the
dynamically loaded shared object referred to by handle.  If the reference
count drops to zero, then the object is unloaded.  All shared
       objects that were automatically loaded when dlopen() was invoked on
the object referred to by handle are recursively closed in the same manner.

       A successful return from dlclose() does not guarantee that the
symbols associated with handle are removed from the caller's address
space.  In addition to references resulting from explicit dlopen()
       calls,  a shared object may have been implicitly loaded (and
reference counted) because of dependencies in other shared objects.  Only
when all references have been released can the shared object be
       removed from the address space."



> 1. Still  I do not know what method should be used to make dynamic linker
> look again into executable and resolve symbols of the newly appeared
> symbols ?
>

If you are using the dlopen/dlclose functions, you are responsible for
symbol resolution for symbols provided by your shared object. After you get
a handle from dlopen(), you need to call dlsym() in order to get a pointer
to your symbol. You can then call that symbol when you need it.


> 2. And  how to tell the dynamic linker ld-linux.so to postpone the symbol
> resolution until dlopen()  will load  the library?
>

When you use dlopen() your compilation shouldn't need to resolve any
symbols from your dlopen()-ed libraries.


> 3. Whether to compile and link executable with this library or leave
> unresolved symbols?
>

You shouldn't have any unresolved symbols, nor should you link to your
library during compilation. I guess the best thing is to show an example:

Given the following library source (printint.c) :

------------------------
#include <stdio.h>

void printint(int num) {
    printf("Called with num=%d \n", num);
}
------------------------

Create a shared object from it using
         gcc -shared -o libprintint.so  printint.c

Now consider the following test program (main.c) which uses printint with
dlopen (removed most of the error handling for clarity here):
-------------------
#include <dlfcn.h>

typedef void printint(int num);

int main(int argc, char *argv[]) {
    void *handle = NULL;
    handle = dlopen("./libprintint.so", RTLD_LAZY);
    if (handle == NULL ) {
        // use dlerror to find out what went wrong
        return -1;
    }
    printint *sym = NULL;
    sym = (printint*)dlsym(handle, "printint");
    if (sym == NULL ) {
        // use dlerror to find out what went wrong
        return -1;
    }
    sym(argc);
    dlclose(handle);
    return 0;
}
--------------

You compile this program like this:

              gcc  main.c -ldl -o a.out

You can verify that the program doesn't dynamically link to libprintint.so
by running "ldd ./a.out". When you run it with libprintit.so in the same
directory it will load the shared library and call the correct function.



> Waiting for your advises,
> Lev.
>
> _______________________________________________
> Linux-il mailing list
> Linux-il@cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>

[-- Attachment #1.2: Type: text/html, Size: 5688 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to signal kernel that shared library is not in use by any process anymore ?

[Lev Olshvang]

[-- Attachment #1: Type: text/html, Size: 7441 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to signal kernel that shared library is not in use by any process anymore ?

[Bernd Petrovitsch]

Hi all!

On 21/12/2018 15:20, Lev Olshvang wrote:
[...]
> I have  an executable (C++) which is the exclusive user of the some shared library that it uses only during  the initialization phase.
> 
> I would like to free memory used by this shared library, because I am running on embedded system.
> 
> How can I achieve this?
> 
> I know that dlopen() will load shared library, and hope that following dlclose() will free this lib memory. 
> 
> 1. Still  I do not know what method should be used to make dynamic linker look again into executable and resolve symbols of the newly appeared symbols ?

There is no such method TTBOMK. Since the linker (and thus the shared
loader) do not know of this library, they can't do anything (see below).

The C++ application has to resolve the symbols with dlsym() (after a
successful dlopen()) and call it with an appropriate type cast.
After dlclose(), the pointers from dlasym() are invalid (obviously - as
the lib is gone/unmapped).

> 2. And  how to tell the dynamic linker ld-linux.so to postpone the symbol resolution until dlopen()  will load  the library?

Do not mention that library (when linking the application).

> 3. Whether to compile and link executable with this library or leave unresolved symbols?

You do not call the functions directly (and you cannot as the linker
cannot resolve them anyways as the linker doesn't know the library).

MfG,
	Bernd
-- 
Bernd Petrovitsch                  Email : bernd@petrovitsch.priv.at
                     LUGA : http://www.luga.at

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies