All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-06-02  2:19 Trevor Woerner
  0 siblings, 0 replies; 9+ messages in thread
From: Trevor Woerner @ 2018-06-02  2:19 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 4410 bytes --]

It seems odd for this project would say "we don't want to have to deal with
versioning". Aren't we now, as we speak, witnessing a
non-backwards-compatible API version break with TSS? How are users supposed
to deal with this? There are existing apps that expect the old TSS and they
will never be updated. Now the TSS is changing in such a way that all those
programs will break if they try to link to and run with the new shared
libraries. If those libraries aren't versioned, how are they, or the
runtime linker supposed to know what to do?

If everyone followed convention, the old apps would look for and
runtime-link against, say, libtss.so.0. Now that the API has changed, the
new library should be called libtss.so.1. New apps can link against this
new library, and everything will work. Distributions are welcome to provide
the old TSS library, or the new one, or both, or neither. Applications that
aren't updated and expect the old API will continue to runtime-link against
libtss.so.0, and new apps or old apps that are updated will runtime-link
against libtss.so.1. No conflicts, no issues.

The "shared library hell" occurs when all versioning is removed and
everyone assumes there's just one, unversioned library against which to
link: libtss.so. How will distros provide support for old and new apps? How
will developers ever know which one is installed?

Versioned libraries are not a new idea. Unversioned shared-object files
will make it difficult for distributions to package these libraries, to the
point where they might decide to not bother trying to package them at all.
Unversioned shared-objects will always be a continual source of instability
for anything that tries to use them. With unversioned shared libraries, the
only way old and new apps can exist on the same system is if these
libraries are installed to different locations and the ELF headers are
manually hard-coded to look in one place and not the other for the correct
library against which to link. What about configuring and compiling apps
that want to use this library? The ./configure step will succeed (because
it would look for libtss.so and find it) but the actual compile-time
linking would fail because, inevitably, symbols won't be found or resolved.
Never mind what will happen at runtime :-)

These conventions (that library.so.N is what should be provided and linked
against and that library.so is just a developer crutch for development time
only) are very old and ingrained in the way all Linux distributions package
libraries.

Unless, of course, you want to take on the task of finding and updating
every single application that ever links to your library every time there
is an API change, for the rest of eternity ;-)

On Fri, Jun 1, 2018 at 12:51 PM, Fuchs, Andreas <
andreas.fuchs(a)sit.fraunhofer.de> wrote:

> P.S. or should we just install everything as .so.0 and merely never
> imagine to have a .so.1 file for
> any tcti ever ?
>
> I'm not that familiar with conventions here.
> ------------------------------
> *From:* Fuchs, Andreas
> *Sent:* Friday, June 01, 2018 18:49
> *To:* Trevor Woerner; Philip Tricca
> *Cc:* tpm2(a)lists.01.org
> *Subject:* RE: [tpm2] dlopen()'ing raw .so files considered bad
>
> I just wanted to make sure we're not doing the same work twice.
>
> The (one) symbol Tss2_Tcti_GetInfo() exported by TCTI-so-files is fixed.
> The data structures
> returned by this (the info itself and the function lookup table) are
> versioned in their first field.
>
> The purpose of this was to not require so-file versioning.
>
> Given this background, would you agree that we should not even start
> introducing versioned
> so-file (e.g. tss2_tcti_socket.so.0) but only ever deploy unversioned
> so-files (tss2_tcti_socket.so).
>
>
> ------------------------------
> *From:* tpm2 [tpm2-bounces(a)lists.01.org] on behalf of Trevor Woerner [
> twoerner(a)gmail.com]
> *Sent:* Friday, June 01, 2018 02:15
> *To:* Philip Tricca
> *Cc:* tpm2(a)lists.01.org
> *Subject:* Re: [tpm2] dlopen()'ing raw .so files considered bad
>
> Thank you. I had been working on this for part of the afternoon and wasn't
> successful, but I wanted to raise the flag sooner rather than later.
>
> I had tried William's patch in isolation, but found more was needed. I was
> going to dive into it harder tomorrow.
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 6260 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-06-12 11:43 Trevor Woerner
  0 siblings, 0 replies; 9+ messages in thread
From: Trevor Woerner @ 2018-06-12 11:43 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

Thank you everyone!! With the latest master, the .so handling is great:

	# tpm2_pcrlist -v
	tool="tpm2_pcrlist" version="3.0.2-537-ge25e9ee" tctis="dynamic" tcti-default=tabrmd dlclose=enabled

	# /etc/init.d/tpm2-abrmd start
	Starting TCG TSS2 Access Broker and Resource Management daemon: tpm2-abrmd.

	# tpm2_pcrlist 
	sha1:
	  0 : 0x0000000000000000000000000000000000000000
	  1 : 0x0000000000000000000000000000000000000000
	  2 : 0x0000000000000000000000000000000000000000
	[...]

	# /etc/init.d/tpm2-abrmd stop
	Stopping TCG TSS2 Access Broker and Resource Management daemon: tpm2-abrmd.

	# tpm2_pcrlist -T device:/dev/tpm0
	sha1:
	  0 : 0x0000000000000000000000000000000000000000
	  1 : 0x0000000000000000000000000000000000000000
	  2 : 0x0000000000000000000000000000000000000000
	[...]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-06-02  7:39 Fuchs, Andreas
  0 siblings, 0 replies; 9+ messages in thread
From: Fuchs, Andreas @ 2018-06-02  7:39 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 5375 bytes --]

Sorry I guess I was not clear enough. I understand and totally agree with everything you say here. And we want this for all of our "regular" libraries.

It's just the tcti at the time was considered its own (special) beast. That's why we only have 1 symbol exported and why the data-type is versioned. The thought was that we want to have interchangable IPC/RPC layer and that tcti-implementations could be provided at any later time independently of the rest of the tss. Similar to OpenSSL engines or PKCS11 .so files. So we can consider tcti libraries to actually never change.

Given this, what would you consider the best approach ? Version them anyway is was I understand from this ?

________________________________
From: Trevor Woerner [twoerner(a)gmail.com]
Sent: Saturday, June 02, 2018 04:19
To: Fuchs, Andreas
Cc: Philip Tricca; tpm2(a)lists.01.org
Subject: Re: [tpm2] dlopen()'ing raw .so files considered bad

It seems odd for this project would say "we don't want to have to deal with versioning". Aren't we now, as we speak, witnessing a non-backwards-compatible API version break with TSS? How are users supposed to deal with this? There are existing apps that expect the old TSS and they will never be updated. Now the TSS is changing in such a way that all those programs will break if they try to link to and run with the new shared libraries. If those libraries aren't versioned, how are they, or the runtime linker supposed to know what to do?

If everyone followed convention, the old apps would look for and runtime-link against, say, libtss.so.0. Now that the API has changed, the new library should be called libtss.so.1. New apps can link against this new library, and everything will work. Distributions are welcome to provide the old TSS library, or the new one, or both, or neither. Applications that aren't updated and expect the old API will continue to runtime-link against libtss.so.0, and new apps or old apps that are updated will runtime-link against libtss.so.1. No conflicts, no issues.

The "shared library hell" occurs when all versioning is removed and everyone assumes there's just one, unversioned library against which to link: libtss.so. How will distros provide support for old and new apps? How will developers ever know which one is installed?

Versioned libraries are not a new idea. Unversioned shared-object files will make it difficult for distributions to package these libraries, to the point where they might decide to not bother trying to package them at all. Unversioned shared-objects will always be a continual source of instability for anything that tries to use them. With unversioned shared libraries, the only way old and new apps can exist on the same system is if these libraries are installed to different locations and the ELF headers are manually hard-coded to look in one place and not the other for the correct library against which to link. What about configuring and compiling apps that want to use this library? The ./configure step will succeed (because it would look for libtss.so and find it) but the actual compile-time linking would fail because, inevitably, symbols won't be found or resolved. Never mind what will happen at runtime :-)

These conventions (that library.so.N is what should be provided and linked against and that library.so is just a developer crutch for development time only) are very old and ingrained in the way all Linux distributions package libraries.

Unless, of course, you want to take on the task of finding and updating every single application that ever links to your library every time there is an API change, for the rest of eternity ;-)

On Fri, Jun 1, 2018 at 12:51 PM, Fuchs, Andreas <andreas.fuchs(a)sit.fraunhofer.de<mailto:andreas.fuchs(a)sit.fraunhofer.de>> wrote:
P.S. or should we just install everything as .so.0 and merely never imagine to have a .so.1 file for
any tcti ever ?

I'm not that familiar with conventions here.
________________________________
From: Fuchs, Andreas
Sent: Friday, June 01, 2018 18:49
To: Trevor Woerner; Philip Tricca
Cc: tpm2(a)lists.01.org<mailto:tpm2(a)lists.01.org>
Subject: RE: [tpm2] dlopen()'ing raw .so files considered bad

I just wanted to make sure we're not doing the same work twice.

The (one) symbol Tss2_Tcti_GetInfo() exported by TCTI-so-files is fixed. The data structures
returned by this (the info itself and the function lookup table) are versioned in their first field.

The purpose of this was to not require so-file versioning.

Given this background, would you agree that we should not even start introducing versioned
so-file (e.g. tss2_tcti_socket.so.0) but only ever deploy unversioned so-files (tss2_tcti_socket.so).


________________________________
From: tpm2 [tpm2-bounces(a)lists.01.org<mailto:tpm2-bounces(a)lists.01.org>] on behalf of Trevor Woerner [twoerner(a)gmail.com<mailto:twoerner(a)gmail.com>]
Sent: Friday, June 01, 2018 02:15
To: Philip Tricca
Cc: tpm2(a)lists.01.org<mailto:tpm2(a)lists.01.org>
Subject: Re: [tpm2] dlopen()'ing raw .so files considered bad

Thank you. I had been working on this for part of the afternoon and wasn't successful, but I wanted to raise the flag sooner rather than later.

I had tried William's patch in isolation, but found more was needed. I was going to dive into it harder tomorrow.


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 7981 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-06-01 16:51 Fuchs, Andreas
  0 siblings, 0 replies; 9+ messages in thread
From: Fuchs, Andreas @ 2018-06-01 16:51 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]

P.S. or should we just install everything as .so.0 and merely never imagine to have a .so.1 file for
any tcti ever ?

I'm not that familiar with conventions here.
________________________________
From: Fuchs, Andreas
Sent: Friday, June 01, 2018 18:49
To: Trevor Woerner; Philip Tricca
Cc: tpm2(a)lists.01.org
Subject: RE: [tpm2] dlopen()'ing raw .so files considered bad

I just wanted to make sure we're not doing the same work twice.

The (one) symbol Tss2_Tcti_GetInfo() exported by TCTI-so-files is fixed. The data structures
returned by this (the info itself and the function lookup table) are versioned in their first field.

The purpose of this was to not require so-file versioning.

Given this background, would you agree that we should not even start introducing versioned
so-file (e.g. tss2_tcti_socket.so.0) but only ever deploy unversioned so-files (tss2_tcti_socket.so).


________________________________
From: tpm2 [tpm2-bounces(a)lists.01.org] on behalf of Trevor Woerner [twoerner(a)gmail.com]
Sent: Friday, June 01, 2018 02:15
To: Philip Tricca
Cc: tpm2(a)lists.01.org
Subject: Re: [tpm2] dlopen()'ing raw .so files considered bad

Thank you. I had been working on this for part of the afternoon and wasn't successful, but I wanted to raise the flag sooner rather than later.

I had tried William's patch in isolation, but found more was needed. I was going to dive into it harder tomorrow.

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 2688 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-06-01 16:49 Fuchs, Andreas
  0 siblings, 0 replies; 9+ messages in thread
From: Fuchs, Andreas @ 2018-06-01 16:49 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

I just wanted to make sure we're not doing the same work twice.

The (one) symbol Tss2_Tcti_GetInfo() exported by TCTI-so-files is fixed. The data structures
returned by this (the info itself and the function lookup table) are versioned in their first field.

The purpose of this was to not require so-file versioning.

Given this background, would you agree that we should not even start introducing versioned
so-file (e.g. tss2_tcti_socket.so.0) but only ever deploy unversioned so-files (tss2_tcti_socket.so).


________________________________
From: tpm2 [tpm2-bounces(a)lists.01.org] on behalf of Trevor Woerner [twoerner(a)gmail.com]
Sent: Friday, June 01, 2018 02:15
To: Philip Tricca
Cc: tpm2(a)lists.01.org
Subject: Re: [tpm2] dlopen()'ing raw .so files considered bad

Thank you. I had been working on this for part of the afternoon and wasn't successful, but I wanted to raise the flag sooner rather than later.

I had tried William's patch in isolation, but found more was needed. I was going to dive into it harder tomorrow.

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 1878 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-06-01  0:15 Trevor Woerner
  0 siblings, 0 replies; 9+ messages in thread
From: Trevor Woerner @ 2018-06-01  0:15 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 264 bytes --]

Thank you. I had been working on this for part of the afternoon and wasn't
successful, but I wanted to raise the flag sooner rather than later.

I had tried William's patch in isolation, but found more was needed. I was
going to dive into it harder tomorrow.

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 383 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-05-31 23:18 Philip Tricca
  0 siblings, 0 replies; 9+ messages in thread
From: Philip Tricca @ 2018-05-31 23:18 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 2305 bytes --]

On Thu, May 31, 2018 at 09:12:45PM +0000, Roberts, William C wrote:
> 
> 
> > -----Original Message-----
> > From: tpm2 [mailto:tpm2-bounces(a)lists.01.org] On Behalf Of Trevor Woerner
> > Sent: Thursday, May 31, 2018 1:47 PM
> > To: tpm2(a)lists.01.org
> > Subject: [tpm2] dlopen()'ing raw .so files considered bad
> > 
> > Hey everyone,
> > 
> > I was playing around with the latest tpm2 tools (i.e. the git master) and I noticed
> > that this code is now dlopen()'ing the libraries at runtime.
> > 
> > That's a very nice feature, but the way it's been implemented is not entirely
> > correct.
> > 
> > The filename parameter to dlopen() should be a library's SONAME, not the raw
> > .so file. The raw .so file is considered a -dev or -devel package component, not a
> > component of the library package itself. The raw .so exists as a develop crutch,
> > but shouldn't be used in production.
> > 
> > The raw .so will always point to the latest shared library object. But in an ideal
> > world, the shared library's API should be denoted by the first number after ".so.".
> > It would be extremely unlikely that the programs linking to today's TSS (with its
> > current API) would continue to work should an API-breaking change occur in the
> > future (thus requiring the first number after ".so." to be incremented).
> 
> Per the spec we work on, we can't break it.
> 
> > 
> > In other words, programs should dlopen() a specific API (e.g. ".so.0") and not just
> > "the latest" (i.e. ".so").
> 
> Good to know, well make this change in the tools:
> https://github.com/tpm2-software/tpm2-tools/pull/1052

Have one up for tabrmd as well:
https://github.com/tpm2-software/tpm2-abrmd/pull/471

Philip

> > 
> > Please see https://lists.debian.org/debian-policy/2002/12/msg00041.html, for
> > example.
> > 
> > As is, this is going to mess up people packaging this software for various distros,
> > and users trying to use this software (since they'll have to install the -dev
> > packages for it to work, which can get messy for non-developer users).
> > 
> > 
> > Thanks and best regards,
> >     Trevor
> 
> _______________________________________________
> tpm2 mailing list
> tpm2(a)lists.01.org
> https://lists.01.org/mailman/listinfo/tpm2

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-05-31 21:12 Roberts, William C
  0 siblings, 0 replies; 9+ messages in thread
From: Roberts, William C @ 2018-05-31 21:12 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 1902 bytes --]



> -----Original Message-----
> From: tpm2 [mailto:tpm2-bounces(a)lists.01.org] On Behalf Of Trevor Woerner
> Sent: Thursday, May 31, 2018 1:47 PM
> To: tpm2(a)lists.01.org
> Subject: [tpm2] dlopen()'ing raw .so files considered bad
> 
> Hey everyone,
> 
> I was playing around with the latest tpm2 tools (i.e. the git master) and I noticed
> that this code is now dlopen()'ing the libraries at runtime.
> 
> That's a very nice feature, but the way it's been implemented is not entirely
> correct.
> 
> The filename parameter to dlopen() should be a library's SONAME, not the raw
> .so file. The raw .so file is considered a -dev or -devel package component, not a
> component of the library package itself. The raw .so exists as a develop crutch,
> but shouldn't be used in production.
> 
> The raw .so will always point to the latest shared library object. But in an ideal
> world, the shared library's API should be denoted by the first number after ".so.".
> It would be extremely unlikely that the programs linking to today's TSS (with its
> current API) would continue to work should an API-breaking change occur in the
> future (thus requiring the first number after ".so." to be incremented).

Per the spec we work on, we can't break it.

> 
> In other words, programs should dlopen() a specific API (e.g. ".so.0") and not just
> "the latest" (i.e. ".so").

Good to know, well make this change in the tools:
https://github.com/tpm2-software/tpm2-tools/pull/1052

> 
> Please see https://lists.debian.org/debian-policy/2002/12/msg00041.html, for
> example.
> 
> As is, this is going to mess up people packaging this software for various distros,
> and users trying to use this software (since they'll have to install the -dev
> packages for it to work, which can get messy for non-developer users).
> 
> 
> Thanks and best regards,
>     Trevor


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [tpm2] dlopen()'ing raw .so files considered bad
@ 2018-05-31 20:46 Trevor Woerner
  0 siblings, 0 replies; 9+ messages in thread
From: Trevor Woerner @ 2018-05-31 20:46 UTC (permalink / raw)
  To: tpm2

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

Hey everyone,

I was playing around with the latest tpm2 tools (i.e. the git master) and I
noticed that this code is now dlopen()'ing the libraries at runtime.

That's a very nice feature, but the way it's been implemented is not
entirely correct.

The filename parameter to dlopen() should be a library's SONAME, not the
raw .so file. The raw .so file is considered a -dev or -devel package
component, not a component of the library package itself. The raw .so
exists as a develop crutch, but shouldn't be used in production.

The raw .so will always point to the latest shared library object. But in
an ideal world, the shared library's API should be denoted by the first
number after ".so.". It would be extremely unlikely that the programs
linking to today's TSS (with its current API) would continue to work should
an API-breaking change occur in the future (thus requiring the first number
after ".so." to be incremented).

In other words, programs should dlopen() a specific API (e.g. ".so.0") and
not just "the latest" (i.e. ".so").

Please see https://lists.debian.org/debian-policy/2002/12/msg00041.html,
for example.

As is, this is going to mess up people packaging this software for various
distros, and users trying to use this software (since they'll have to
install the -dev packages for it to work, which can get messy for
non-developer users).

Thanks and best regards,
    Trevor

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 1801 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2018-06-12 11:43 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-02  2:19 [tpm2] dlopen()'ing raw .so files considered bad Trevor Woerner
  -- strict thread matches above, loose matches on Subject: below --
2018-06-12 11:43 Trevor Woerner
2018-06-02  7:39 Fuchs, Andreas
2018-06-01 16:51 Fuchs, Andreas
2018-06-01 16:49 Fuchs, Andreas
2018-06-01  0:15 Trevor Woerner
2018-05-31 23:18 Philip Tricca
2018-05-31 21:12 Roberts, William C
2018-05-31 20:46 Trevor Woerner

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.