Re: [PATCH 0/4] Add DSO symbol versioning to support backwards compatibility

["Richardson, Bruce" <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>]

> -----Original Message-----
> From: Neil Horman [mailto:nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org]
> Sent: Thursday, September 18, 2014 8:14 PM
> To: Thomas Monjalon
> Cc: dev-VfR2kkLFssw@public.gmane.org; Richardson, Bruce
> Subject: Re: [PATCH 0/4] Add DSO symbol versioning to support backwards
> compatibility
> 
> On Thu, Sep 18, 2014 at 08:23:36PM +0200, Thomas Monjalon wrote:
> > Hi Neil,
> >
> > 2014-09-15 15:23, Neil Horman:
> > > The DPDK ABI develops and changes quickly, which makes it difficult for
> > > applications to keep up with the latest version of the library, especially when
> > > it (the DPDK) is built as a set of shared objects, as applications may be built
> > > against an older version of the library.
> > >
> > > To mitigate this, this patch series introduces support for library and symbol
> > > versioning when the DPDK is built as a DSO.  Specifically, it does 4 things:
> > >
> > > 1) Adds initial support for library versioning.  Each library now has a version
> > > map that explicitly calls out what symbols are exported to using applications,
> > > and assigns version(s) to them
> > >
> > > 2) Adds support macros so that when libraries create incompatible ABI's,
> > > multiple versions may be supported so that applications linked against older
> > > DPDK releases can continue to function
> > >
> > > 3) Adds library soname versioning suffixes so that when ABI's must be broken
> in
> > > a fashion that requires a rebuild of older applications, they will break at load
> > > time, rather than cause unexpected issues at run time.
> > >
> > > 4) Adds documentation for ABI policy, and provides space to document
> deprecated
> > > ABI versions, so that applications might be warned of impending changes.
> > >
> > > With these elements in place the DPDK has some support to allow for the
> extended
> > > maintenence of older API's while still allowing the freedom to develop new
> and
> > > improved API's.
> > >
> > > Implementing this feature will require some additional effort on the part of
> > > developers and reviewers.  When reviewing patches, must be checked
> against
> > > existing exports to ensure that the function prototypes are not changing.  If
> > > they are, the versioning macros must be used, and the library export map
> should
> > > be updated to reflect the new version of the function.
> > >
> > > When data structures change, if those structures are application accessible,
> > > apis that accept or return instances of those data structures should have new
> > > versions created so that users of the old data structure version might co-
> exist
> > > at the same time.
> >
> > Thanks for your efforts.
> > But I feel this change has too many constraints for the current status of
> > the DPDK. It's probably too early to adopt such policy.
> >
> I think you may be misunderstanding something.  What constraints do you
> beleive
> that this patch imposes?  Note it doesn't in any way prevent changes to the ABI
> of the DPDK, but rather gives us infrastructure to support multiple ABI
> revisions at the same time, so that applications built against DPDK shared
> libraries can continue to function properly at least for some time until we
> decide to deprecate that ABI level.
> 

I view all this as a positive step. I consider backward compatibility as something that should always be encouraged, and I agree with Neil that this should allow us to guarantee compatibility for our customers while still having a path open to us to change things if we really need to.

> This is all based on the versioning strategy outlined here:
> http://www.akkadia.org/drepper/dsohowto.pdf
> 
> That may help clarify things for you.
> 
> > By the way, this versioning doesn't cover structure changes.
> No, it doesn't.  No link-time mechanism does so.
> 
> > How could it be managed?
> Thats a subject that is open to discussion, but my initial thinking is that we
> need to handle it on a case by case basis:
> 
> * For minor updates, where allocation of a structure is done on the heap and
> new
> fields need to be added, appending them to the end of a structure and providing
> an initial value is sufficient.
> 
> * For major changes, where fields need to be removed, or re-arranged, mostly
> likely the API surfaces which accept or return those structures as
> inputs/outputs will need to have new versions written to accept the new version
> of the structure, and internally we will have to support both formats for a time
> (according to the policy I documented, that is currently a single major
> release).  I.e. if you want to change struct foo, which is accepted as a
> parameter for the function bar(struct foo *ptr), then for a release we would
> need to create struct foo_v2 with the new format, map a new function foo_v2
> to
> the exported foo@@DPDK_1.(X+1), and internally make the foo functions
> understand
> both the origional and v2 versions of the structure.  Then in DPDK release
> 1.X+2, we can remove the old version after posting a deprecation notice with
> version 1.(X+1)

I really, really like having an official deprecation policy. The one proposed seems reasonable as a start point - we can always decide later whether we want a 1, 2 or 3 release gap between marking something as deprecated and having it removed.

/Bruce