Re: [dpdk-techboard] DPDK ABI/API Stability

[Ray Kinsella <mdr@ashroe.eu>]

On 04/04/2019 13:02, Luca Boccassi wrote:
> On Thu, 2019-04-04 at 11:54 +0100, Bruce Richardson wrote:
>> On Thu, Apr 04, 2019 at 10:29:19AM +0100, Burakov, Anatoly wrote:
>>> On 03-Apr-19 4:42 PM, Ray Kinsella wrote:
[SNIP]
>>>
>>
>> Actually, I think we *do* need to constrain the pace of development
>> for the
>> sake of ABI stability. At this stage DPDK has been around for quite a
>> number of years and so should be considered a fairly mature project -
>> it
>> should just start acting like it.
>>
>> Now, in terms of features like the memory rework, that is indeed a
>> case
>> that there was no alternative other than a massive ABI break.
>> However, for
>> that rework there was a strong need for improvement in that area that
>> we
>> can make the case for an ABI break to support it - and it is of a
>> scale
>> that nothing other than an ABI change would do. For other areas and
>> examples, I doubt there are many in the last couple of years that are
>> of
>> that scale.
> 
> Fully agree.
> 
> It's normal for new project, big and small, to start without a
> stability promise as development ramps up, and then steer toward
> stability as the user base grows. Sometimes the switch is made explicit
> by crossing from a 0.x to a 1.x version numbering, sometimes it's not.
> I don't think we crossed that boundary yet in this project, and I
> believe that's the main question Ray was trying to raise: has the time
> finally come for DPDK to do this phase shift?

Yes - we never had a 1.0, where we cut an API and stood behind it
similar to GStreamer. There a number of reasons why this happened not
worth going into, however you make the point very well Luca - this phase
shift is long over due.

> 
> Of course it comes with a price for all developers, and that's again
> common.

Agreed - nothing is for free.
The question is this something we value and is it something worth doing.

> 
>> My thoughts on the matter are:
>> 1. I think we really need to do work to start hiding more of our data
>> structures - like what Stephen's latest RFC does. This hiding should
>> reduce
>> the scope for ABI breaks.
> 
> Yes, I'm a big fan of accessors and opaque structs.

+1, me too.

> 
>> 2. Once done, I think we should commit to having an ABI break only in
>> the
>> rarest of circumstances, and only with very large justification. I
>> want us
>> to get to the point where DPDK releases can immediately be picked up
>> by all
>> linux distros and rolled out because they are ABI compatible.
>>
>> I'm not sure I like the idea of planned ABI break releases - that
>> strikes
>> me as a plan where we end up with the same number of ABI breaks as
>> before,
>> just balled into one release.
> 
> I think that was intended as a compromise, especially as we move from
> one model to the other, and more of a "if a breakage has to happen, it
> must be in the X release" rather than "let's always break in the X
> release" :-)
> 
>> Question for Kevin, Luca and others who look at distro-packaging: is
>> it the
>> case that each distro will only ship one version of DPDK, or is it
>> possible
>> that if we have ABI breaks, a distro will provide two copies of DPDK
>> simultaneously, e.g. a 19.11 ABI version and a 20.11 ABI version?
> 
> We can ship multiple versions, although it's more work so there should
> be a good reason to do it. 

Well you already kind of do right.
You ship 16.11.8 with Debian 9 and then 18.11 with Debian 9 backports.

> At the moment in Debian and Ubuntu we don't,
> and we tend to ship whatever the latest LTS version is at the distro
> freeze milestone - for example Debian 10 which will be released soon
> (TM) will have 18.11.0.

Presumably when 19.11 arrives, it will land in Debian 10 backports
similarly.

I assume anything that lands in backports is not guaranteed to be ABI
compatible with stable?

> 
>> So, in short, I'm generally in favour of a zero-tolerance approach
>> for DPDK
>> ABI breaks, and having ABI breaks as a major event reserved only for
>> massive rework changes, such as major mbuf changes, or new memory
>> layout or
>> similar.
>>
>> Regards,
>> /Bruce
>>