[dpdk-dev] DPDK Release Status Meeting 16/01/2020

[dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Ferruh Yigit]

Minutes 16 January 2020
-----------------------

Agenda:
* Release Dates
* Subtrees
* OvS

Participants:
* Debian/Microsoft
* Intel
* Marvell
* Mellanox
* NXP
* Red Hat


Release Dates
-------------

* v20.02 dates:
  * Integration deadline passed, it was on Wednesday 15 January 2020
    * RC1 is expected before the weekend, possibly on Friday 17 January
    * PRC holidays on 24-20 Jan (inclusive)
  * Release:			Friday 14 February 2020

* v20.05 proposal:
  * Proposal/V1:		Friday 6 March 2020
  * Integration/Merge/RC1:	Friday 10 April 2020
  * Release:			Friday 1 May || Wed 13 May
    * 1-5 May holiday on PRC, we need to do the release before or after that
      Please comment on between 1 May or 13 May.


Subtrees
--------

* main
  * Will try to do -rc1 before Monday with best effort
  * Two series from arm
    * "ring library" updates
      * May go in in this release
    * Use WFE for aarch64
  * ABI tooling patches
    * Tech-board selected David's approach
    * Automated checks will be in CI after David's patch
      * We need automated checks to be sure we are not breaking the ABI
        otherwise it is not always easy for developer to catch the break
    * Waiting new version of Neil's __rte_internal pathcset

* next-net
  * Almost all ethdev changes merged, planning to finalize last
    remaining 1-2 ones today
  * All sub-trees pulled except some mlx and brcm one that are waiting fix
  * Some PMD patches may be postponed to -rc2

* next-net-crypto
  * Pull request sent
  * There is a performance concern on some ipsec-gw patches,
    they can go in -rc2 if the issue is solved
  * CPU crypto from last release may be breaking ABI, need to confirm
    and discussed dummy variable is missing, may be postponed to next release
  * Some PMD changes are postponed to -rc2

* next-net-eventdev
  * Pull request sent, it is a small set

* next-net-virtio
  * All commit pulled to next-net
  * virtio vDPA drive will be postponed to next release
  * Some patches waiting user change and some are under review
  * Can be some more patches for -rc2

* next-net-intel
  * Nothing critical, most of the patches have been pulled
  * 'fm10k' patches are not reviewed, may go in -rc2 if reviewed
    otherwise will be postponed to next release

* LTS

  * 17.11.10-rc1 released, please test and report results
    * https://mails.dpdk.org/archives/dev/2020-January/154915.html
    * Release planned on January / February

  * 18.11.6-rc2 released, please test and report results
    * https://mails.dpdk.org/archives/dev/2020-January/155132.html
    * Only a few patches on top of -rc1
    * Intel will plan a validation, it won't be full validation
    * Release planned on 31st January


OvS
---

* Using DPDK experimental APIs discussed, it is accepted to use once as
  exception, so feature won't be blocked
  * DPDK need to figure out how to backport removing experimental tags to the
    LTS releases



DPDK Release Status Meetings
============================

The DPDK Release Status Meeting is intended for DPDK Committers to discuss
the status of the master tree and sub-trees, and for project managers to
track progress or milestone dates.

The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just
send an email to "John McNamara <john.mcnamara@intel.com>" for the invite.

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Ananyev, Konstantin]

Hi everyone,

> 
> * next-net-crypto
>   * Pull request sent
>   * There is a performance concern on some ipsec-gw patches,
>     they can go in -rc2 if the issue is solved
>   * CPU crypto from last release may be breaking ABI, need to confirm

AFAIK, there is no ABI breakage.

>     and discussed dummy variable is missing, may be postponed to next release

Not sure I understand last sentence, could the author explain
what dummy variables we are talking about.

Konstantin

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Akhil Goyal]

Hi Konstantin,

> Hi everyone,
> 
> >
> > * next-net-crypto
> >   * Pull request sent
> >   * There is a performance concern on some ipsec-gw patches,
> >     they can go in -rc2 if the issue is solved
> >   * CPU crypto from last release may be breaking ABI, need to confirm
> 
> AFAIK, there is no ABI breakage.

This is the output of validate-abi.sh.

	Change 							Effect
1 Field sym_cpu_process has been added to this type. 	              1) This field will not be initialized by old clients.
                                                                                                                   2) Size of the inclusive type has been changed.

								NOTE: this field should be accessed only from the new library functions, otherwise it may result in crash or incorrect behavior of applications.
2 Size of this type has been changed from 128 bytes to 136 bytes. 	The fields or parameters of such data type may be incorrectly initialized or accessed by old client applications.

Apart from that, IPSEC also has breakage, but that is experimental, so not an issue.

> 
> >     and discussed dummy variable is missing, may be postponed to next release
> 
> Not sure I understand last sentence, could the author explain
> what dummy variables we are talking about.

In one of the techboard meeting around 19.11 timeframe, during the discussion for approving methodology for CPU-crypto, it was proposed that in order to avoid delay, a dummy variable can be introduced in cryptodev API/ABI to avoid any ABI breakage in upcoming releases. But this was not done.


> 
> Konstantin
> 

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Akhil Goyal]

Hi Ferruh,

> 
> * v20.02 dates:
>   * Integration deadline passed, it was on Wednesday 15 January 2020
>     * RC1 is expected before the weekend, possibly on Friday 17 January
>     * PRC holidays on 24-20 Jan (inclusive)
>   * Release:			Friday 14 February 2020
> 
What is the proposed date for RC2?

-Akhil

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Ananyev, Konstantin]

Hi Akhil,

> > >
> > > * next-net-crypto
> > >   * Pull request sent
> > >   * There is a performance concern on some ipsec-gw patches,
> > >     they can go in -rc2 if the issue is solved
> > >   * CPU crypto from last release may be breaking ABI, need to confirm
> >
> > AFAIK, there is no ABI breakage.
> 
> This is the output of validate-abi.sh.
> 
> 	Change 							Effect
> 1 Field sym_cpu_process has been added to this type. 	              1) This field will not be initialized by old clients.
>                                                                                                                    2) Size of the inclusive type has been changed.
> 
> 								NOTE: this field should be accessed only from the new library
> functions, otherwise it may result in crash or incorrect behavior of applications.
> 2 Size of this type has been changed from 128 bytes to 136 bytes. 	The fields or parameters of such data type may be incorrectly
> initialized or accessed by old client applications.

This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
So not sure, why do you consider it as ABI breakage?

> 
> Apart from that, IPSEC also has breakage, but that is experimental, so not an issue.
> 
> >
> > >     and discussed dummy variable is missing, may be postponed to next release
> >
> > Not sure I understand last sentence, could the author explain
> > what dummy variables we are talking about.
> 
> In one of the techboard meeting around 19.11 timeframe, during the discussion for approving methodology for CPU-crypto, it was
> proposed that in order to avoid delay, a dummy variable can be introduced in cryptodev API/ABI to avoid any ABI breakage in
> upcoming releases. But this was not done.

Dummy variable for what?
If you are talking about sym_cpu_process - we just added it into rte_cryptodev_ops, instead of 
' struct rte_cryptodev' instead.
That way we avoid any ABI breakage without introducing any churn in rte_cryptodev itself , see above.
Konstantin

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Akhil Goyal]

Hi Konstantin,

> Hi Akhil,
> 
> > > >
> > > > * next-net-crypto
> > > >   * Pull request sent
> > > >   * There is a performance concern on some ipsec-gw patches,
> > > >     they can go in -rc2 if the issue is solved
> > > >   * CPU crypto from last release may be breaking ABI, need to confirm
> > >
> > > AFAIK, there is no ABI breakage.
> >
> > This is the output of validate-abi.sh.
> >
> > 	Change 							Effect
> > 1 Field sym_cpu_process has been added to this type. 	              1) This field will
> not be initialized by old clients.
> >                                                                                                                    2) Size of the
> inclusive type has been changed.
> >
> > 								NOTE: this
> field should be accessed only from the new library
> > functions, otherwise it may result in crash or incorrect behavior of applications.
> > 2 Size of this type has been changed from 128 bytes to 136 bytes. 	The
> fields or parameters of such data type may be incorrectly
> > initialized or accessed by old client applications.
> 
> This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
> So not sure, why do you consider it as ABI breakage?
> 

If this is not an issue, than I am fine with it.

> >
> > Apart from that, IPSEC also has breakage, but that is experimental, so not an
> issue.
> >
> > >
> > > >     and discussed dummy variable is missing, may be postponed to next
> release
> > >
> > > Not sure I understand last sentence, could the author explain
> > > what dummy variables we are talking about.
> >
> > In one of the techboard meeting around 19.11 timeframe, during the
> discussion for approving methodology for CPU-crypto, it was
> > proposed that in order to avoid delay, a dummy variable can be introduced in
> cryptodev API/ABI to avoid any ABI breakage in
> > upcoming releases. But this was not done.
> 
> Dummy variable for what?
> If you are talking about sym_cpu_process - we just added it into
> rte_cryptodev_ops, instead of
> ' struct rte_cryptodev' instead.
> That way we avoid any ABI breakage without introducing any churn in
> rte_cryptodev itself , see above.

Then why was there so much resistance on this approach when there is no ABI breakage.
I thought there was ABI breakage because of this change.

BTW this patchset is a bit late and it came after merge deadline 15 Jan.
Ideally all library related patches should go in RC1.
I would check if I could make it to the RC2.
I have 3 IPSec series to work on before RC2.

-Akhil

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Ferruh Yigit]

On 1/16/2020 4:42 PM, Akhil Goyal wrote:
> Hi Konstantin,
> 
>> Hi Akhil,
>>
>>>>>
>>>>> * next-net-crypto
>>>>>   * Pull request sent
>>>>>   * There is a performance concern on some ipsec-gw patches,
>>>>>     they can go in -rc2 if the issue is solved
>>>>>   * CPU crypto from last release may be breaking ABI, need to confirm
>>>>
>>>> AFAIK, there is no ABI breakage.
>>>
>>> This is the output of validate-abi.sh.
>>>
>>> 	Change 							Effect
>>> 1 Field sym_cpu_process has been added to this type. 	              1) This field will
>> not be initialized by old clients.
>>>                                                                                                                    2) Size of the
>> inclusive type has been changed.
>>>
>>> 								NOTE: this
>> field should be accessed only from the new library
>>> functions, otherwise it may result in crash or incorrect behavior of applications.
>>> 2 Size of this type has been changed from 128 bytes to 136 bytes. 	The
>> fields or parameters of such data type may be incorrectly
>>> initialized or accessed by old client applications.
>>
>> This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
>> So not sure, why do you consider it as ABI breakage?
>>
> 
> If this is not an issue, than I am fine with it.

The ABI change between cryptodev and PMDs are allowed, that is contained within
DPDK and not a user interface [1].

[1] Unless some inline functions are directly accessing the dev_ops, as
(unfortunately) done in the ethdev.

> 
>>>
>>> Apart from that, IPSEC also has breakage, but that is experimental, so not an
>> issue.
>>>
>>>>
>>>>>     and discussed dummy variable is missing, may be postponed to next
>> release
>>>>
>>>> Not sure I understand last sentence, could the author explain
>>>> what dummy variables we are talking about.
>>>
>>> In one of the techboard meeting around 19.11 timeframe, during the
>> discussion for approving methodology for CPU-crypto, it was
>>> proposed that in order to avoid delay, a dummy variable can be introduced in
>> cryptodev API/ABI to avoid any ABI breakage in
>>> upcoming releases. But this was not done.
>>
>> Dummy variable for what?
>> If you are talking about sym_cpu_process - we just added it into
>> rte_cryptodev_ops, instead of
>> ' struct rte_cryptodev' instead.
>> That way we avoid any ABI breakage without introducing any churn in
>> rte_cryptodev itself , see above.
> 
> Then why was there so much resistance on this approach when there is no ABI breakage.
> I thought there was ABI breakage because of this change.
> 
> BTW this patchset is a bit late and it came after merge deadline 15 Jan.
> Ideally all library related patches should go in RC1.
> I would check if I could make it to the RC2.
> I have 3 IPSec series to work on before RC2.
> 
> -Akhil
> 

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Ferruh Yigit]

On 1/16/2020 1:19 PM, Akhil Goyal wrote:
> Hi Ferruh,
> 
>>
>> * v20.02 dates:
>>   * Integration deadline passed, it was on Wednesday 15 January 2020
>>     * RC1 is expected before the weekend, possibly on Friday 17 January
>>     * PRC holidays on 24-20 Jan (inclusive)
>>   * Release:			Friday 14 February 2020
>>
> What is the proposed date for RC2?
> 

We talked in the meeting most probably there will be two weeks in between but
not set a date, it will be much more clear next week based on -rc1 status.

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Honnappa Nagarahalli]

<snip>
> 
> Minutes 16 January 2020
> -----------------------
> 
> Agenda:
> * Release Dates
> * Subtrees
> * OvS
> 
> Participants:
> * Debian/Microsoft
> * Intel
> * Marvell
> * Mellanox
> * NXP
> * Red Hat
> 
> 
> Release Dates
> -------------
> 
> * v20.02 dates:
>   * Integration deadline passed, it was on Wednesday 15 January 2020
>     * RC1 is expected before the weekend, possibly on Friday 17 January
>     * PRC holidays on 24-20 Jan (inclusive)
>   * Release:			Friday 14 February 2020
> 
> * v20.05 proposal:
>   * Proposal/V1:		Friday 6 March 2020
>   * Integration/Merge/RC1:	Friday 10 April 2020
>   * Release:			Friday 1 May || Wed 13 May
>     * 1-5 May holiday on PRC, we need to do the release before or after that
>       Please comment on between 1 May or 13 May.
> 
> 
> Subtrees
> --------
> 
> * main
>   * Will try to do -rc1 before Monday with best effort
>   * Two series from arm
>     * "ring library" updates
>       * May go in in this release
>     * Use WFE for aarch64
Arm has one more series for replacing Marvell's crypto library. The Aarch64 crypto library needed API renaming, clean up work and tagging for the release. The RFC series will be converted to a patch. The patch itself is not different from the RFC except of API and public structure renaming. The patch should be on the mailing list on 19th (Sunday).

>   * ABI tooling patches
>     * Tech-board selected David's approach
>     * Automated checks will be in CI after David's patch
>       * We need automated checks to be sure we are not breaking the ABI
>         otherwise it is not always easy for developer to catch the break
>     * Waiting new version of Neil's __rte_internal pathcset
> 
> * next-net
>   * Almost all ethdev changes merged, planning to finalize last
>     remaining 1-2 ones today
>   * All sub-trees pulled except some mlx and brcm one that are waiting fix
>   * Some PMD patches may be postponed to -rc2
> 
> * next-net-crypto
>   * Pull request sent
>   * There is a performance concern on some ipsec-gw patches,
>     they can go in -rc2 if the issue is solved
>   * CPU crypto from last release may be breaking ABI, need to confirm
>     and discussed dummy variable is missing, may be postponed to next release
>   * Some PMD changes are postponed to -rc2
> 
> * next-net-eventdev
>   * Pull request sent, it is a small set
> 
> * next-net-virtio
>   * All commit pulled to next-net
>   * virtio vDPA drive will be postponed to next release
>   * Some patches waiting user change and some are under review
>   * Can be some more patches for -rc2
> 
> * next-net-intel
>   * Nothing critical, most of the patches have been pulled
>   * 'fm10k' patches are not reviewed, may go in -rc2 if reviewed
>     otherwise will be postponed to next release
> 
> * LTS
> 
>   * 17.11.10-rc1 released, please test and report results
>     * https://mails.dpdk.org/archives/dev/2020-January/154915.html
>     * Release planned on January / February
> 
>   * 18.11.6-rc2 released, please test and report results
>     * https://mails.dpdk.org/archives/dev/2020-January/155132.html
>     * Only a few patches on top of -rc1
>     * Intel will plan a validation, it won't be full validation
>     * Release planned on 31st January
> 
> 
> OvS
> ---
> 
> * Using DPDK experimental APIs discussed, it is accepted to use once as
>   exception, so feature won't be blocked
>   * DPDK need to figure out how to backport removing experimental tags to the
>     LTS releases
> 
> 
> 
> DPDK Release Status Meetings
> ============================
> 
> The DPDK Release Status Meeting is intended for DPDK Committers to discuss
> the status of the master tree and sub-trees, and for project managers to track
> progress or milestone dates.
> 
> The meeting occurs on Thursdays at 8:30 UTC. If you wish to attend just send
> an email to "John McNamara <john.mcnamara@intel.com>" for the invite.

Re: [dpdk-dev] DPDK Release Status Meeting 16/01/2020

[Ananyev, Konstantin]

Hi lads,

> >>>>> * next-net-crypto
> >>>>>   * Pull request sent
> >>>>>   * There is a performance concern on some ipsec-gw patches,
> >>>>>     they can go in -rc2 if the issue is solved
> >>>>>   * CPU crypto from last release may be breaking ABI, need to confirm
> >>>>
> >>>> AFAIK, there is no ABI breakage.
> >>>
> >>> This is the output of validate-abi.sh.
> >>>
> >>> 	Change 							Effect
> >>> 1 Field sym_cpu_process has been added to this type. 	              1) This field will
> >> not be initialized by old clients.
> >>>                                                                                                                    2) Size of the
> >> inclusive type has been changed.
> >>>
> >>> 								NOTE: this
> >> field should be accessed only from the new library
> >>> functions, otherwise it may result in crash or incorrect behavior of applications.
> >>> 2 Size of this type has been changed from 128 bytes to 136 bytes. 	The
> >> fields or parameters of such data type may be incorrectly
> >>> initialized or accessed by old client applications.
> >>
> >> This is struct rte_cryptodev_ops, which is AFAIK, not part of public API.
> >> So not sure, why do you consider it as ABI breakage?
> >>
> >
> > If this is not an issue, than I am fine with it.
> 
> The ABI change between cryptodev and PMDs are allowed, that is contained within
> DPDK and not a user interface [1].
> 
> [1] Unless some inline functions are directly accessing the dev_ops, as
> (unfortunately) done in the ethdev.

Thanks Ferruh for confirmation.
For cryptodev we don't have such inline functions, plus the we add
new entry at the bottom of struct rte_cryptodev_ops, so I believe we
are safe here.

> 
> >
> >>>
> >>> Apart from that, IPSEC also has breakage, but that is experimental, so not an
> >> issue.
> >>>
> >>>>
> >>>>>     and discussed dummy variable is missing, may be postponed to next
> >> release
> >>>>
> >>>> Not sure I understand last sentence, could the author explain
> >>>> what dummy variables we are talking about.
> >>>
> >>> In one of the techboard meeting around 19.11 timeframe, during the
> >> discussion for approving methodology for CPU-crypto, it was
> >>> proposed that in order to avoid delay, a dummy variable can be introduced in
> >> cryptodev API/ABI to avoid any ABI breakage in
> >>> upcoming releases. But this was not done.
> >>
> >> Dummy variable for what?
> >> If you are talking about sym_cpu_process - we just added it into
> >> rte_cryptodev_ops, instead of
> >> ' struct rte_cryptodev' instead.
> >> That way we avoid any ABI breakage without introducing any churn in
> >> rte_cryptodev itself , see above.
> >
> > Then why was there so much resistance on this approach when there is no ABI breakage.
> > I thought there was ABI breakage because of this change.
> >
> > BTW this patchset is a bit late and it came after merge deadline 15 Jan.
> > Ideally all library related patches should go in RC1.
> > I would check if I could make it to the RC2.
> > I have 3 IPSec series to work on before RC2.
> >