Re: 17.02 Roadmap

Re: 17.02 Roadmap

[O'Driscoll, Tim]

We published our initial roadmap for 17.02 at the end of August. Since then we've been doing more detailed planning and would like to provide an update on the features that we plan to submit for this release. This is our current plan, which should hopefully remain fairly stable now:

Consistent Filter API: Add support for the Consistent Filter API (see http://dpdk.org/ml/archives/dev/2016-September/047924.html) for IGB, IXGBE and I40E.

Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-based load balancing library which scales linearly for both lookup and insert with the number of threads or cores.  EFD lookup uses a "perfect hashing" scheme where only the information needed to compute a key's value (and not the key itself) is stored in the lookup table, thus reducing CPU cache storage requirements. 

Extended Stats (Latency and Bit Rate Statistics): Enhance the Extended NIC Stats (Xstats) implementation to support the collection and reporting of latency and bit rate measurements. Latency statistics will include min, max and average latency, and jitter. Bit rate statistics will include peak and average bit rate aggregated over a user-defined time period. This will be implemented for IXGBE and I40E.

Run-Time Configuration of Packet Type (PTYPE) for I40E: At the moment all packet types in DPDK are statically defined. This makes impossible to add new values without first defining them statically and then recompiling DPDK. The ability to configure packet types at run time will be added for I40E.

Packet Distributor Enhancements: Enhancements will be made to the Packet Distributor library to improve performance:
1. Introduce burst functionality to allow batches of packets to be sent to workers.
2. Improve the performance of the flow/core affinity through the use of SSE/AVX instructions.

Add MACsec for IXGBE: MACsec support will be added for IXGBE. Ethdev API primitives will be added to create/delete/enable/disable SC/SA, Next_PN etc. similar to those used in Linux for the macsec_ops. Sample apps (l3fwd, testpmd, etc.) will be updated to support MACsec for the IXGBE. 

Enhance AESNI_GCM PMD: The current AESNI_GCM PMD is limited to AES-128 and does not support other features such as "any AAD length value". It will be updated to use a newer GCM implementation supporting AES128/192/256 and other features.

Create Crypto Performance Test App: A new app, similar to testpmd, will be created to allow crypto performance to be tested using any crypto PMD and any supported crypto algorithm.

Enable Cipher-Only and Hash-Only Support in AESNI_MB PMD: Support will be added for cipher-only and hash-only operations in the AESNI_MB PMD.

Support Chained Mbufs in Cryptodev: Currently, an application using the cryptodev API needs to reserve a continuous block of memory for mbufs. Support will be added for chaining of mbufs in both the QAT and SW PMDs supported by cryptodev.

Optimize Vhost-User Performance for Large Packets: A new memory copy function optimized for core-to-core memory copy which will be added. This will be beneficial for virtualization cases involving large packets, but it can be used for other core-to-core cases as well.

Support New Device Types in Vhost-User: Support will be added to vhost-user for new device types including vhost-scsi and vhost-blk.

Interrupt Mode Support in Virtio PMD: Support for interrupt mode will be added to the virtio PMD.

Virtio-User as an Alternative Exception Path: Investigate the use of virtio-user and vhost-net as an alternative exception path to KNI that does not require out of tree drivers. This work is still at an experimental stage, so it may not be included in 17.02.


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of O'Driscoll, Tim
> Sent: Wednesday, August 31, 2016 11:32 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] 17.02 Roadmap
> 
> Below are the features that we're planning to submit for the 17.02
> release. We'll submit a patch to update the roadmap page with this info.
> 
> Some things will obviously change during planning/development, so we'll
> provide a more detailed update in late September/early October. After
> that, things should hopefully be relatively stable.
> 
> It would be good if others are also willing to share their plans so that
> we can build up a complete picture of what's planned for 17.02 and make
> sure there's no duplication.
> 
> 
> Consistent Filter API phase 2: Extend support for the Consistent Filter
> API that will be first implemented in 16.11 to IGB and FM10K.
> 
> Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-
> based load balancing library which scales linearly for both lookup and
> insert with the number of threads or cores.  EFD lookup uses a "perfect
> hashing" scheme where only the information needed to compute a key's
> value (and not the key itself) is stored in the lookup table, thus
> reducing CPU cache storage requirements.
> 
> Cryptodev: Additional Software Algorithms:
> - Optimize the AES-GCM software PMD.
> - Enhance the Intel(r) AES-NI MB PMD to add support for cipher-only and
> authentication-only operations. Add support for AES_CBC_MAC,
> AES_CMAC_128, AES_GMAC_128.
> - Add software support for: 3DES_ECB_128/192, AES_ECB_128/192/256,
> AES_F8, AES_XTS, ARC4.
> 
> Run-Time Configuration of Flow Type (PCTYPE) and Packet Type (PTYPE) for
> I40E: At the moment all flow types and packet types in DPDK are
> statically defined. This makes impossible to add new values without
> first defining them statically and then recompiling DPDK. The ability to
> configure flow types and packet types at run time will be added for
> I40E.
> 
> Extended Stats Latency and Bit Rate Statistics: Enhance the Extended NIC
> Stats (Xstats) implementation to support the collection and reporting of
> latency and bit rate measurements. Latency statistics will include min,
> max and average latency, and jitter. Bit rate statistics will include
> peak and average bit rate aggregated over a user-defined time period.
> This will be implemented for IXGBE and I40E.
> 
> Hardware QoS for IXGBE and I40E: Implement support for Hardware QoS for
> IXGBE and I40E. This involves using DCB so that a PF or VF can receive
> packets for each of the Traffic Classes (TCs) based on the User Priority
> field (in the VLAN header). Bandwidth on transmit for each TC is
> programmable.

Re: 17.02 Roadmap

[Thomas Monjalon]

Thanks Tim for the interesting inputs.
Some of them may require a dedicated thread to continue the discussion
based on some preliminary specifications or drafts.

2016-10-10 16:13, O'Driscoll, Tim:
> Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-based load balancing library which scales linearly for both lookup and insert with the number of threads or cores.  EFD lookup uses a "perfect hashing" scheme where only the information needed to compute a key's value (and not the key itself) is stored in the lookup table, thus reducing CPU cache storage requirements.

What is the scope of this library? Just apply rte_hash to a flow table?
Or is it also sending the packets in some queues?
Does it depend of librte_distributor?

> Extended Stats (Latency and Bit Rate Statistics): Enhance the Extended NIC Stats (Xstats) implementation to support the collection and reporting of latency and bit rate measurements. Latency statistics will include min, max and average latency, and jitter. Bit rate statistics will include peak and average bit rate aggregated over a user-defined time period. This will be implemented for IXGBE and I40E.

Are they retrieved from hardware or just computed in software?
Could we have some drivers hook to compute them in a generic layer?

> Run-Time Configuration of Packet Type (PTYPE) for I40E: At the moment all packet types in DPDK are statically defined. This makes impossible to add new values without first defining them statically and then recompiling DPDK. The ability to configure packet types at run time will be added for I40E.

The packet types are part of the API.
Although not yet convinced by the packet types, I do not understand how
to benefit from some run-time defined packet types.

> Packet Distributor Enhancements: Enhancements will be made to the Packet Distributor library to improve performance:
> 1. Introduce burst functionality to allow batches of packets to be sent to workers.
> 2. Improve the performance of the flow/core affinity through the use of SSE/AVX instructions.

The packet distributor looks more and more like a DPDK application
at the same level of BESS, VPP, OVS, etc.

> Add MACsec for IXGBE: MACsec support will be added for IXGBE. Ethdev API primitives will be added to create/delete/enable/disable SC/SA, Next_PN etc. similar to those used in Linux for the macsec_ops. Sample apps (l3fwd, testpmd, etc.) will be updated to support MACsec for the IXGBE. 

How the ethdev interface and the cryptodev capabilities will be linked for MACsec?

[...]
> Create Crypto Performance Test App: A new app, similar to testpmd, will be created to allow crypto performance to be tested using any crypto PMD and any supported crypto algorithm.

Good idea :)
When I read "testpmd", I tend to think that it could test any PMD,
including crypto. Are you saying that we should read dpdk-netdev-test?
And you would introduce dpdk-cryptodev-test?

[...]
> Optimize Vhost-User Performance for Large Packets: A new memory copy function optimized for core-to-core memory copy which will be added. This will be beneficial for virtualization cases involving large packets, but it can be used for other core-to-core cases as well.

Is it an enhancement of rte_memcpy or something else?

> Support New Device Types in Vhost-User: Support will be added to vhost-user for new device types including vhost-scsi and vhost-blk.

Great!
Is it only related to networking or should we expect some storage-related
code or drivers to come up?

> Interrupt Mode Support in Virtio PMD: Support for interrupt mode will be added to the virtio PMD.

I guess you mean Rx interrupt in virtio PMD to avoid 100% polling in the VM?

> Virtio-User as an Alternative Exception Path: Investigate the use of virtio-user and vhost-net as an alternative exception path to KNI that does not require out of tree drivers. This work is still at an experimental stage, so it may not be included in 17.02.

Interesting. Please share more details of the design you are thinking of.

Re: 17.02 Roadmap

[Yuanhan Liu]

On Mon, Oct 10, 2016 at 10:42:58PM +0200, Thomas Monjalon wrote:
> > Support New Device Types in Vhost-User: Support will be added to vhost-user for new device types including vhost-scsi and vhost-blk.
> 
> Great!
> Is it only related to networking or should we expect some storage-related
> code or drivers to come up?

It's not only netowrking related. It just introduces few more APIs to
export those buf infos (virt addr, phys addr, len, etc) to applications,
so that they can handle/parse the data in the way they want. For example,
for SPDK (https://01.org/spdk), they can use those APIs to fetch guest
data and parse it following virtio-SCSI spec.

The dequeue path (guest Tx) might look like something below:

	rte_vhost_dequeue_vring_desc_burst(vid, queue_id, iov, count)
	{
		for (i = 0; i < count; i++) {
			desc_idx = get_next_desc();

			iov[i]->addr      = desc_virt_addr(desc[desc_idx]->addr);
			iov[i]->phys_addr = desc_phys_addr(desc[desc_idx]->addr);
			iov[i]->len       = desc[desc_idx]->len;
			iov[i]->desc      = desc_idx;
		}

		return i;
	}

	rte_vhost_update_used_ring(vid, queue_id, descs, count)
	{
		for (i = 0; i < count; i++) {
			used_idx = get_next_used_idx();
			vq->used->ring[used_idx] = descs[i];
		}

		vq->used->idx += i;
	}

And we introduce similar APIs to the enqueue path.

	--yliu

Re: 17.02 Roadmap

[O'Driscoll, Tim]

> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> 
> Thanks Tim for the interesting inputs.
> Some of them may require a dedicated thread to continue the discussion
> based on some preliminary specifications or drafts.

Agreed. To avoid me acting as a middle man, I'm going to ask the individual developers to respond initially to this thread. If more discussion is required on some of the items, then we should create separate threads.

Re: 17.02 Roadmap

[Pattan, Reshma]

Hi Thomas,


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Monday, October 10, 2016 9:43 PM
> To: O'Driscoll, Tim <tim.odriscoll@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] 17.02 Roadmap
> 
> Thanks Tim for the interesting inputs.
> Some of them may require a dedicated thread to continue the discussion based
> on some preliminary specifications or drafts.
> 
> 2016-10-10 16:13, O'Driscoll, Tim:
> > Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-based load
> balancing library which scales linearly for both lookup and insert with the
> number of threads or cores.  EFD lookup uses a "perfect hashing" scheme where
> only the information needed to compute a key's value (and not the key itself) is
> stored in the lookup table, thus reducing CPU cache storage requirements.
> 
> What is the scope of this library? Just apply rte_hash to a flow table?
> Or is it also sending the packets in some queues?
> Does it depend of librte_distributor?
> 
> > Extended Stats (Latency and Bit Rate Statistics): Enhance the Extended NIC
> Stats (Xstats) implementation to support the collection and reporting of latency
> and bit rate measurements. Latency statistics will include min, max and average
> latency, and jitter. Bit rate statistics will include peak and average bit rate
> aggregated over a user-defined time period. This will be implemented for IXGBE
> and I40E.
> 
> Are they retrieved from hardware or just computed in software?

Computed in software. 

> Could we have some drivers hook to compute them in a generic layer?
Since more stats are coming into DPDK , we planned to avoid adding new code into ethdev library. So adding couple of new libraries to deal with future stats.
1)There will be a new stats library which will provide APIs like, stats registration,  stats update and get stats .

2) Another latency stats, bitrate library. This new library uses rte_eth rx/tx callbacks  to mark  Rx timestamp in Rx callback and calculate latency in Tx callback.

The new stats library(1)  shall be used by latency stats , bit rate stats (2) and further new stats to register themselves and push their values to stats library. 
Stats library get APIs will be called by existing xstats get API of the ethdev library  to display all new stats as part of xstats.

Thanks,
Reshma

Re: 17.02 Roadmap

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Monday, October 10, 2016 1:43 PM
> To: O'Driscoll, Tim
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] 17.02 Roadmap
> 
> Thanks Tim for the interesting inputs.
> Some of them may require a dedicated thread to continue the discussion
> based on some preliminary specifications or drafts.
> 
> 2016-10-10 16:13, O'Driscoll, Tim:
> > Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-based
> load balancing library which scales linearly for both lookup and insert with
> the number of threads or cores.  EFD lookup uses a "perfect hashing" scheme
> where only the information needed to compute a key's value (and not the
> key itself) is stored in the lookup table, thus reducing CPU cache storage
> requirements.
> 
> What is the scope of this library? Just apply rte_hash to a flow table?
> Or is it also sending the packets in some queues?
> Does it depend of librte_distributor?

This is a going to be flow-level load balancer/filter based on perfect hashing,
but it doesn't store the key as a regular hash table.
It won't be an extension to the distributor library.

Will send an RFC soon (next week), where we can continue the discussion,
with code, documentation and sample app included.

> 
> > Extended Stats (Latency and Bit Rate Statistics): Enhance the Extended NIC
> Stats (Xstats) implementation to support the collection and reporting of
> latency and bit rate measurements. Latency statistics will include min, max
> and average latency, and jitter. Bit rate statistics will include peak and
> average bit rate aggregated over a user-defined time period. This will be
> implemented for IXGBE and I40E.
> 
> Are they retrieved from hardware or just computed in software?
> Could we have some drivers hook to compute them in a generic layer?
> 
> > Run-Time Configuration of Packet Type (PTYPE) for I40E: At the moment all
> packet types in DPDK are statically defined. This makes impossible to add
> new values without first defining them statically and then recompiling DPDK.
> The ability to configure packet types at run time will be added for I40E.
> 
> The packet types are part of the API.
> Although not yet convinced by the packet types, I do not understand how
> to benefit from some run-time defined packet types.
> 
> > Packet Distributor Enhancements: Enhancements will be made to the
> Packet Distributor library to improve performance:
> > 1. Introduce burst functionality to allow batches of packets to be sent to
> workers.
> > 2. Improve the performance of the flow/core affinity through the use of
> SSE/AVX instructions.
> 
> The packet distributor looks more and more like a DPDK application
> at the same level of BESS, VPP, OVS, etc.
> 
> > Add MACsec for IXGBE: MACsec support will be added for IXGBE. Ethdev API
> primitives will be added to create/delete/enable/disable SC/SA, Next_PN etc.
> similar to those used in Linux for the macsec_ops. Sample apps (l3fwd,
> testpmd, etc.) will be updated to support MACsec for the IXGBE.
> 
> How the ethdev interface and the cryptodev capabilities will be linked for
> MACsec?
> 
> [...]
> > Create Crypto Performance Test App: A new app, similar to testpmd, will be
> created to allow crypto performance to be tested using any crypto PMD and
> any supported crypto algorithm.
> 
> Good idea :)
> When I read "testpmd", I tend to think that it could test any PMD,
> including crypto. Are you saying that we should read dpdk-netdev-test?
> And you would introduce dpdk-cryptodev-test?
> 
> [...]
> > Optimize Vhost-User Performance for Large Packets: A new memory copy
> function optimized for core-to-core memory copy which will be added. This
> will be beneficial for virtualization cases involving large packets, but it can be
> used for other core-to-core cases as well.
> 
> Is it an enhancement of rte_memcpy or something else?
> 
> > Support New Device Types in Vhost-User: Support will be added to vhost-
> user for new device types including vhost-scsi and vhost-blk.
> 
> Great!
> Is it only related to networking or should we expect some storage-related
> code or drivers to come up?
> 
> > Interrupt Mode Support in Virtio PMD: Support for interrupt mode will be
> added to the virtio PMD.
> 
> I guess you mean Rx interrupt in virtio PMD to avoid 100% polling in the VM?
> 
> > Virtio-User as an Alternative Exception Path: Investigate the use of virtio-
> user and vhost-net as an alternative exception path to KNI that does not
> require out of tree drivers. This work is still at an experimental stage, so it
> may not be included in 17.02.
> 
> Interesting. Please share more details of the design you are thinking of.

Re: 17.02 Roadmap

[Tan, Jianfeng]

Hi Thomas,


On 10/11/2016 4:42 AM, Thomas Monjalon wrote:
> Thanks Tim for the interesting inputs.
> Some of them may require a dedicated thread to continue the discussion
> based on some preliminary specifications or drafts.
[...]
>> Interrupt Mode Support in Virtio PMD: Support for interrupt mode will be added to the virtio PMD.
> I guess you mean Rx interrupt in virtio PMD to avoid 100% polling in the VM?

Yes, it is. Previously, Stephen Hemminger has worked out a version which 
requires MSI support in uio driver, which is not upstreamed. And as 
Stephen said here 
(http://dpdk.org/ml/archives/dev/2015-December/030913.html), we will go 
with VFIO with no-IOMMU. And this dependence has been addressed in Linux 
4.8,
033291eccbd ("vfio: Include No-IOMMU mode").

Hi Stephen,

Sorry that I did not sync up with you about this topic before it's sent 
out. Have you already started the work on this topic? Or I'll work on 
this and ask your review?

>
>> Virtio-User as an Alternative Exception Path: Investigate the use of virtio-user and vhost-net as an alternative exception path to KNI that does not require out of tree drivers. This work is still at an experimental stage, so it may not be included in 17.02.
> Interesting. Please share more details of the design you are thinking of.
>
Currently, virtio-user just has the vhost-user backend (and the 
vhost-kernel path has been dropped before merge). The path of 
vhost-kernel has been dropped for bad performance comparing to 
vhost-user. And we want keep the code as simple as possible.

But after a second thought, virtio_user + vhost-kernel is a good 
candidate for exceptional path.

1. maintenance: vhost-net (kernel) is upstreamed and extensively used 
kernel module. We don't need any out-of-tree module like KNI.
2. performance: this solution would have at least similar performance 
with KNI.
3. feature: multi queue, tso, multi-seg mbuf, etc.

Any suggestions?

Thanks,
Jianfeng

Re: 17.02 Roadmap

[Hunt, David]

Hi Thomas,


On 10/10/2016 9:42 PM, Thomas Monjalon wrote:
> Thanks Tim for the interesting inputs.
> Some of them may require a dedicated thread to continue the discussion
> based on some preliminary specifications or drafts.
>
> 2016-10-10 16:13, O'Driscoll, Tim:
>> Packet Distributor Enhancements: Enhancements will be made to the Packet Distributor library to improve performance:
>> 1. Introduce burst functionality to allow batches of packets to be sent to workers.
>> 2. Improve the performance of the flow/core affinity through the use of SSE/AVX instructions.
> The packet distributor looks more and more like a DPDK application
> at the same level of BESS, VPP, OVS, etc.

Lets discuss this further.  Would you see other libraries heading in 
this direction also (reorder, acl, hash, etc)?
Do you think it would be an idea to add as an item of discussion for the 
technical steering group when we're all at Userspace next week?

Regards,
Dave.

Re: 17.02 Roadmap

[Thomas Monjalon]

2016-10-13 15:18, Hunt, David:
> Hi Thomas,
> 
> On 10/10/2016 9:42 PM, Thomas Monjalon wrote:
> > 2016-10-10 16:13, O'Driscoll, Tim:
> >> Packet Distributor Enhancements: Enhancements will be made to the Packet Distributor library to improve performance:
> >> 1. Introduce burst functionality to allow batches of packets to be sent to workers.
> >> 2. Improve the performance of the flow/core affinity through the use of SSE/AVX instructions.
> > 
> > The packet distributor looks more and more like a DPDK application
> > at the same level of BESS, VPP, OVS, etc.
> 
> Lets discuss this further.  Would you see other libraries heading in 
> this direction also (reorder, acl, hash, etc)?

Not so easy to put things in a category.

> Do you think it would be an idea to add as an item of discussion for the 
> technical steering group when we're all at Userspace next week?

If others are interested to discuss about libraries categories and growth,
yes I'll jump in.

Re: 17.02 Roadmap

[Stephen Hemminger]

On Mon, 10 Oct 2016 16:13:42 +0000
"O'Driscoll, Tim" <tim.odriscoll@intel.com> wrote:

> We published our initial roadmap for 17.02 at the end of August. Since then we've been doing more detailed planning and would like to provide an update on the features that we plan to submit for this release. This is our current plan, which should hopefully remain fairly stable now:
> 
> Consistent Filter API: Add support for the Consistent Filter API (see http://dpdk.org/ml/archives/dev/2016-September/047924.html) for IGB, IXGBE and I40E.
> 
> Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-based load balancing library which scales linearly for both lookup and insert with the number of threads or cores.  EFD lookup uses a "perfect hashing" scheme where only the information needed to compute a key's value (and not the key itself) is stored in the lookup table, thus reducing CPU cache storage requirements. 
> 
> Extended Stats (Latency and Bit Rate Statistics): Enhance the Extended NIC Stats (Xstats) implementation to support the collection and reporting of latency and bit rate measurements. Latency statistics will include min, max and average latency, and jitter. Bit rate statistics will include peak and average bit rate aggregated over a user-defined time period. This will be implemented for IXGBE and I40E.
> 
> Run-Time Configuration of Packet Type (PTYPE) for I40E: At the moment all packet types in DPDK are statically defined. This makes impossible to add new values without first defining them statically and then recompiling DPDK. The ability to configure packet types at run time will be added for I40E.
> 
> Packet Distributor Enhancements: Enhancements will be made to the Packet Distributor library to improve performance:
> 1. Introduce burst functionality to allow batches of packets to be sent to workers.
> 2. Improve the performance of the flow/core affinity through the use of SSE/AVX instructions.
> 
> Add MACsec for IXGBE: MACsec support will be added for IXGBE. Ethdev API primitives will be added to create/delete/enable/disable SC/SA, Next_PN etc. similar to those used in Linux for the macsec_ops. Sample apps (l3fwd, testpmd, etc.) will be updated to support MACsec for the IXGBE. 
> 
> Enhance AESNI_GCM PMD: The current AESNI_GCM PMD is limited to AES-128 and does not support other features such as "any AAD length value". It will be updated to use a newer GCM implementation supporting AES128/192/256 and other features.
> 
> Create Crypto Performance Test App: A new app, similar to testpmd, will be created to allow crypto performance to be tested using any crypto PMD and any supported crypto algorithm.
> 
> Enable Cipher-Only and Hash-Only Support in AESNI_MB PMD: Support will be added for cipher-only and hash-only operations in the AESNI_MB PMD.
> 
> Support Chained Mbufs in Cryptodev: Currently, an application using the cryptodev API needs to reserve a continuous block of memory for mbufs. Support will be added for chaining of mbufs in both the QAT and SW PMDs supported by cryptodev.
> 
> Optimize Vhost-User Performance for Large Packets: A new memory copy function optimized for core-to-core memory copy which will be added. This will be beneficial for virtualization cases involving large packets, but it can be used for other core-to-core cases as well.
> 
> Support New Device Types in Vhost-User: Support will be added to vhost-user for new device types including vhost-scsi and vhost-blk.
> 
> Interrupt Mode Support in Virtio PMD: Support for interrupt mode will be added to the virtio PMD.
> 
> Virtio-User as an Alternative Exception Path: Investigate the use of virtio-user and vhost-net as an alternative exception path to KNI that does not require out of tree drivers. This work is still at an experimental stage, so it may not be included in 17.02.
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of O'Driscoll, Tim
> > Sent: Wednesday, August 31, 2016 11:32 AM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] 17.02 Roadmap
> > 
> > Below are the features that we're planning to submit for the 17.02
> > release. We'll submit a patch to update the roadmap page with this info.
> > 
> > Some things will obviously change during planning/development, so we'll
> > provide a more detailed update in late September/early October. After
> > that, things should hopefully be relatively stable.
> > 
> > It would be good if others are also willing to share their plans so that
> > we can build up a complete picture of what's planned for 17.02 and make
> > sure there's no duplication.
> > 
> > 
> > Consistent Filter API phase 2: Extend support for the Consistent Filter
> > API that will be first implemented in 16.11 to IGB and FM10K.
> > 
> > Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-
> > based load balancing library which scales linearly for both lookup and
> > insert with the number of threads or cores.  EFD lookup uses a "perfect
> > hashing" scheme where only the information needed to compute a key's
> > value (and not the key itself) is stored in the lookup table, thus
> > reducing CPU cache storage requirements.
> > 
> > Cryptodev: Additional Software Algorithms:
> > - Optimize the AES-GCM software PMD.
> > - Enhance the Intel(r) AES-NI MB PMD to add support for cipher-only and
> > authentication-only operations. Add support for AES_CBC_MAC,
> > AES_CMAC_128, AES_GMAC_128.
> > - Add software support for: 3DES_ECB_128/192, AES_ECB_128/192/256,
> > AES_F8, AES_XTS, ARC4.
> > 
> > Run-Time Configuration of Flow Type (PCTYPE) and Packet Type (PTYPE) for
> > I40E: At the moment all flow types and packet types in DPDK are
> > statically defined. This makes impossible to add new values without
> > first defining them statically and then recompiling DPDK. The ability to
> > configure flow types and packet types at run time will be added for
> > I40E.
> > 
> > Extended Stats Latency and Bit Rate Statistics: Enhance the Extended NIC
> > Stats (Xstats) implementation to support the collection and reporting of
> > latency and bit rate measurements. Latency statistics will include min,
> > max and average latency, and jitter. Bit rate statistics will include
> > peak and average bit rate aggregated over a user-defined time period.
> > This will be implemented for IXGBE and I40E.
> > 
> > Hardware QoS for IXGBE and I40E: Implement support for Hardware QoS for
> > IXGBE and I40E. This involves using DCB so that a PF or VF can receive
> > packets for each of the Traffic Classes (TCs) based on the User Priority
> > field (in the VLAN header). Bandwidth on transmit for each TC is
> > programmable.  
> 

It seems like a lot of these feature are focused too narrowly on exposing
features that exist on specific Intel hardware. The concept of a general
purpose Dataplane Development Kit is that applications can be written that
have a generic API (like any operating system) that will run on a wide
variety of hardware.  This concept seems to getting lost as the DPDK is
becoming more of a platform for exposing what ever cool hardware features
exist.

I would propose that no new feature be allowed in the DPDK unless it
can be supported on all device types. Yes that means you have to build
and test software emulation layers for all other devices. The current
model is more of a hardware test bed.

Re: 17.02 Roadmap

[Thomas Monjalon]

2016-10-14 10:29, Stephen Hemminger:
> It seems like a lot of these feature are focused too narrowly on exposing
> features that exist on specific Intel hardware. The concept of a general
> purpose Dataplane Development Kit is that applications can be written that
> have a generic API (like any operating system) that will run on a wide
> variety of hardware.  This concept seems to getting lost as the DPDK is
> becoming more of a platform for exposing what ever cool hardware features
> exist.
> 
> I would propose that no new feature be allowed in the DPDK unless it
> can be supported on all device types. Yes that means you have to build
> and test software emulation layers for all other devices. The current
> model is more of a hardware test bed.

Thanks for the reminder Stephen. It is good goal.
I think the software emulation idea is finding its way.
About forbidding new hardware feature without emulation support,
it has to be discussed.

Re: 17.02 Roadmap

[O'Driscoll, Tim]

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Friday, October 14, 2016 9:19 PM
> To: Stephen Hemminger <stephen@networkplumber.org>
> Cc: dev@dpdk.org; O'Driscoll, Tim <tim.odriscoll@intel.com>
> Subject: Re: [dpdk-dev] 17.02 Roadmap
> 
> 2016-10-14 10:29, Stephen Hemminger:
> > It seems like a lot of these feature are focused too narrowly on
> exposing
> > features that exist on specific Intel hardware. The concept of a
> general
> > purpose Dataplane Development Kit is that applications can be written
> that
> > have a generic API (like any operating system) that will run on a wide
> > variety of hardware.  This concept seems to getting lost as the DPDK
> is
> > becoming more of a platform for exposing what ever cool hardware
> features
> > exist.
> >
> > I would propose that no new feature be allowed in the DPDK unless it
> > can be supported on all device types. Yes that means you have to build
> > and test software emulation layers for all other devices. The current
> > model is more of a hardware test bed.
> 
> Thanks for the reminder Stephen. It is good goal.
> I think the software emulation idea is finding its way.
> About forbidding new hardware feature without emulation support,
> it has to be discussed.

We have a roadmap discussion on Thursday morning at the Userspace event in
Dublin. The aim is to discuss any gaps in the roadmap and see what can be 
done to address them. I think this would be a great topic to discuss then.

Re: 17.02 Roadmap

[Thomas Monjalon]

2016-08-31 10:31, O'Driscoll, Tim:
> Below are the features that we're planning to submit for the 17.02
> release. We'll submit a patch to update the roadmap page with this info.
> 
> Some things will obviously change during planning/development, so we'll
> provide a more detailed update in late September/early October.
> After that, things should hopefully be relatively stable.
> 
> It would be good if others are also willing to share their plans so
> that we can build up a complete picture of what's planned for 17.02
> and make sure there's no duplication.

Thanks Tim. I would like to stress that it would be really better to
know the intents of the other contributors.

17.02 Roadmap

[O'Driscoll, Tim]

Below are the features that we're planning to submit for the 17.02 release. We'll submit a patch to update the roadmap page with this info.

Some things will obviously change during planning/development, so we'll provide a more detailed update in late September/early October. After that, things should hopefully be relatively stable.

It would be good if others are also willing to share their plans so that we can build up a complete picture of what's planned for 17.02 and make sure there's no duplication.


Consistent Filter API phase 2: Extend support for the Consistent Filter API that will be first implemented in 16.11 to IGB and FM10K.

Elastic Flow Distributor: The Elastic Flow Distributor (EFD) is a flow-based load balancing library which scales linearly for both lookup and insert with the number of threads or cores.  EFD lookup uses a "perfect hashing" scheme where only the information needed to compute a key's value (and not the key itself) is stored in the lookup table, thus reducing CPU cache storage requirements.

Cryptodev: Additional Software Algorithms: 
- Optimize the AES-GCM software PMD.
- Enhance the Intel(r) AES-NI MB PMD to add support for cipher-only and authentication-only operations. Add support for AES_CBC_MAC, AES_CMAC_128, AES_GMAC_128.
- Add software support for: 3DES_ECB_128/192, AES_ECB_128/192/256, AES_F8, AES_XTS, ARC4.

Run-Time Configuration of Flow Type (PCTYPE) and Packet Type (PTYPE) for I40E: At the moment all flow types and packet types in DPDK are statically defined. This makes impossible to add new values without first defining them statically and then recompiling DPDK. The ability to configure flow types and packet types at run time will be added for I40E.

Extended Stats Latency and Bit Rate Statistics: Enhance the Extended NIC Stats (Xstats) implementation to support the collection and reporting of latency and bit rate measurements. Latency statistics will include min, max and average latency, and jitter. Bit rate statistics will include peak and average bit rate aggregated over a user-defined time period. This will be implemented for IXGBE and I40E.

Hardware QoS for IXGBE and I40E: Implement support for Hardware QoS for IXGBE and I40E. This involves using DCB so that a PF or VF can receive packets for each of the Traffic Classes (TCs) based on the User Priority field (in the VLAN header). Bandwidth on transmit for each TC is programmable.