[GIT net-next] Open vSwitch

[GIT net-next] Open vSwitch

[Pravin B Shelar]

First two patches are related to OVS MPLS support. Rest of patches
are various refactoring and minor improvements to openvswitch.

----------------------------------------------------------------

The following changes since commit 30349bdbc4da5ecf0efa25556e3caff9c9b8c5f7:

  net: phy: spi_ks8995: remove sysfs bin file by registered attribute (2014-11-04 17:18:45 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git net_next_ovs

for you to fetch changes up to fb90c8d8d5169d4dfbe5896721367e1904638a91:

  openvswitch: Avoid NULL mask check while building mask (2014-11-04 22:20:33 -0800)

----------------------------------------------------------------
Andy Zhou (2):
      openvswitch: refactor do_output() to move NULL check out of fast path
      openvswitch: Refactor get_dp() function into multiple access APIs.

Chunhe Li (1):
      openvswitch: Drop packets when interdev is not up

Jarno Rajahalme (1):
      openvswitch: Fix the type of struct ovs_key_nd nd_target field.

Jesse Gross (1):
      openvswitch: Additional logging for -EINVAL on flow setups.

Joe Stringer (3):
      openvswitch: Remove redundant tcp_flags code.
      openvswitch: Refactor ovs_flow_cmd_fill_info().
      openvswitch: Move key_attr_size() to flow_netlink.h.

Lorand Jakab (1):
      openvswitch: Remove flow member from struct ovs_skb_cb

Pravin B Shelar (4):
      net: Remove MPLS GSO feature.
      openvswitch: Move table destroy to dp-rcu callback.
      openvswitch: Refactor action alloc and copy api.
      openvswitch: Avoid NULL mask check while building mask

Simon Horman (1):
      openvswitch: Add basic MPLS support to kernel

 include/linux/netdev_features.h      |   7 +-
 include/linux/netdevice.h            |   1 -
 include/linux/skbuff.h               |   3 -
 include/net/mpls.h                   |  39 +++++
 include/uapi/linux/openvswitch.h     |  38 ++++-
 net/core/dev.c                       |   3 +-
 net/core/ethtool.c                   |   1 -
 net/ipv4/af_inet.c                   |   1 -
 net/ipv4/tcp_offload.c               |   1 -
 net/ipv4/udp_offload.c               |   3 +-
 net/ipv6/ip6_offload.c               |   1 -
 net/ipv6/udp_offload.c               |   3 +-
 net/mpls/mpls_gso.c                  |   3 +-
 net/openvswitch/Kconfig              |   1 +
 net/openvswitch/actions.c            | 136 ++++++++++++---
 net/openvswitch/datapath.c           | 215 ++++++++++++-----------
 net/openvswitch/datapath.h           |   4 +-
 net/openvswitch/flow.c               |  30 ++++
 net/openvswitch/flow.h               |  17 +-
 net/openvswitch/flow_netlink.c       | 322 +++++++++++++++++++++++++----------
 net/openvswitch/flow_netlink.h       |   5 +-
 net/openvswitch/flow_table.c         |  11 +-
 net/openvswitch/flow_table.h         |   2 +-
 net/openvswitch/vport-internal_dev.c |   5 +
 24 files changed, 606 insertions(+), 246 deletions(-)
 create mode 100644 include/net/mpls.h

Re: [GIT net-next] Open vSwitch

[David Miller]

Please do not submit your patches such that the email Date: field is
the commit's date.  You're not posting these on Nov. 4th, yet that
is the Date: field on all of the individual patch emails.

I want them to be the date at the time you post the patch to the mailing
list.

Otherwise the ordering in patchwork is not cronological wrt. the list's
postings and this makes my work more difficult than it needs to be.

Thanks.

Re: [GIT net-next] Open vSwitch

[Pravin Shelar]

On Wed, Nov 5, 2014 at 12:10 PM, David Miller <davem@davemloft.net> wrote:
>
> Please do not submit your patches such that the email Date: field is
> the commit's date.  You're not posting these on Nov. 4th, yet that
> is the Date: field on all of the individual patch emails.
>
> I want them to be the date at the time you post the patch to the mailing
> list.
>
> Otherwise the ordering in patchwork is not cronological wrt. the list's
> postings and this makes my work more difficult than it needs to be.
>
Sorry about the Date field. NTP stopped working on my machine thats
why the date got messed up.

[GIT net-next] Open vSwitch

[Pravin B Shelar]

Following batch of patches brings feature parity between upstream
ovs and out of tree ovs module.

Two features are added, first adds support to export egress
tunnel information for a packet. This is used to improve
visibility in network traffic. Second feature allows userspace
vswitchd process to probe ovs module features. Other patches
are optimization and code cleanup.

----------------------------------------------------------------
The following changes since commit c0560b9c523341516eabf0f3b51832256caa7bbb:

  dccp: Convert DCCP_WARN to net_warn_ratelimited (2014-11-08 21:22:54 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git net_next_ovs

for you to fetch changes up to 05da5898a96c05e32aa9850c9cd89eef29471b13:

  openvswitch: Add support for OVS_FLOW_ATTR_PROBE. (2014-11-09 18:58:44 -0800)

----------------------------------------------------------------
Jarno Rajahalme (1):
      openvswitch: Add support for OVS_FLOW_ATTR_PROBE.

Pravin B Shelar (3):
      openvswitch: Export symbols as GPL symbols.
      openvswitch: Optimize recirc action.
      openvswitch: Remove redundant key ref from upcall_info.

Thomas Graf (1):
      openvswitch: Constify various function arguments

Wenyu Zhang (1):
      openvswitch: Extend packet attribute for egress tunnel info

 include/uapi/linux/openvswitch.h |  15 ++
 net/openvswitch/actions.c        | 180 ++++++++++++++------
 net/openvswitch/datapath.c       | 129 ++++++++------
 net/openvswitch/datapath.h       |  22 +--
 net/openvswitch/flow.c           |   8 +-
 net/openvswitch/flow.h           |  71 ++++++--
 net/openvswitch/flow_netlink.c   | 357 +++++++++++++++++++++++----------------
 net/openvswitch/flow_netlink.h   |  13 +-
 net/openvswitch/flow_table.c     |  12 +-
 net/openvswitch/flow_table.h     |   8 +-
 net/openvswitch/vport-geneve.c   |  23 ++-
 net/openvswitch/vport-gre.c      |  12 +-
 net/openvswitch/vport-netdev.c   |   2 +-
 net/openvswitch/vport-vxlan.c    |  24 ++-
 net/openvswitch/vport.c          |  81 +++++++--
 net/openvswitch/vport.h          |  20 ++-
 16 files changed, 664 insertions(+), 313 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Pravin B Shelar <pshelar@nicira.com>
Date: Sun,  9 Nov 2014 19:58:59 -0800

> Following batch of patches brings feature parity between upstream
> ovs and out of tree ovs module.
> 
> Two features are added, first adds support to export egress
> tunnel information for a packet. This is used to improve
> visibility in network traffic. Second feature allows userspace
> vswitchd process to probe ovs module features. Other patches
> are optimization and code cleanup.

Pulled, thanks Pravin.

[GIT net-next] Open vSwitch

[Pravin B Shelar]

Following patches adds recirculation and hash action to OVS.
First three patches does code restructuring which is required
for last patch.
Recirculation implementation is changed, according to comments from
David Miller, to avoid using recursive calls in OVS. It is using
queue to record recirc action and deferred recirc is executed at
the end of current actions execution.

----------------------------------------------------------------
The following changes since commit b954d83421d51d822c42e5ab7b65069b25ad3005:

  net: bpf: only build bpf_jit_binary_{alloc, free}() when jit selected (2014-09-10 14:05:07 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git net_next_ovs

for you to fetch changes up to 9b8ede54a8bd319789e8bceb19789463bb944701:

  openvswitch: Add recirc and hash action. (2014-09-11 13:35:29 -0700)

----------------------------------------------------------------
Andy Zhou (2):
      datapath: simplify sample action implementation
      openvswitch: Add recirc and hash action.

Pravin B Shelar (2):
      datapath: refactor ovs flow extract API.
      datapath: Use tun_key only for egress tunnel path.

 include/uapi/linux/openvswitch.h |  26 +++++
 net/openvswitch/actions.c        | 247 ++++++++++++++++++++++++++++++++++-----
 net/openvswitch/datapath.c       |  52 +++++----
 net/openvswitch/datapath.h       |  17 ++-
 net/openvswitch/flow.c           |  54 +++++++--
 net/openvswitch/flow.h           |  10 +-
 net/openvswitch/flow_netlink.c   |  63 +++++++---
 net/openvswitch/flow_netlink.h   |   4 +-
 net/openvswitch/vport-gre.c      |  22 ++--
 net/openvswitch/vport-vxlan.c    |  20 ++--
 net/openvswitch/vport.c          |  13 ++-
 11 files changed, 419 insertions(+), 109 deletions(-)

Re: [GIT net-next] Open vSwitch

[Pravin Shelar]

Please ignore this series, I have used datapath as subsystem name
rather than openvswitch. I will respin it shortly.

On Thu, Sep 11, 2014 at 3:01 PM, Pravin B Shelar <pshelar@nicira.com> wrote:
> Following patches adds recirculation and hash action to OVS.
> First three patches does code restructuring which is required
> for last patch.
> Recirculation implementation is changed, according to comments from
> David Miller, to avoid using recursive calls in OVS. It is using
> queue to record recirc action and deferred recirc is executed at
> the end of current actions execution.
>
> ----------------------------------------------------------------
> The following changes since commit b954d83421d51d822c42e5ab7b65069b25ad3005:
>
>   net: bpf: only build bpf_jit_binary_{alloc, free}() when jit selected (2014-09-10 14:05:07 -0700)
>
> are available in the git repository at:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git net_next_ovs
>
> for you to fetch changes up to 9b8ede54a8bd319789e8bceb19789463bb944701:
>
>   openvswitch: Add recirc and hash action. (2014-09-11 13:35:29 -0700)
>
> ----------------------------------------------------------------
> Andy Zhou (2):
>       datapath: simplify sample action implementation
>       openvswitch: Add recirc and hash action.
>
> Pravin B Shelar (2):
>       datapath: refactor ovs flow extract API.
>       datapath: Use tun_key only for egress tunnel path.
>
>  include/uapi/linux/openvswitch.h |  26 +++++
>  net/openvswitch/actions.c        | 247 ++++++++++++++++++++++++++++++++++-----
>  net/openvswitch/datapath.c       |  52 +++++----
>  net/openvswitch/datapath.h       |  17 ++-
>  net/openvswitch/flow.c           |  54 +++++++--
>  net/openvswitch/flow.h           |  10 +-
>  net/openvswitch/flow_netlink.c   |  63 +++++++---
>  net/openvswitch/flow_netlink.h   |   4 +-
>  net/openvswitch/vport-gre.c      |  22 ++--
>  net/openvswitch/vport-vxlan.c    |  20 ++--
>  net/openvswitch/vport.c          |  13 ++-
>  11 files changed, 419 insertions(+), 109 deletions(-)

[GIT net-next] Open vSwitch

[Pravin B Shelar]

Following patches introduces flow mask cache. To process any packet
OVS need to apply flow mask to the flow and lookup the flow in flow table.
so packet processing performance is directly dependant on number of entries
in mask list.

Following patch adds mask cache so that we do not need to iterate over
all entries in mask list on every packet. We have seen good performance
improvement with this patch.

Before the mask-cache, a single stream which matched the first mask
got a throughput of about 900K pps. A stream which matched the 20th mask
got a throughput of about 400K pps. After the mask-cache patch, all
streams throughput went back up to 900K pps.

----------------------------------------------------------------

The following changes since commit 2f55daa5464e8dfc8787ec863b6d1094522dbd69:

  net: stmmac: Support devicetree configs for mcast and ucast filter entries (2014-07-31 15:31:02 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git net_next_ovs

for you to fetch changes up to 4955f0f9cbefa73577cd30ec262538ffc73dd4c2:

  openvswitch: Introduce flow mask cache. (2014-07-31 15:49:55 -0700)

----------------------------------------------------------------
Pravin B Shelar (3):
      openvswitch: Move table destroy to dp-rcu callback.
      openvswitch: Convert mask list into mask array.
      openvswitch: Introduce flow mask cache.

 net/openvswitch/datapath.c   |   8 +-
 net/openvswitch/flow.h       |   1 -
 net/openvswitch/flow_table.c | 293 +++++++++++++++++++++++++++++++++++++------
 net/openvswitch/flow_table.h |  21 +++-
 4 files changed, 275 insertions(+), 48 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Pravin B Shelar <pshelar@nicira.com>
Date: Thu, 31 Jul 2014 16:57:37 -0700

> Following patch adds mask cache so that we do not need to iterate over
> all entries in mask list on every packet. We have seen good performance
> improvement with this patch.

How much have you thought about the DoS'ability of openvswitch's
datastructures?

What are the upper bounds for performance of packet switching?

To be quite honest, a lot of the openvswitch data structures
adjustments that hit my inbox seem to me to only address specific
situations that specific user configurations have run into.

It took us two decades, but we ripped out the ipv4 routing cache
because external entities could provoke unreasonable worst case
behavior in routing lookups.

With openvswitch you guys have a unique opportunity to try and design
all of your features such that they absolutely can use scalable
datastructures from the beginning that provide reasonable performance
in the common case and precise upper bounds for any possible sequence
of incoming packets.

New features tend to blind the developer to the eventual long term
ramifications on performance.  Would you add a new feature if you
could know ahead of time that you'll never be able to design a
datastructure which supports that feature and is not DoS'able by a
remote entity?

Re: [GIT net-next] Open vSwitch

[Pravin Shelar]

On Sat, Aug 2, 2014 at 3:16 PM, David Miller <davem@davemloft.net> wrote:
> From: Pravin B Shelar <pshelar@nicira.com>
> Date: Thu, 31 Jul 2014 16:57:37 -0700
>
>> Following patch adds mask cache so that we do not need to iterate over
>> all entries in mask list on every packet. We have seen good performance
>> improvement with this patch.
>
> How much have you thought about the DoS'ability of openvswitch's
> datastructures?
>
This cache is populated by flow lookup in fast path. The mask cache is
fixed in size. Userspace or number of packets can not change its size.
Memory is statically allocated, no garbage collection. So DoS attack
can not exploit this cache to increase ovs memory footprint.

> What are the upper bounds for performance of packet switching?
>
Cache is keyed on packet RSS. Worst case scenario this cache adds one
extra flow-table lookup for the flow if RSS hash matches but packet
belong to different flow (hash collision).
This is designed to be lightweight, stateless cache (does not take any
reference on other data structures) to have least impact on
DoS'ability of Open vSwitch.

> To be quite honest, a lot of the openvswitch data structures
> adjustments that hit my inbox seem to me to only address specific
> situations that specific user configurations have run into.
>

Overall OVS DoS defense has improved since introduction of mega-flow.
Recently introduced OVS feature allows userspace to set multiple
sockets for upcall processing for given vport. This adds fairness by
separating upcall from different flows to a socket. Userspace process
upcall from these sockets in round-robin fashion. This helps to avoid
one port monopolize upcall communication path.

I agree there is scope for improving DoS defense and we will keep
working on this issue.

> It took us two decades, but we ripped out the ipv4 routing cache
> because external entities could provoke unreasonable worst case
> behavior in routing lookups.
>
> With openvswitch you guys have a unique opportunity to try and design
> all of your features such that they absolutely can use scalable
> datastructures from the beginning that provide reasonable performance
> in the common case and precise upper bounds for any possible sequence
> of incoming packets.
>
> New features tend to blind the developer to the eventual long term
> ramifications on performance.  Would you add a new feature if you
> could know ahead of time that you'll never be able to design a
> datastructure which supports that feature and is not DoS'able by a
> remote entity?
>

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Pravin Shelar <pshelar@nicira.com>
Date: Sun, 3 Aug 2014 12:20:32 -0700

> On Sat, Aug 2, 2014 at 3:16 PM, David Miller <davem@davemloft.net> wrote:
>> From: Pravin B Shelar <pshelar@nicira.com>
>> Date: Thu, 31 Jul 2014 16:57:37 -0700
>>
>>> Following patch adds mask cache so that we do not need to iterate over
>>> all entries in mask list on every packet. We have seen good performance
>>> improvement with this patch.
>>
>> How much have you thought about the DoS'ability of openvswitch's
>> datastructures?
>>
> This cache is populated by flow lookup in fast path. The mask cache is
> fixed in size. Userspace or number of packets can not change its size.
> Memory is statically allocated, no garbage collection. So DoS attack
> can not exploit this cache to increase ovs memory footprint.

An attacker can construct a packet sequence such that every mask cache
lookup misses, making the cache effectively useless.

Re: [GIT net-next] Open vSwitch

[Pravin Shelar]

On Sun, Aug 3, 2014 at 9:21 PM, David Miller <davem@davemloft.net> wrote:
> From: Pravin Shelar <pshelar@nicira.com>
> Date: Sun, 3 Aug 2014 12:20:32 -0700
>
>> On Sat, Aug 2, 2014 at 3:16 PM, David Miller <davem@davemloft.net> wrote:
>>> From: Pravin B Shelar <pshelar@nicira.com>
>>> Date: Thu, 31 Jul 2014 16:57:37 -0700
>>>
>>>> Following patch adds mask cache so that we do not need to iterate over
>>>> all entries in mask list on every packet. We have seen good performance
>>>> improvement with this patch.
>>>
>>> How much have you thought about the DoS'ability of openvswitch's
>>> datastructures?
>>>
>> This cache is populated by flow lookup in fast path. The mask cache is
>> fixed in size. Userspace or number of packets can not change its size.
>> Memory is statically allocated, no garbage collection. So DoS attack
>> can not exploit this cache to increase ovs memory footprint.
>
> An attacker can construct a packet sequence such that every mask cache
> lookup misses, making the cache effectively useless.

Yes, but it does work in normal traffic situations. I have posted
performance numbers in the cover letter.
Under DoS attack as you said attacker need to build sequence of
packets to make cache ineffective. Which results in cache miss and a
full in-kernel flow lookup. Therefore with this cache there is one
more lookup done under DoS. But this is not very different than
current OVS anyways.

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Pravin Shelar <pshelar@nicira.com>
Date: Mon, 4 Aug 2014 12:35:59 -0700

> On Sun, Aug 3, 2014 at 9:21 PM, David Miller <davem@davemloft.net> wrote:
>> From: Pravin Shelar <pshelar@nicira.com>
>> Date: Sun, 3 Aug 2014 12:20:32 -0700
>>
>>> On Sat, Aug 2, 2014 at 3:16 PM, David Miller <davem@davemloft.net> wrote:
>>>> From: Pravin B Shelar <pshelar@nicira.com>
>>>> Date: Thu, 31 Jul 2014 16:57:37 -0700
>>>>
>>>>> Following patch adds mask cache so that we do not need to iterate over
>>>>> all entries in mask list on every packet. We have seen good performance
>>>>> improvement with this patch.
>>>>
>>>> How much have you thought about the DoS'ability of openvswitch's
>>>> datastructures?
>>>>
>>> This cache is populated by flow lookup in fast path. The mask cache is
>>> fixed in size. Userspace or number of packets can not change its size.
>>> Memory is statically allocated, no garbage collection. So DoS attack
>>> can not exploit this cache to increase ovs memory footprint.
>>
>> An attacker can construct a packet sequence such that every mask cache
>> lookup misses, making the cache effectively useless.
> 
> Yes, but it does work in normal traffic situations.

You're basically just reiterating the point I'm trying to make.

Your caches are designed for specific configuration and packet traffic
pattern cases, and can be easily duped into a worse case performance
scenerio by an attacker.

Caches, basically, do not work on the real internet.

Make the fundamental core data structures fast and scalable enough,
rather than bolting caches (which are basically hacks) on top every
time they don't perform to your expectations.

What if you made the full flow lookup fundamentally faster?  Then an
attacker can't do anything about that.  That's a real performance
improvement, one that sustains arbitrary traffic patterns.

Re: [GIT net-next] Open vSwitch

[Alexei Starovoitov]

On Mon, Aug 4, 2014 at 12:42 PM, David Miller <davem@davemloft.net> wrote:
> From: Pravin Shelar <pshelar@nicira.com>
> Date: Mon, 4 Aug 2014 12:35:59 -0700
>
>> On Sun, Aug 3, 2014 at 9:21 PM, David Miller <davem@davemloft.net> wrote:
>>> An attacker can construct a packet sequence such that every mask cache
>>> lookup misses, making the cache effectively useless.
>>
>> Yes, but it does work in normal traffic situations.
>
> You're basically just reiterating the point I'm trying to make.
>
> Your caches are designed for specific configuration and packet traffic
> pattern cases, and can be easily duped into a worse case performance
> scenerio by an attacker.
>
> Caches, basically, do not work on the real internet.
>
> Make the fundamental core data structures fast and scalable enough,
> rather than bolting caches (which are basically hacks) on top every
> time they don't perform to your expectations.
>
> What if you made the full flow lookup fundamentally faster?  Then an

I suspect that the flow lookup in ovs is as fast as it can be, yet
ovs is still dos-able, since kernel datapath (flow lookup and action)
is considered to be first level cache for user space. By design flow
miss is always punted to userspace. Therefore netperf TCP_CRR test
from a VM is not cheap for host userspace component. Mega-flows and
multiple upcall pids are partially addressing this fundamental
problem. Consider simple distributed virtual bridge with VMs
distributed across multiple hosts. Mega-flow mask that selects dmac
can solve CRR case for well behaving VMs, but rogue VM that spams
random dmac will keep taxing host userspace. So we'd need to add
another flow mask to match the rest of traffic unconditionally and
drop it. Now consider virtual bridge-router-bridge topology (two
subnets and router using openstack names). Since VMs on the same
host may be in different subnets their macs can be the same, so
'mega-flow mask dmac' approach won't work and CRR test again is
getting costly to userspace. We can try to use 'in_port + dmac'
mask, but as network topology grows the flow mask tricks get out
of hand. Situation is worse when ovs works as gateway and receives
internet traffic. Since flow miss goes to userspace remote attacker
can find a way to saturate gateway cpu with certain traffic.
I guess none of this is new to ovs and there is probably a solution
that I don't know about.

Re: [GIT net-next] Open vSwitch

[Nicolas Dichtel]

Le 04/08/2014 21:42, David Miller a écrit :
[snip]
> Caches, basically, do not work on the real internet.
A bit late, but I completely agree!

[GIT net-next] Open vSwitch

[Pravin B Shelar]

Following patches adds three features to OVS
1. Add fairness to upcall processing.
2. Recirculation and Hash action.
3. Enable Tunnel GSO features.
Rest of patches are bug fixes related to patches from same series.

The following changes since commit 279f64b7a771d84cbdea51ac2f794becfb06bcd4:

  net/hsr: Remove left-over never-true conditional code. (2014-07-11 15:04:40 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git net_next_ovs

for you to fetch changes up to 3dec4774b6343e5db5d346f1f05c6a883e0069db:

  openvswitch: Add skb_clone NULL check for the sampling action. (2014-07-13 12:02:12 -0700)

----------------------------------------------------------------
Alex Wang (1):
      openvswitch: Allow each vport to have an array of 'port_id's.

Andy Zhou (6):
      openvswitch: Add hash action
      openvswitch: Add recirc action
      openvswitch: Fix key size computation in key_attr_size()
      openvswitch: Avoid memory corruption in queue_userspace_packet()
      openvswitch: Add skb_clone NULL check in the recirc action.
      openvswitch: Add skb_clone NULL check for the sampling action.

Pravin B Shelar (2):
      openvswitch: Enable tunnel GSO for OVS bridge.
      net: Export xmit_recursion

Simon Horman (2):
      openvswitch: Free skb(s) on recirculation error
      openvswitch: Sample action without side effects

 include/linux/netdev_features.h      |   8 +++
 include/linux/netdevice.h            |   3 +
 include/uapi/linux/openvswitch.h     |  39 ++++++++++--
 net/core/dev.c                       |  10 +--
 net/openvswitch/actions.c            | 119 +++++++++++++++++++++++++++++++----
 net/openvswitch/datapath.c           |  79 ++++++++++++++++-------
 net/openvswitch/datapath.h           |   8 ++-
 net/openvswitch/flow.h               |   2 +
 net/openvswitch/flow_netlink.c       |  43 ++++++++++++-
 net/openvswitch/vport-internal_dev.c |   5 +-
 net/openvswitch/vport.c              | 102 +++++++++++++++++++++++++++++-
 net/openvswitch/vport.h              |  27 ++++++--
 12 files changed, 393 insertions(+), 52 deletions(-)

[GIT net-next] Open vSwitch

[Pravin B Shelar]

A set of OVS changes for net-next/3.16.

Most of change are related to improving performance of flow setup by
minimizing critical sections.

The following changes since commit 091b64868b43ed84334c6623ea6a08497529d4ff:

  Merge branch 'mlx4-next' (2014-05-22 17:17:34 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pshelar/openvswitch.git master

for you to fetch changes up to 0c200ef94c9492205e18a18c25650cf27939889c:

  openvswitch: Simplify genetlink code. (2014-05-22 16:27:37 -0700)

----------------------------------------------------------------
Jarno Rajahalme (12):
  openvswitch: Compact sw_flow_key.
  openvswitch: Avoid assigning a NULL pointer to flow actions.
  openvswitch: Clarify locking.
  openvswitch: Build flow cmd netlink reply only if needed.
  openvswitch: Make flow mask removal symmetric.
  openvswitch: Minimize dp and vport critical sections.
  openvswitch: Fix typo.
  openvswitch: Fix ovs_flow_stats_get/clear RCU dereference.
  openvswitch: Reduce locking requirements.
  openvswitch: Minimize ovs_flow_cmd_del critical section.
  openvswitch: Split ovs_flow_cmd_new_or_set().
  openvswitch: Minimize ovs_flow_cmd_new|set critical sections.

Pravin B Shelar (1):
  openvswitch: Simplify genetlink code.

 include/uapi/linux/openvswitch.h |   4 +-
 net/openvswitch/datapath.c       | 771 +++++++++++++++++++++++----------------
 net/openvswitch/flow.c           |  53 ++-
 net/openvswitch/flow.h           |  35 +-
 net/openvswitch/flow_netlink.c   | 112 ++----
 net/openvswitch/flow_table.c     |  46 ++-
 6 files changed, 558 insertions(+), 463 deletions(-)

-- 
1.9.0

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Pravin B Shelar <pshelar@nicira.com>
Date: Tue, 20 May 2014 01:59:38 -0700

> A set of OVS changes for net-next/3.16.
> 
> Most of change are related to improving performance of flow setup by
> minimizing critical sections.

Pulled, thanks Pravin.

In the future please make your postings so that they have the current date
and time when you make the postings, not when the commits when into your
tree.

Otherwise it messed up the order in which your changes appear in patchwork
wrt. other submissions.

Re: [GIT net-next] Open vSwitch

[Pravin Shelar]

On Fri, May 23, 2014 at 11:46 AM, David Miller <davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org> wrote:
> From: Pravin B Shelar <pshelar-l0M0P4e3n4LQT0dZR+AlfA@public.gmane.org>
> Date: Tue, 20 May 2014 01:59:38 -0700
>
>> A set of OVS changes for net-next/3.16.
>>
>> Most of change are related to improving performance of flow setup by
>> minimizing critical sections.
>
> Pulled, thanks Pravin.
>
> In the future please make your postings so that they have the current date
> and time when you make the postings, not when the commits when into your
> tree.
>
> Otherwise it messed up the order in which your changes appear in patchwork
> wrt. other submissions.

ok, I will keep it in mind.

Thanks,
Pravin.

[GIT net-next] Open vSwitch

[Jesse Gross]

A set of OVS changes for net-next/3.16.

The major change here is a switch from per-CPU to per-NUMA flow
statistics. This improves scalability by reducing kernel overhead
in flow setup and maintenance.

The following changes since commit a188a54d11629bef2169052297e61f3767ca8ce5:

  macvlan: simplify the structure port (2014-05-15 23:35:16 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 944df8ae84d88f5e8eb027990dad2cfa4fbe4be5:

  net/openvswitch: Use with RCU_INIT_POINTER(x, NULL) in vport-gre.c (2014-05-16 13:40:29 -0700)

----------------------------------------------------------------
Daniele Di Proietto (4):
      openvswitch: use const in some local vars and casts
      openvswitch: avoid warnings in vport_from_priv
      openvswitch: avoid cast-qual warning in vport_priv
      openvswitch: Added (unsigned long long) cast in printf

Jarno Rajahalme (4):
      openvswitch: Remove 5-tuple optimization.
      openvswitch: Per NUMA node flow stats.
      openvswitch: Fix output of SCTP mask.
      openvswitch: Use TCP flags in the flow key for stats.

Joe Perches (3):
      openvswitch: Use net_ratelimit in OVS_NLERR
      openvswitch: flow_netlink: Use pr_fmt to OVS_NLERR output
      openvswitch: Use ether_addr_copy

Monam Agarwal (1):
      net/openvswitch: Use with RCU_INIT_POINTER(x, NULL) in vport-gre.c

 net/openvswitch/actions.c      |   4 +-
 net/openvswitch/datapath.c     |  11 ++-
 net/openvswitch/datapath.h     |   8 ++-
 net/openvswitch/flow.c         | 149 ++++++++++++++++++++++++-----------------
 net/openvswitch/flow.h         |  18 +++--
 net/openvswitch/flow_netlink.c |  82 +++++------------------
 net/openvswitch/flow_netlink.h |   1 -
 net/openvswitch/flow_table.c   |  75 ++++++++++++---------
 net/openvswitch/flow_table.h   |   4 +-
 net/openvswitch/vport-gre.c    |   2 +-
 net/openvswitch/vport.h        |   6 +-
 11 files changed, 176 insertions(+), 184 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse-l0M0P4e3n4LQT0dZR+AlfA@public.gmane.org>
Date: Fri, 16 May 2014 14:07:27 -0700

> A set of OVS changes for net-next/3.16.
> 
> The major change here is a switch from per-CPU to per-NUMA flow
> statistics. This improves scalability by reducing kernel overhead
> in flow setup and maintenance.

Pulled, thanks Jesse.

[GIT net-next] Open vSwitch

[Jesse Gross]

Open vSwitch changes for net-next/3.14. Highlights are:
 * Performance improvements in the mechanism to get packets to userspace
   using memory mapped netlink and skb zero copy where appropriate.
 * Per-cpu flow stats in situations where flows are likely to be shared
   across CPUs. Standard flow stats are used in other situations to save
   memory and allocation time.
 * A handful of code cleanups and rationalization.

The following changes since commit 6ce4eac1f600b34f2f7f58f9cd8f0503d79e42ae:

  Linux 3.13-rc1 (2013-11-22 11:30:55 -0800)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 443cd88c8a31379e95326428bbbd40af25c1d440:

  ovs: make functions local (2014-01-06 15:54:39 -0800)

----------------------------------------------------------------
Andy Zhou (1):
      openvswitch: Change ovs_flow_tbl_lookup_xx() APIs

Ben Pfaff (2):
      openvswitch: Correct comment.
      openvswitch: Shrink sw_flow_mask by 8 bytes (64-bit) or 4 bytes (32-bit).

Daniel Borkmann (1):
      net: ovs: use kfree_rcu instead of rcu_free_{sw_flow_mask_cb,acts_callback}

Jesse Gross (1):
      openvswitch: Silence RCU lockdep checks from flow lookup.

Pravin B Shelar (1):
      openvswitch: Per cpu flow stats.

Stephen Hemminger (1):
      ovs: make functions local

Thomas Graf (9):
      genl: Add genlmsg_new_unicast() for unicast message allocation
      netlink: Avoid netlink mmap alloc if msg size exceeds frame size
      openvswitch: Enable memory mapped Netlink i/o
      net: Export skb_zerocopy() to zerocopy from one skb to another
      openvswitch: Allow user space to announce ability to accept unaligned Netlink messages
      openvswitch: Drop user features if old user space attempted to create datapath
      openvswitch: Pass datapath into userspace queue functions
      openvswitch: Use skb_zerocopy() for upcall
      openvswitch: Compute checksum in skb_gso_segment() if needed

Wei Yongjun (1):
      openvswitch: remove duplicated include from flow_table.c

 include/linux/skbuff.h               |   3 +
 include/net/genetlink.h              |   4 +
 include/uapi/linux/openvswitch.h     |  14 ++-
 net/core/skbuff.c                    |  85 +++++++++++++
 net/netfilter/nfnetlink_queue_core.c |  59 +--------
 net/netlink/af_netlink.c             |   4 +
 net/netlink/genetlink.c              |  21 ++++
 net/openvswitch/datapath.c           | 231 +++++++++++++++++++----------------
 net/openvswitch/datapath.h           |   6 +-
 net/openvswitch/flow.c               |  96 +++++++++++++--
 net/openvswitch/flow.h               |  33 +++--
 net/openvswitch/flow_netlink.c       |  66 ++++++++--
 net/openvswitch/flow_netlink.h       |   1 +
 net/openvswitch/flow_table.c         |  60 ++++++---
 net/openvswitch/flow_table.h         |   6 +-
 net/openvswitch/vport.c              |   6 +-
 net/openvswitch/vport.h              |   1 -
 17 files changed, 483 insertions(+), 213 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse-l0M0P4e3n4LQT0dZR+AlfA@public.gmane.org>
Date: Mon,  6 Jan 2014 16:15:59 -0800

> Open vSwitch changes for net-next/3.14. Highlights are:
>  * Performance improvements in the mechanism to get packets to userspace
>    using memory mapped netlink and skb zero copy where appropriate.
>  * Per-cpu flow stats in situations where flows are likely to be shared
>    across CPUs. Standard flow stats are used in other situations to save
>    memory and allocation time.
>  * A handful of code cleanups and rationalization.

Lots of good stuff in here, pulled, thanks Jesse.

Re: [ovs-dev] [GIT net-next] Open vSwitch

[Zoltan Kiss]

Hi,

I've tried the latest net-next on a Xenserver install with 1.9.3 
userspace, and it seems this patch series broke it (at least after 
reverting that locally it works now). I haven't went too far yet 
checking what's the problem, but it seems the xenbrX device doesn't 
really receive too much of the traffic coming through the NIC. Is it 
expected?

Regards,

Zoli

On 07/01/14 00:15, Jesse Gross wrote:
> Open vSwitch changes for net-next/3.14. Highlights are:
>   * Performance improvements in the mechanism to get packets to userspace
>     using memory mapped netlink and skb zero copy where appropriate.
>   * Per-cpu flow stats in situations where flows are likely to be shared
>     across CPUs. Standard flow stats are used in other situations to save
>     memory and allocation time.
>   * A handful of code cleanups and rationalization.
>
> The following changes since commit 6ce4eac1f600b34f2f7f58f9cd8f0503d79e42ae:
>
>    Linux 3.13-rc1 (2013-11-22 11:30:55 -0800)
>
> are available in the git repository at:
>
>    git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master
>
> for you to fetch changes up to 443cd88c8a31379e95326428bbbd40af25c1d440:
>
>    ovs: make functions local (2014-01-06 15:54:39 -0800)
>
> ----------------------------------------------------------------
> Andy Zhou (1):
>        openvswitch: Change ovs_flow_tbl_lookup_xx() APIs
>
> Ben Pfaff (2):
>        openvswitch: Correct comment.
>        openvswitch: Shrink sw_flow_mask by 8 bytes (64-bit) or 4 bytes (32-bit).
>
> Daniel Borkmann (1):
>        net: ovs: use kfree_rcu instead of rcu_free_{sw_flow_mask_cb,acts_callback}
>
> Jesse Gross (1):
>        openvswitch: Silence RCU lockdep checks from flow lookup.
>
> Pravin B Shelar (1):
>        openvswitch: Per cpu flow stats.
>
> Stephen Hemminger (1):
>        ovs: make functions local
>
> Thomas Graf (9):
>        genl: Add genlmsg_new_unicast() for unicast message allocation
>        netlink: Avoid netlink mmap alloc if msg size exceeds frame size
>        openvswitch: Enable memory mapped Netlink i/o
>        net: Export skb_zerocopy() to zerocopy from one skb to another
>        openvswitch: Allow user space to announce ability to accept unaligned Netlink messages
>        openvswitch: Drop user features if old user space attempted to create datapath
>        openvswitch: Pass datapath into userspace queue functions
>        openvswitch: Use skb_zerocopy() for upcall
>        openvswitch: Compute checksum in skb_gso_segment() if needed
>
> Wei Yongjun (1):
>        openvswitch: remove duplicated include from flow_table.c
>
>   include/linux/skbuff.h               |   3 +
>   include/net/genetlink.h              |   4 +
>   include/uapi/linux/openvswitch.h     |  14 ++-
>   net/core/skbuff.c                    |  85 +++++++++++++
>   net/netfilter/nfnetlink_queue_core.c |  59 +--------
>   net/netlink/af_netlink.c             |   4 +
>   net/netlink/genetlink.c              |  21 ++++
>   net/openvswitch/datapath.c           | 231 +++++++++++++++++++----------------
>   net/openvswitch/datapath.h           |   6 +-
>   net/openvswitch/flow.c               |  96 +++++++++++++--
>   net/openvswitch/flow.h               |  33 +++--
>   net/openvswitch/flow_netlink.c       |  66 ++++++++--
>   net/openvswitch/flow_netlink.h       |   1 +
>   net/openvswitch/flow_table.c         |  60 ++++++---
>   net/openvswitch/flow_table.h         |   6 +-
>   net/openvswitch/vport.c              |   6 +-
>   net/openvswitch/vport.h              |   1 -
>   17 files changed, 483 insertions(+), 213 deletions(-)
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
>

Re: [GIT net-next] Open vSwitch

[Jesse Gross]

On Wed, Jan 8, 2014 at 9:49 AM, Zoltan Kiss <zoltan.kiss-Sxgqhf6Nn4DQT0dZR+AlfA@public.gmane.org> wrote:
> Hi,
>
> I've tried the latest net-next on a Xenserver install with 1.9.3 userspace,
> and it seems this patch series broke it (at least after reverting that
> locally it works now). I haven't went too far yet checking what's the
> problem, but it seems the xenbrX device doesn't really receive too much of
> the traffic coming through the NIC. Is it expected?

What do you mean by doesn't receive too much traffic? What does it get?

Re: [ovs-dev] [GIT net-next] Open vSwitch

[Zoltan Kiss]

On 08/01/14 15:10, Jesse Gross wrote:
> On Wed, Jan 8, 2014 at 9:49 AM, Zoltan Kiss <zoltan.kiss@citrix.com> wrote:
>> Hi,
>>
>> I've tried the latest net-next on a Xenserver install with 1.9.3 userspace,
>> and it seems this patch series broke it (at least after reverting that
>> locally it works now). I haven't went too far yet checking what's the
>> problem, but it seems the xenbrX device doesn't really receive too much of
>> the traffic coming through the NIC. Is it expected?
>
> What do you mean by doesn't receive too much traffic? What does it get?
>

Sorry for the vague error description, now I had more time to look into 
this. I think the problem boils down to this:

Jan 13 17:55:07 localhost ovs-vswitchd: 
07890|netlink_socket|DBG|nl_sock_recv__ (Success): nl(len:274, 
type=29(ovs_packet), flags=0, seq=0, pid=0,genl(cmd=1,version=1)
Jan 13 17:55:07 localhost ovs-vswitchd: 07891|netlink|DBG|attributes 
followed by garbage
Jan 13 17:55:07 localhost ovs-vswitchd: 07892|dpif|WARN|system@xenbr0: 
recv failed (Invalid argument)

That's just keep repeating. I'm keep looking.

Zoli

Re: [GIT net-next] Open vSwitch

[Zoltan Kiss]

On 13/01/14 18:04, Zoltan Kiss wrote:
> On 08/01/14 15:10, Jesse Gross wrote:
>> On Wed, Jan 8, 2014 at 9:49 AM, Zoltan Kiss <zoltan.kiss-Sxgqhf6Nn4DQT0dZR+AlfA@public.gmane.org> 
>> wrote:
>>> Hi,
>>>
>>> I've tried the latest net-next on a Xenserver install with 1.9.3 
>>> userspace,
>>> and it seems this patch series broke it (at least after reverting that
>>> locally it works now). I haven't went too far yet checking what's the
>>> problem, but it seems the xenbrX device doesn't really receive too 
>>> much of
>>> the traffic coming through the NIC. Is it expected?
>>
>> What do you mean by doesn't receive too much traffic? What does it get?
>>
>
> Sorry for the vague error description, now I had more time to look 
> into this. I think the problem boils down to this:
>
> Jan 13 17:55:07 localhost ovs-vswitchd: 
> 07890|netlink_socket|DBG|nl_sock_recv__ (Success): nl(len:274, 
> type=29(ovs_packet), flags=0, seq=0, pid=0,genl(cmd=1,version=1)
> Jan 13 17:55:07 localhost ovs-vswitchd: 07891|netlink|DBG|attributes 
> followed by garbage
> Jan 13 17:55:07 localhost ovs-vswitchd: 07892|dpif|WARN|system@xenbr0: 
> recv failed (Invalid argument)
>
> That's just keep repeating. I'm keep looking.

Now I reverted these top 3 commits:

ovs: make functions local
openvswitch: Compute checksum in skb_gso_segment() if needed
openvswitch: Use skb_zerocopy() for upcall

And it works. I guess the last one causing the problem. Might be an 
important factor, I'm using 32 bit Dom0.

Zoli

Re: [GIT net-next] Open vSwitch

[Jesse Gross]

On Mon, Jan 13, 2014 at 4:31 PM, Zoltan Kiss <zoltan.kiss-Sxgqhf6Nn4DQT0dZR+AlfA@public.gmane.org> wrote:
> On 13/01/14 18:04, Zoltan Kiss wrote:
>>
>> On 08/01/14 15:10, Jesse Gross wrote:
>>>
>>> On Wed, Jan 8, 2014 at 9:49 AM, Zoltan Kiss <zoltan.kiss-Sxgqhf6Nn4DQT0dZR+AlfA@public.gmane.org>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I've tried the latest net-next on a Xenserver install with 1.9.3
>>>> userspace,
>>>> and it seems this patch series broke it (at least after reverting that
>>>> locally it works now). I haven't went too far yet checking what's the
>>>> problem, but it seems the xenbrX device doesn't really receive too much
>>>> of
>>>> the traffic coming through the NIC. Is it expected?
>>>
>>>
>>> What do you mean by doesn't receive too much traffic? What does it get?
>>>
>>
>> Sorry for the vague error description, now I had more time to look into
>> this. I think the problem boils down to this:
>>
>> Jan 13 17:55:07 localhost ovs-vswitchd:
>> 07890|netlink_socket|DBG|nl_sock_recv__ (Success): nl(len:274,
>> type=29(ovs_packet), flags=0, seq=0, pid=0,genl(cmd=1,version=1)
>> Jan 13 17:55:07 localhost ovs-vswitchd: 07891|netlink|DBG|attributes
>> followed by garbage
>> Jan 13 17:55:07 localhost ovs-vswitchd: 07892|dpif|WARN|system@xenbr0:
>> recv failed (Invalid argument)
>>
>> That's just keep repeating. I'm keep looking.
>
>
> Now I reverted these top 3 commits:
>
> ovs: make functions local
>
> openvswitch: Compute checksum in skb_gso_segment() if needed
> openvswitch: Use skb_zerocopy() for upcall
>
> And it works. I guess the last one causing the problem. Might be an
> important factor, I'm using 32 bit Dom0.

I think you're probably right. Thomas - can you take a look?

We shouldn't be doing any zerocopy in this situation but it looks to
me like we don't do any padding at all, even in situations where we
are copying the data.

Re: [GIT net-next] Open vSwitch

[Thomas Graf]

On 01/14/2014 02:30 AM, Jesse Gross wrote:
>> And it works. I guess the last one causing the problem. Might be an
>> important factor, I'm using 32 bit Dom0.
>
> I think you're probably right. Thomas - can you take a look?
>
> We shouldn't be doing any zerocopy in this situation but it looks to
> me like we don't do any padding at all, even in situations where we
> are copying the data.

I'm looking into this now. The zerocopy method should only be attempted
if user space has announced the ability to received unaligned messages.

@Zoltan: I assume you are using an unmodified OVS 1.9.3?

[GIT net-next] Open vSwitch

[Jesse Gross]

A set of updates for net-next/3.13. Major changes are:
 * Restructure flow handling code to be more logically organized and
   easier to read.
 * Rehashing of the flow table is moved from a workqueue to flow
   installation time. Before, heavy load could block the workqueue for
   excessive periods of time.
 * Additional debugging information is provided to help diagnose megaflows.
 * It's now possible to match on TCP flags.

The following changes since commit 272b98c6455f00884f0350f775c5342358ebb73f:

  Linux 3.12-rc1 (2013-09-16 16:17:51 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 8ddd094675cfd453fc9838caa46ea108a4107183:

  openvswitch: Use flow hash during flow lookup operation. (2013-11-01 18:43:46 -0700)

----------------------------------------------------------------
Andy Zhou (1):
      openvswitch: collect mega flow mask stats

Jarno Rajahalme (2):
      openvswitch: Widen TCP flags handling.
      openvswitch: TCP flags matching support.

Pravin B Shelar (6):
      openvswitch: Move flow table rehashing to flow install.
      openvswitch: Restructure datapath.c and flow.c
      openvswitch: Move mega-flow list out of rehashing struct.
      openvswitch: Simplify mega-flow APIs.
      openvswitch: Enable all GSO features on internal port.
      openvswitch: Use flow hash during flow lookup operation.

Wei Yongjun (2):
      openvswitch: remove duplicated include from vport-vxlan.c
      openvswitch: remove duplicated include from vport-gre.c

 include/uapi/linux/openvswitch.h     |   18 +-
 net/openvswitch/Makefile             |    2 +
 net/openvswitch/datapath.c           |  668 ++------------
 net/openvswitch/datapath.h           |    9 +-
 net/openvswitch/flow.c               | 1605 +--------------------------------
 net/openvswitch/flow.h               |  132 +--
 net/openvswitch/flow_netlink.c       | 1630 ++++++++++++++++++++++++++++++++++
 net/openvswitch/flow_netlink.h       |   60 ++
 net/openvswitch/flow_table.c         |  592 ++++++++++++
 net/openvswitch/flow_table.h         |   81 ++
 net/openvswitch/vport-gre.c          |    2 -
 net/openvswitch/vport-internal_dev.c |    2 +-
 net/openvswitch/vport-vxlan.c        |    1 -
 13 files changed, 2511 insertions(+), 2291 deletions(-)
 create mode 100644 net/openvswitch/flow_netlink.c
 create mode 100644 net/openvswitch/flow_netlink.h
 create mode 100644 net/openvswitch/flow_table.c
 create mode 100644 net/openvswitch/flow_table.h

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse@nicira.com>
Date: Sat,  2 Nov 2013 00:43:39 -0700

> A set of updates for net-next/3.13. Major changes are:
>  * Restructure flow handling code to be more logically organized and
>    easier to read.
>  * Rehashing of the flow table is moved from a workqueue to flow
>    installation time. Before, heavy load could block the workqueue for
>    excessive periods of time.
>  * Additional debugging information is provided to help diagnose megaflows.
>  * It's now possible to match on TCP flags.

Looks good, pulled, thanks Jesse.

[GIT net-next] Open vSwitch

[Jesse Gross]

A set of updates for net-next/3.13. Major changes are:
 * Restructure flow handling code to be more logically organized and
   easier to read.
 * Previously flow state was effectively per-CPU but this is no longer
   true with the addition of wildcarded flows (megaflows). While good
   for flow setup rates, it is bad for stats updates. Stats are now
   per-CPU again to get the best of both worlds.
 * Rehashing of the flow table is moved from a workqueue to flow
   installation time. Before, heavy load could block the workqueue for
   excessive periods of time.
 * Additional debugging information is provided to help diagnose megaflows.
 * It's now possible to match on TCP flags.

The following changes since commit 272b98c6455f00884f0350f775c5342358ebb73f:

  Linux 3.12-rc1 (2013-09-16 16:17:51 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to c7e8b9659587449b21ee68d7aee0cacadf650cce:

  openvswitch: TCP flags matching support. (2013-10-23 02:19:50 -0700)

----------------------------------------------------------------
Andy Zhou (1):
      openvswitch: collect mega flow mask stats

Jarno Rajahalme (2):
      openvswitch: Widen TCP flags handling.
      openvswitch: TCP flags matching support.

Pravin B Shelar (6):
      openvswitch: Move flow table rehashing to flow install.
      openvswitch: Restructure datapath.c and flow.c
      openvswitch: Move mega-flow list out of rehashing struct.
      openvswitch: Simplify mega-flow APIs.
      openvswitch: Per cpu flow stats.
      openvswitch: Enable all GSO features on internal port.

Wei Yongjun (2):
      openvswitch: remove duplicated include from vport-vxlan.c
      openvswitch: remove duplicated include from vport-gre.c

 include/uapi/linux/openvswitch.h     |   18 +-
 net/openvswitch/Makefile             |    2 +
 net/openvswitch/datapath.c           |  721 ++-------------
 net/openvswitch/datapath.h           |    9 +-
 net/openvswitch/flow.c               | 1631 ++--------------------------------
 net/openvswitch/flow.h               |  148 +--
 net/openvswitch/flow_netlink.c       | 1630 +++++++++++++++++++++++++++++++++
 net/openvswitch/flow_netlink.h       |   60 ++
 net/openvswitch/flow_table.c         |  600 +++++++++++++
 net/openvswitch/flow_table.h         |   81 ++
 net/openvswitch/vport-gre.c          |    2 -
 net/openvswitch/vport-internal_dev.c |    2 +-
 net/openvswitch/vport-vxlan.c        |    1 -
 13 files changed, 2590 insertions(+), 2315 deletions(-)
 create mode 100644 net/openvswitch/flow_netlink.c
 create mode 100644 net/openvswitch/flow_netlink.h
 create mode 100644 net/openvswitch/flow_table.c
 create mode 100644 net/openvswitch/flow_table.h

[GIT net-next] Open vSwitch

[Jesse Gross]

A number of significant new features and optimizations for net-next/3.12.
Highlights are:
 * "Megaflows", an optimization that allows userspace to specify which
   flow fields were used to compute the results of the flow lookup.
   This allows for a major reduction in flow setups (the major
   performance bottleneck in Open vSwitch) without reducing flexibility.
 * Converting netlink dump operations to use RCU, allowing for
   additional parallelism in userspace.
 * Matching and modifying SCTP protocol fields.

The following changes since commit 2771399ac9986c75437a83b1c723493cfcdfa439:

  fs_enet: cleanup clock API use (2013-08-22 22:13:54 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 5828cd9a68873df1340b420371c02c47647878fb:

  openvswitch: optimize flow compare and mask functions (2013-08-27 13:13:09 -0700)

----------------------------------------------------------------
Andy Zhou (3):
      openvswitch: Mega flow implementation
      openvswitch: Rename key_len to key_end
      openvswitch: optimize flow compare and mask functions

Cong Wang (1):
      openvswitch: check CONFIG_OPENVSWITCH_GRE in makefile

Jiri Pirko (1):
      openvswitch:: link upper device for port devices

Joe Stringer (2):
      net: Add NEXTHDR_SCTP to ipv6.h
      openvswitch: Add SCTP support

Justin Pettit (1):
      openvswitch: Fix argument descriptions in vport.c.

Pravin B Shelar (3):
      openvswitch: Use RCU lock for flow dump operation.
      openvswitch: Use RCU lock for dp dump operation.
      openvswitch: Use non rcu hlist_del() flow table entry.

 Documentation/networking/openvswitch.txt |   40 +
 include/net/ipv6.h                       |    1 +
 include/uapi/linux/openvswitch.h         |   15 +-
 net/openvswitch/Kconfig                  |    1 +
 net/openvswitch/Makefile                 |    5 +-
 net/openvswitch/actions.c                |   45 +-
 net/openvswitch/datapath.c               |  176 ++--
 net/openvswitch/datapath.h               |    6 +
 net/openvswitch/flow.c                   | 1485 +++++++++++++++++++++---------
 net/openvswitch/flow.h                   |   89 +-
 net/openvswitch/vport-gre.c              |    3 -
 net/openvswitch/vport-netdev.c           |   20 +-
 net/openvswitch/vport.c                  |    3 +-
 13 files changed, 1346 insertions(+), 543 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse@nicira.com>
Date: Tue, 27 Aug 2013 13:20:37 -0700

> A number of significant new features and optimizations for net-next/3.12.
> Highlights are:
>  * "Megaflows", an optimization that allows userspace to specify which
>    flow fields were used to compute the results of the flow lookup.
>    This allows for a major reduction in flow setups (the major
>    performance bottleneck in Open vSwitch) without reducing flexibility.
>  * Converting netlink dump operations to use RCU, allowing for
>    additional parallelism in userspace.
>  * Matching and modifying SCTP protocol fields.

Pulled, thanks Jesse.

[GIT net-next] Open vSwitch

[Jesse Gross]

A few miscellaneous improvements and cleanups before the GRE tunnel
integration series. Intended for net-next/3.11.

The following changes since commit f722406faae2d073cc1d01063d1123c35425939e:

  Linux 3.10-rc1 (2013-05-11 17:14:08 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 93d8fd1514b6862c3370ea92be3f3b4216e0bf8f:

  openvswitch: Simplify interface ovs_flow_metadata_from_nlattrs() (2013-06-14 15:09:12 -0700)

----------------------------------------------------------------
Andy Hill (1):
      openvswitch: Fix misspellings in comments and docs.

Jesse Gross (2):
      openvswitch: Immediately exit on error in ovs_vport_cmd_set().
      openvswitch: Remove unused get_config vport op.

Lorand Jakab (1):
      openvswitch: fix variable names in comment

Pravin B Shelar (4):
      openvswitch: Unify vport error stats handling.
      openvswitch: Fix struct comment.
      openvswitch: make skb->csum consistent with rest of networking stack.
      openvswitch: Simplify interface ovs_flow_metadata_from_nlattrs()

 include/uapi/linux/openvswitch.h     |  1 -
 net/openvswitch/actions.c            |  4 ++++
 net/openvswitch/datapath.c           | 17 ++++++++---------
 net/openvswitch/flow.c               | 29 +++++++++++++++--------------
 net/openvswitch/flow.h               |  4 ++--
 net/openvswitch/vport-internal_dev.c |  1 +
 net/openvswitch/vport-netdev.c       |  7 ++++---
 net/openvswitch/vport-netdev.h       |  1 -
 net/openvswitch/vport.c              | 11 ++++++++---
 net/openvswitch/vport.h              | 13 +++++++++----
 10 files changed, 51 insertions(+), 37 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse@nicira.com>
Date: Fri, 14 Jun 2013 15:28:49 -0700

> A few miscellaneous improvements and cleanups before the GRE tunnel
> integration series. Intended for net-next/3.11.
 ...
>   git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

Pulled, thanks Jesse.

[GIT net-next] Open vSwitch

[Jesse Gross]

A number of improvements for net-next/3.10.

Highlights include:
 * Properly exposing linux/openvswitch.h to userspace after the uapi changes.
 * Simplification of locking. It immediately makes things simpler to reason about and avoids holding RTNL mutex for longer than necessary. In the near future it will also enable tunnel registration and more fine-grained locking.
 * Miscellaneous cleanups and simplifications.

The following changes since commit f498354793d57479d4e1b0f39969acd66737234c:

  qlcnic: Bump up the version to 5.2.39 (2013-03-29 15:51:06 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to e0f0ecf33c3f13401f90bff5afdc3ed1bb40b9af:

  openvswitch: Use generic struct pcpu_tstats. (2013-04-15 14:56:25 -0700)

----------------------------------------------------------------
Andy Zhou (1):
      openvswitch: datapath.h: Fix a stale comment.

Pravin B Shelar (2):
      openvswitch: Simplify datapath locking.
      openvswitch: Use generic struct pcpu_tstats.

Thomas Graf (7):
      openvswitch: Specify the minimal length of OVS_PACKET_ATTR_PACKET in the policy
      openvswitch: Use nla_memcpy() to memcpy() data from attributes
      openvswitch: Refine Netlink message size calculation and kill FLOW_BUFSIZE
      openvswitch: Move common genl notify code into ovs_notify()
      openvswitch: Use ETH_ALEN to define ethernet addresses
      openvswitch: Expose <linux/openvswitch.h> to userspace
      openvswitch: Don't insert empty OVS_VPORT_ATTR_OPTIONS attribute

 include/linux/openvswitch.h          |  432 +-------------------------------
 include/uapi/linux/Kbuild            |    1 +
 include/uapi/linux/openvswitch.h     |  456 ++++++++++++++++++++++++++++++++++
 net/openvswitch/datapath.c           |  393 +++++++++++++++++------------
 net/openvswitch/datapath.h           |   70 ++++--
 net/openvswitch/dp_notify.c          |   82 ++++--
 net/openvswitch/flow.c               |    2 +-
 net/openvswitch/flow.h               |   21 --
 net/openvswitch/vport-internal_dev.c |    6 +
 net/openvswitch/vport-netdev.c       |    8 +-
 net/openvswitch/vport.c              |   58 +++--
 net/openvswitch/vport.h              |   15 +-
 12 files changed, 849 insertions(+), 695 deletions(-)
 create mode 100644 include/uapi/linux/openvswitch.h

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse@nicira.com>
Date: Tue, 16 Apr 2013 14:00:09 -0700

> A number of improvements for net-next/3.10.
> 
> Highlights include:
>  * Properly exposing linux/openvswitch.h to userspace after the uapi changes.
>  * Simplification of locking. It immediately makes things simpler to reason about and avoids holding RTNL mutex for longer than necessary. In the near future it will also enable tunnel registration and more fine-grained locking.
>  * Miscellaneous cleanups and simplifications.

Pulled, but please don't make your email look so silly with those 500
character long lines, make them not exceed 80 columns.

Thanks.

[GIT net-next] Open vSwitch

[Jesse Gross]

A couple of minor enhancements for net-next/3.10.  The largest is an
extension to allow variable length metadata to be passed to userspace
with packets.

There is a merge conflict in net/openvswitch/vport-internal_dev.c:
A existing commit modifies internal_dev_mac_addr() and a new commit
deletes it.  The new one is correct, so you can just remove that function.

The following changes since commit a5a81f0b9025867efb999d14a8dfc1907c5a4c3b:

  ipv6: Fix default route failover when CONFIG_IPV6_ROUTER_PREF=n (2012-12-03 15:34:47 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 4490108b4a5ada14c7be712260829faecc814ae5:

  openvswitch: Allow OVS_USERSPACE_ATTR_USERDATA to be variable length. (2013-02-22 16:29:22 -0800)

----------------------------------------------------------------
Ben Pfaff (1):
      openvswitch: Allow OVS_USERSPACE_ATTR_USERDATA to be variable length.

Jarno Rajahalme (2):
      linux/openvswitch.h: Make OVSP_LOCAL 32-bit.
      openvswitch: Change ENOENT return value to ENODEV in lookup_vport().

Thomas Graf (2):
      openvswitch: Use eth_mac_addr() instead of duplicating it
      openvswitch: Avoid useless holes in struct vport

 include/linux/openvswitch.h          |   13 +++++++------
 net/openvswitch/datapath.c           |   13 +++++++------
 net/openvswitch/datapath.h           |    2 +-
 net/openvswitch/vport-internal_dev.c |   14 ++------------
 net/openvswitch/vport.h              |    4 ++--
 5 files changed, 19 insertions(+), 27 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse@nicira.com>
Date: Fri, 15 Mar 2013 10:38:46 -0700

> A couple of minor enhancements for net-next/3.10.  The largest is an
> extension to allow variable length metadata to be passed to userspace
> with packets.
> 
> There is a merge conflict in net/openvswitch/vport-internal_dev.c:
> A existing commit modifies internal_dev_mac_addr() and a new commit
> deletes it.  The new one is correct, so you can just remove that function.

Pulled, thanks Jesse.

Thanks, in particular, for the heads up about the merge conflict.

[GIT net-next] Open vSwitch

[Jesse Gross]

This series of improvements for 3.8/net-next contains four components:
 * Support for modifying IPv6 headers
 * Support for matching and setting skb->mark for better integration with
   things like iptables
 * Ability to recognize the EtherType for RARP packets
 * Two small performance enhancements

The movement of ipv6_find_hdr() into exthdrs_core.c causes two small merge
conflicts.  I left it as is but can do the merge if you want.  The conflicts
are:
 * ipv6_find_hdr() and ipv6_find_tlv() were both moved to the bottom of
   exthdrs_core.c.  Both should stay.
 * A new use of ipv6_find_hdr() was added to net/netfilter/ipvs/ip_vs_core.c
   after this patch.  The IPVS user has two instances of the old constant
   name IP6T_FH_F_FRAG which has been renamed to IP6_FH_F_FRAG.

The following changes since commit d04d382980c86bdee9960c3eb157a73f8ed230cc:

  openvswitch: Store flow key len if ARP opcode is not request or reply. (2012-10-30 17:17:09 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 92eb1d477145b2e7780b5002e856f70b8c3d74da:

  openvswitch: Use RCU callback when detaching netdevices. (2012-11-28 14:04:34 -0800)

----------------------------------------------------------------
Ansis Atteka (3):
      ipv6: improve ipv6_find_hdr() to skip empty routing headers
      openvswitch: add ipv6 'set' action
      openvswitch: add skb mark matching and set action

Jesse Gross (2):
      ipv6: Move ipv6_find_hdr() out of Netfilter code.
      openvswitch: Use RCU callback when detaching netdevices.

Mehak Mahajan (1):
      openvswitch: Process RARP packets with ethertype 0x8035 similar to ARP packets.

Shan Wei (1):
      net: openvswitch: use this_cpu_ptr per-cpu helper

 include/linux/netfilter_ipv6/ip6_tables.h |    9 ---
 include/linux/openvswitch.h               |    1 +
 include/net/ipv6.h                        |   10 +++
 net/ipv6/exthdrs_core.c                   |  123 +++++++++++++++++++++++++++++
 net/ipv6/netfilter/ip6_tables.c           |  103 ------------------------
 net/netfilter/xt_HMARK.c                  |    8 +-
 net/openvswitch/actions.c                 |   97 +++++++++++++++++++++++
 net/openvswitch/datapath.c                |   27 ++++++-
 net/openvswitch/flow.c                    |   28 ++++++-
 net/openvswitch/flow.h                    |    8 +-
 net/openvswitch/vport-netdev.c            |   14 +++-
 net/openvswitch/vport-netdev.h            |    3 +
 net/openvswitch/vport.c                   |    5 +-
 13 files changed, 304 insertions(+), 132 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse@nicira.com>
Date: Thu, 29 Nov 2012 10:35:42 -0800

> This series of improvements for 3.8/net-next contains four components:
>  * Support for modifying IPv6 headers
>  * Support for matching and setting skb->mark for better integration with
>    things like iptables
>  * Ability to recognize the EtherType for RARP packets
>  * Two small performance enhancements
> 
> The movement of ipv6_find_hdr() into exthdrs_core.c causes two small merge
> conflicts.  I left it as is but can do the merge if you want.  The conflicts
> are:
>  * ipv6_find_hdr() and ipv6_find_tlv() were both moved to the bottom of
>    exthdrs_core.c.  Both should stay.
>  * A new use of ipv6_find_hdr() was added to net/netfilter/ipvs/ip_vs_core.c
>    after this patch.  The IPVS user has two instances of the old constant
>    name IP6T_FH_F_FRAG which has been renamed to IP6_FH_F_FRAG.

Pulled, thanks Jesse.

The merge conflict directions were particularly helpful.

If you ever do the merge yourself (I'm ambivalent about where you or I
do it), make sure you force the merge commit message to have a
description of the conflict resolution similarly to what you provided
here.

Thanks again.

[GIT net-next] Open vSwitch

[Jesse Gross]

Two feature additions to Open vSwitch for net-next/3.7.

The following changes since commit 0d7614f09c1ebdbaa1599a5aba7593f147bf96ee:

  Linux 3.6-rc1 (2012-08-02 16:38:10 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to 15eac2a74277bc7de68a7c2a64a7c91b4b6f5961:

  openvswitch: Increase maximum number of datapath ports. (2012-09-03 19:20:49 -0700)

----------------------------------------------------------------
Pravin B Shelar (2):
      openvswitch: Add support for network namespaces.
      openvswitch: Increase maximum number of datapath ports.

 net/openvswitch/actions.c            |    2 +-
 net/openvswitch/datapath.c           |  375 +++++++++++++++++++++-------------
 net/openvswitch/datapath.h           |   50 ++++-
 net/openvswitch/dp_notify.c          |    8 +-
 net/openvswitch/flow.c               |   11 +-
 net/openvswitch/flow.h               |    3 +-
 net/openvswitch/vport-internal_dev.c |    7 +-
 net/openvswitch/vport-netdev.c       |    2 +-
 net/openvswitch/vport.c              |   23 ++-
 net/openvswitch/vport.h              |    7 +-
 10 files changed, 317 insertions(+), 171 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse-l0M0P4e3n4LQT0dZR+AlfA@public.gmane.org>
Date: Tue,  4 Sep 2012 12:14:07 -0700

> Two feature additions to Open vSwitch for net-next/3.7.
> 
> The following changes since commit 0d7614f09c1ebdbaa1599a5aba7593f147bf96ee:
> 
>   Linux 3.6-rc1 (2012-08-02 16:38:10 -0700)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master
> 
> for you to fetch changes up to 15eac2a74277bc7de68a7c2a64a7c91b4b6f5961:
> 
>   openvswitch: Increase maximum number of datapath ports. (2012-09-03 19:20:49 -0700)

Pulled, thanks Jesse.

[GIT net-next] Open vSwitch

[Jesse Gross]

A few bug fixes and small enhancements for net-next/3.6.

The following changes since commit bf32fecdc1851ad9ca960f56771b798d17c26cf1:

  openvswitch: Add length check when retrieving TCP flags. (2012-04-02 14:28:57 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

for you to fetch changes up to efaac3bf087b1a6cec28f2a041e01c874d65390c:

  openvswitch: Fix typo in documentation. (2012-07-20 14:51:07 -0700)

----------------------------------------------------------------
Ansis Atteka (1):
      openvswitch: Do not send notification if ovs_vport_set_options() failed

Ben Pfaff (1):
      openvswitch: Check gso_type for correct sk_buff in queue_gso_packets().

Jesse Gross (2):
      openvswitch: Enable retrieval of TCP flags from IPv6 traffic.
      openvswitch: Reset upper layer protocol info on internal devices.

Leo Alterman (1):
      openvswitch: Fix typo in documentation.

Pravin B Shelar (1):
      openvswitch: Check currect return value from skb_gso_segment()

Raju Subramanian (1):
      openvswitch: Replace Nicira Networks.

 Documentation/networking/openvswitch.txt |    2 +-
 net/openvswitch/actions.c                |    2 +-
 net/openvswitch/datapath.c               |   13 ++++++++-----
 net/openvswitch/datapath.h               |    2 +-
 net/openvswitch/dp_notify.c              |    2 +-
 net/openvswitch/flow.c                   |    5 +++--
 net/openvswitch/flow.h                   |    2 +-
 net/openvswitch/vport-internal_dev.c     |   10 +++++++++-
 net/openvswitch/vport-internal_dev.h     |    2 +-
 net/openvswitch/vport-netdev.c           |    2 +-
 net/openvswitch/vport-netdev.h           |    2 +-
 net/openvswitch/vport.c                  |    2 +-
 net/openvswitch/vport.h                  |    2 +-
 13 files changed, 30 insertions(+), 18 deletions(-)

Re: [GIT net-next] Open vSwitch

[David Miller]

From: Jesse Gross <jesse-l0M0P4e3n4LQT0dZR+AlfA@public.gmane.org>
Date: Fri, 20 Jul 2012 15:26:43 -0700

> A few bug fixes and small enhancements for net-next/3.6.
> 
> The following changes since commit bf32fecdc1851ad9ca960f56771b798d17c26cf1:
> 
>   openvswitch: Add length check when retrieving TCP flags. (2012-04-02 14:28:57 -0700)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/jesse/openvswitch.git master

Pulled, thanks Jesse.