Netfilter-Devel Archive on
 help / color / Atom feed
From: "Serguei Bezverkhi (sbezverk)" <>
To: Phil Sutter <>
Cc: "" <>
Subject: Re: Numen with reference to vmap
Date: Wed, 4 Dec 2019 13:47:47 +0000
Message-ID: <> (raw)
In-Reply-To: <>

Hello Phil,

Thank you for your reply. It is very unfortunate indeed. Here is the scenario where I thought to use a non-anonymous vmap.

Each k8s service can have 0, 1 or more associated endpoints, backends (pods providing this service). 0 endpoint already taken care of in filter prerouting hook.  When there are 1 or more, proxy needs to load balance incoming connections between endpoints.I thought to create vmap per service with 1 rule per service . When an endpoint gets updated (add/deleted) which could happen anytime then the only vmap get corresponding update and my hope was that automagically load balancing will be adjusted to use updated endpoints list.

With what you explained, I am not sure if dynamic load balancing is possible at all. If numgen work only with static anonymous vmap and fixed modulus , the only way to address this dynamic nature of endpoints is to recreate service rule everytime when number of endpoints changes (recalculate modulus and entries in vmap). I suspect it is way less efficient.
What will happen to dataplane and packets in transit when the rule will be deleted and then recreated? I suspect it might result in dropped packets, could you please comment on the possible impact?

If you could suggest a better approach for the described scenario, appreciate if you share it.

Thank you

On 2019-12-04, 5:18 AM, " on behalf of Phil Sutter" < on behalf of> wrote:

    Hi Serguei,
    On Wed, Dec 04, 2019 at 12:54:05AM +0000, Serguei Bezverkhi (sbezverk) wrote:
    > Nftables wiki gives this example for numgen:
    > nft add rule nat prerouting numgen random mod 2 vmap { 0 : jump mychain1, 1 : jump mychain2 }
    > I would like to use it but with map reference, like this:
    > nft add rule nat prerouting numgen random mod 2 vmap @service1-endpoints
    > Could you please confirm if it is supported? If it is what would be the type of the key in such map? I thought it would be integer, but command fails.
    > sudo nft --debug all add map ipv4table k8s-57XVOCFNTLTR3Q27-endpoints   { type  integer : verdict \; }
    > Error: unqualified key type integer specified in map definition
    > add map ipv4table k8s-57XVOCFNTLTR3Q27-endpoints { type integer : verdict ; }
    >                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
    Yes, this is sadly not possible right now. numgen type is 32bit integer,
    but we don't have a type definition matching that. Type 'integer' is
    unqualified regarding size, therefore unsuitable for use in map/set
    This all works when using anonymous set/map because key type is
    deduced from map LHS.
    We plan to support a 'typeof' keyword at some point to allow for the
    same deduction from within named map/set declarations, but it needs
    further work as the type info is lost on return path (when listing) so
    it would create a ruleset that can't be fed back.
    > The ultimate  goal is to update dynamically just the  map  with available endpoints and loadbalance between them without  touching the rule.
    I don't quite understand why you need to dynamically change the
    load-balancing rule: numgen modulus is fixed anyway, so the number of
    elements in vmap are fixed. Maybe just jump to chains and dynamically
    update those instead?
    Cheers, Phil

  reply index

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-04  0:54 Serguei Bezverkhi (sbezverk)
2019-12-04 10:18 ` Phil Sutter
2019-12-04 13:47   ` Serguei Bezverkhi (sbezverk) [this message]
2019-12-04 15:17     ` Phil Sutter
2019-12-04 15:42       ` Serguei Bezverkhi (sbezverk)
2019-12-04 15:56         ` Phil Sutter
2019-12-04 16:13           ` Serguei Bezverkhi (sbezverk)
2019-12-04 17:00             ` Phil Sutter
2019-12-04 17:31           ` Arturo Borrero Gonzalez
2019-12-04 17:49             ` Serguei Bezverkhi (sbezverk)
2019-12-04 21:05               ` Serguei Bezverkhi (sbezverk)
2019-12-04 22:32             ` Phil Sutter
2019-12-17  0:51               ` Serguei Bezverkhi (sbezverk)
2019-12-17 12:29                 ` Phil Sutter
2019-12-17 14:05                   ` Serguei Bezverkhi (sbezverk)
2019-12-17 16:41                     ` Phil Sutter
2019-12-18 17:01                       ` Serguei Bezverkhi (sbezverk)
2019-12-18 17:24                         ` Phil Sutter
2019-12-18 19:43                           ` Serguei Bezverkhi (sbezverk)
2019-12-18 19:58                             ` Laura Garcia
2019-12-18 20:54                               ` Serguei Bezverkhi (sbezverk)
2019-12-19 10:48                               ` Phil Sutter
2019-12-19 14:59                                 ` Serguei Bezverkhi (sbezverk)
2019-12-19 15:45                                   ` Phil Sutter
2019-12-19 16:00                                     ` Serguei Bezverkhi (sbezverk)
2019-12-19 18:19                                       ` Serguei Bezverkhi (sbezverk)
2020-01-04 12:30                                         ` Serguei Bezverkhi (sbezverk)

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Netfilter-Devel Archive on

Archives are clonable:
	git clone --mirror netfilter-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netfilter-devel netfilter-devel/ \
	public-inbox-index netfilter-devel

Example config snippet for mirrors

Newsgroup available over NNTP:

AGPL code for this site: git clone