* [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages
@ 2024-03-27 18:16 Donald Hunter
2024-03-27 18:16 ` [PATCH net-next v1 1/2] doc/netlink/specs: Add draft nftables spec Donald Hunter
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Donald Hunter @ 2024-03-27 18:16 UTC (permalink / raw)
To: netdev, Jakub Kicinski, David S. Miller, Eric Dumazet,
Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev
Cc: donald.hunter, Donald Hunter
This series adds a ynl spec for nftables and extends ynl with a --multi
command line option that makes it possible to send transactional batches
for nftables.
An example of usage is:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi batch-begin '{"res-id": 10}' \
--multi newtable '{"name": "test", "nfgen-family": 1}' \
--multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--multi batch-end '{"res-id": 10}'
Donald Hunter (2):
doc/netlink/specs: Add draft nftables spec
tools/net/ynl: Add multi message support to ynl
Documentation/netlink/specs/nftables.yaml | 1264 +++++++++++++++++++++
tools/net/ynl/cli.py | 22 +-
tools/net/ynl/lib/ynl.py | 47 +-
3 files changed, 1315 insertions(+), 18 deletions(-)
create mode 100644 Documentation/netlink/specs/nftables.yaml
--
2.44.0
^ permalink raw reply [flat|nested] 14+ messages in thread* [PATCH net-next v1 1/2] doc/netlink/specs: Add draft nftables spec 2024-03-27 18:16 [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Donald Hunter @ 2024-03-27 18:16 ` Donald Hunter 2024-03-27 18:17 ` [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl Donald Hunter 2024-03-27 22:45 ` [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Pablo Neira Ayuso 2 siblings, 0 replies; 14+ messages in thread From: Donald Hunter @ 2024-03-27 18:16 UTC (permalink / raw) To: netdev, Jakub Kicinski, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev Cc: donald.hunter, Donald Hunter Add a spec for nftables that has nearly complete coverage of the ops, but limited coverage of rule types and subexpressions. Signed-off-by: Donald Hunter <donald.hunter@gmail.com> --- Documentation/netlink/specs/nftables.yaml | 1264 +++++++++++++++++++++ 1 file changed, 1264 insertions(+) create mode 100644 Documentation/netlink/specs/nftables.yaml diff --git a/Documentation/netlink/specs/nftables.yaml b/Documentation/netlink/specs/nftables.yaml new file mode 100644 index 000000000000..dff2a18f3d90 --- /dev/null +++ b/Documentation/netlink/specs/nftables.yaml @@ -0,0 +1,1264 @@ +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) + +name: nftables +protocol: netlink-raw +protonum: 12 + +doc: + Netfilter nftables configuration over netlink. + +definitions: + - + name: nfgenmsg + type: struct + members: + - + name: nfgen-family + type: u8 + - + name: version + type: u8 + - + name: res-id + byte-order: big-endian + type: u16 + - + name: meta-keys + type: enum + entries: + - len + - protocol + - priority + - mark + - iif + - oif + - iifname + - oifname + - iftype + - oiftype + - skuid + - skgid + - nftrace + - rtclassid + - secmark + - nfproto + - l4-proto + - bri-iifname + - bri-oifname + - pkttype + - cpu + - iifgroup + - oifgroup + - cgroup + - prandom + - secpath + - iifkind + - oifkind + - bri-iifpvid + - bri-iifvproto + - time-ns + - time-day + - time-hour + - sdif + - sdifname + - bri-broute + - + name: cmp-ops + type: enum + entries: + - eq + - neq + - lt + - lte + - gt + - gte + - + name: object-type + type: enum + entries: + - unspec + - counter + - quota + - ct-helper + - limit + - connlimit + - tunnel + - ct-timeout + - secmark + - ct-expect + - synproxy + - + name: nat-range-flags + type: flags + entries: + - map-ips + - proto-specified + - proto-random + - persistent + - proto-random-fully + - proto-offset + - netmap + - + name: table-flags + type: flags + entries: + - dormant + - owner + - persist + - + name: chain-flags + type: flags + entries: + - base + - hw-offload + - binding + - + name: set-flags + type: flags + entries: + - anonymous + - constant + - interval + - map + - timeout + - eval + - object + - concat + - expr + +attribute-sets: + - + name: empty-attrs + attributes: + - + name: name + type: string + - + name: batch-attrs + attributes: + - + name: genid + type: u32 + byte-order: big-endian + - + name: table-attrs + attributes: + - + name: name + type: string + doc: name of the table + - + name: flags + type: u32 + byte-order: big-endian + doc: bitmask of flags + enum: table-flags + enum-as-flags: true + - + name: use + type: u32 + byte-order: big-endian + doc: number of chains in this table + - + name: handle + type: u64 + byte-order: big-endian + doc: numeric handle of the table + - + name: userdata + type: binary + doc: user data + - + name: chain-attrs + attributes: + - + name: table + type: string + doc: name of the table containing the chain + - + name: handle + type: u64 + byte-order: big-endian + doc: numeric handle of the chain + - + name: name + type: string + doc: name of the chain + - + name: hook + type: nest + nested-attributes: nft-hook-attrs + doc: hook specification for basechains + - + name: policy + type: u32 + byte-order: big-endian + doc: numeric policy of the chain + - + name: use + type: u32 + byte-order: big-endian + doc: number of references to this chain + - + name: type + type: string + doc: type name of the chain + - + name: counters + type: nest + nested-attributes: nft-counter-attrs + doc: counter specification of the chain + - + name: flags + type: u32 + byte-order: big-endian + doc: chain flags + enum: chain-flags + enum-as-flags: true + - + name: id + type: u32 + byte-order: big-endian + doc: uniquely identifies a chain in a transaction + - + name: userdata + type: binary + doc: user data + - + name: counter-attrs + attributes: + - + name: bytes + type: u64 + byte-order: big-endian + - + name: packets + type: u64 + byte-order: big-endian + - + name: pad + type: pad + - + name: nft-hook-attrs + attributes: + - + name: num + type: u32 + byte-order: big-endian + - + name: priority + type: s32 + byte-order: big-endian + - + name: dev + type: string + doc: net device name + - + name: devs + type: nest + nested-attributes: hook-dev-attrs + doc: list of net devices + - + name: hook-dev-attrs + attributes: + - + name: name + type: string + multi-attr: true + - + name: nft-counter-attrs + attributes: + - + name: bytes + type: u64 + - + name: packets + type: u64 + - + name: rule-attrs + attributes: + - + name: table + type: string + doc: name of the table containing the rule + - + name: chain + type: string + doc: name of the chain containing the rule + - + name: handle + type: u64 + byte-order: big-endian + doc: numeric handle of the rule + - + name: expressions + type: nest + nested-attributes: expr-list-attrs + doc: list of expressions + - + name: compat + type: nest + nested-attributes: rule-compat-attrs + doc: compatibility specifications of the rule + - + name: position + type: u64 + byte-order: big-endian + doc: numeric handle of the previous rule + - + name: userdata + type: binary + doc: user data + - + name: id + type: u32 + doc: uniquely identifies a rule in a transaction + - + name: position-id + type: u32 + doc: transaction unique identifier of the previous rule + - + name: chain-id + type: u32 + doc: add the rule to chain by ID, alternative to chain name + - + name: expr-list-attrs + attributes: + - + name: elem + type: nest + nested-attributes: expr-attrs + multi-attr: true + - + name: expr-attrs + attributes: + - + name: name + type: string + doc: name of the expression type + - + name: data + type: sub-message + sub-message: expr-ops + selector: name + doc: type specific data + - + name: rule-compat-attrs + attributes: + - + name: proto + type: binary + doc: numeric value of the handled protocol + - + name: flags + type: binary + doc: bitmask of flags + - + name: set-attrs + attributes: + - + name: table + type: string + doc: table name + - + name: name + type: string + doc: set name + - + name: flags + type: u32 + enum: set-flags + byte-order: big-endian + doc: bitmask of enum nft_set_flags + - + name: key-type + type: u32 + byte-order: big-endian + doc: key data type, informational purpose only + - + name: key-len + type: u32 + byte-order: big-endian + doc: key data length + - + name: data-type + type: u32 + byte-order: big-endian + doc: mapping data type + - + name: data-len + type: u32 + byte-order: big-endian + doc: mapping data length + - + name: policy + type: u32 + byte-order: big-endian + doc: selection policy + - + name: desc + type: nest + nested-attributes: set-desc-attrs + doc: set description + - + name: id + type: u32 + doc: uniquely identifies a set in a transaction + - + name: timeout + type: u64 + doc: default timeout value + - + name: gc-interval + type: u32 + doc: garbage collection interval + - + name: userdata + type: binary + doc: user data + - + name: pad + type: pad + - + name: obj-type + type: u32 + byte-order: big-endian + doc: stateful object type + - + name: handle + type: u64 + byte-order: big-endian + doc: set handle + - + name: expr + type: nest + nested-attributes: expr-attrs + doc: set expression + multi-attr: true + - + name: expressions + type: nest + nested-attributes: set-list-attrs + doc: list of expressions + - + name: set-desc-attrs + attributes: + - + name: size + type: u32 + byte-order: big-endian + doc: number of elements in set + - + name: concat + type: nest + nested-attributes: set-desc-concat-attrs + doc: description of field concatenation + multi-attr: true + - + name: set-desc-concat-attrs + attributes: + - + name: elem + type: nest + nested-attributes: set-field-attrs + - + name: set-field-attrs + attributes: + - + name: len + type: u32 + byte-order: big-endian + - + name: set-list-attrs + attributes: + - + name: elem + type: nest + nested-attributes: expr-attrs + multi-attr: true + - + name: setelem-attrs + attributes: + - + name: key + type: nest + nested-attributes: data-attrs + doc: key value + - + name: data + type: nest + nested-attributes: data-attrs + doc: data value of mapping + - + name: flags + type: binary + doc: bitmask of nft_set_elem_flags + - + name: timeout + type: u64 + doc: timeout value + - + name: expiration + type: u64 + doc: expiration time + - + name: userdata + type: binary + doc: user data + - + name: expr + type: nest + nested-attributes: expr-attrs + doc: expression + - + name: objref + type: string + doc: stateful object reference + - + name: key-end + type: nest + nested-attributes: data-attrs + doc: closing key value + - + name: expressions + type: nest + nested-attributes: expr-list-attrs + doc: list of expressions + - + name: setelem-list-elem-attrs + attributes: + - + name: elem + type: nest + nested-attributes: setelem-attrs + multi-attr: true + - + name: setelem-list-attrs + attributes: + - + name: table + type: string + - + name: set + type: string + - + name: elements + type: nest + nested-attributes: setelem-list-elem-attrs + - + name: set-id + type: u32 + - + name: gen-attrs + attributes: + - + name: id + type: u32 + byte-order: big-endian + doc: ruleset generation id + - + name: proc-pid + type: u32 + byte-order: big-endian + - + name: proc-name + type: string + - + name: obj-attrs + attributes: + - + name: table + type: string + doc: name of the table containing the expression + - + name: name + type: string + doc: name of this expression type + - + name: type + type: u32 + enum: object-type + byte-order: big-endian + doc: stateful object type + - + name: data + type: sub-message + sub-message: obj-data + selector: type + doc: stateful object data + - + name: use + type: u32 + byte-order: big-endian + doc: number of references to this expression + - + name: handle + type: u64 + byte-order: big-endian + doc: object handle + - + name: pad + type: pad + - + name: userdata + type: binary + doc: user data + - + name: quota-attrs + attributes: + - + name: bytes + type: u64 + byte-order: big-endian + - + name: flags # TODO + type: u32 + byte-order: big-endian + - + name: pad + type: pad + - + name: consumed + type: u64 + byte-order: big-endian + - + name: flowtable-attrs + attributes: + - + name: table + type: string + - + name: name + type: string + - + name: hook + type: nest + nested-attributes: flowtable-hook-attrs + - + name: use + type: u32 + byte-order: big-endian + - + name: handle + type: u64 + byte-order: big-endian + - + name: pad + type: pad + - + name: flags + type: u32 + byte-order: big-endian + - + name: flowtable-hook-attrs + attributes: + - + name: num + type: u32 + byte-order: big-endian + - + name: priority + type: u32 + byte-order: big-endian + - + name: devs + type: nest + nested-attributes: hook-dev-attrs + - + name: expr-cmp-attrs + attributes: + - + name: sreg + type: u32 + byte-order: big-endian + - + name: op + type: u32 + byte-order: big-endian + enum: cmp-ops + - + name: data + type: nest + nested-attributes: data-attrs + - + name: data-attrs + attributes: + - + name: value + type: binary + # sub-type: u8 + - + name: verdict + type: nest + nested-attributes: verdict-attrs + - + name: verdict-attrs + attributes: + - + name: code + type: u32 + byte-order: big-endian + - + name: chain + type: string + - + name: chain-id + type: u32 + - + name: expr-counter-attrs + attributes: + - + name: bytes + type: u64 + doc: Number of bytes + - + name: packets + type: u64 + doc: Number of packets + - + name: pad + type: pad + - + name: expr-flow-offload-attrs + attributes: + - + name: name + type: string + doc: Flow offload table name + - + name: expr-immediate-attrs + attributes: + - + name: dreg + type: u32 + byte-order: big-endian + - + name: data + type: nest + nested-attributes: data-attrs + - + name: expr-meta-attrs + attributes: + - + name: dreg + type: u32 + byte-order: big-endian + - + name: key + type: u32 + byte-order: big-endian + enum: meta-keys + - + name: sreg + type: u32 + byte-order: big-endian + - + name: expr-nat-attrs + attributes: + - + name: type + type: u32 + byte-order: big-endian + - + name: family + type: u32 + byte-order: big-endian + - + name: reg-addr-min + type: u32 + byte-order: big-endian + - + name: reg-addr-max + type: u32 + byte-order: big-endian + - + name: reg-proto-min + type: u32 + byte-order: big-endian + - + name: reg-proto-max + type: u32 + byte-order: big-endian + - + name: flags + type: u32 + byte-order: big-endian + enum: nat-range-flags + enum-as-flags: true + - + name: expr-payload-attrs + attributes: + - + name: dreg + type: u32 + byte-order: big-endian + - + name: base + type: u32 + byte-order: big-endian + - + name: offset + type: u32 + byte-order: big-endian + - + name: len + type: u32 + byte-order: big-endian + - + name: sreg + type: u32 + byte-order: big-endian + - + name: csum-type + type: u32 + byte-order: big-endian + - + name: csum-offset + type: u32 + byte-order: big-endian + - + name: csum-flags + type: u32 + byte-order: big-endian + - + name: expr-tproxy-attrs + attributes: + - + name: family + type: u32 + byte-order: big-endian + - + name: reg-addr + type: u32 + byte-order: big-endian + - + name: reg-port + type: u32 + byte-order: big-endian + +sub-messages: + - + name: expr-ops + formats: + - + value: bitwise # TODO + - + value: cmp + attribute-set: expr-cmp-attrs + - + value: counter + attribute-set: expr-counter-attrs + - + value: ct # TODO + - + value: flow_offload + attribute-set: expr-flow-offload-attrs + - + value: immediate + attribute-set: expr-immediate-attrs + - + value: lookup # TODO + - + value: meta + attribute-set: expr-meta-attrs + - + value: nat + attribute-set: expr-nat-attrs + - + value: payload + attribute-set: expr-payload-attrs + - + value: tproxy + attribute-set: expr-tproxy-attrs + - + name: obj-data + formats: + - + value: counter + attribute-set: counter-attrs + - + value: quota + attribute-set: quota-attrs + +operations: + enum-model: directional + list: + - + name: batch-begin + doc: Start a batch of operations + attribute-set: batch-attrs + fixed-header: nfgenmsg + do: + request: + value: 0x10 + attributes: + - genid + reply: + value: 0x10 + attributes: + - genid + - + name: batch-end + doc: Finish a batch of operations + attribute-set: batch-attrs + fixed-header: nfgenmsg + do: + request: + value: 0x11 + attributes: + - genid + - + name: newtable + doc: Create a new table. + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa00 + attributes: + - name + - + name: gettable + doc: Get / dump tables. + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa01 + attributes: + - name + reply: + value: 0xa00 + attributes: + - name + - + name: deltable + doc: Delete an existing table. + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa02 + attributes: + - name + - + name: destroytable + doc: Delete an existing table with destroy semantics (ignoring ENOENT errors). + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1a + attributes: + - name + - + name: newchain + doc: Create a new chain. + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa03 + attributes: + - name + - + name: getchain + doc: Get / dump chains. + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa04 + attributes: + - name + reply: + value: 0xa03 + attributes: + - name + - + name: delchain + doc: Delete an existing chain. + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa05 + attributes: + - name + - + name: destroychain + doc: Delete an existing chain with destroy semantics (ignoring ENOENT errors). + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1b + attributes: + - name + - + name: newrule + doc: Create a new rule. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa06 + attributes: + - name + - + name: getrule + doc: Get / dump rules. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa07 + attributes: + - name + reply: + value: 0xa06 + attributes: + - name + - + name: getrule-reset + doc: Get / dump rules and reset stateful expressions. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa19 + attributes: + - name + reply: + value: 0xa06 + attributes: + - name + - + name: delrule + doc: Delete an existing rule. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa08 + attributes: + - name + - + name: destroyrule + doc: Delete an existing rule with destroy semantics (ignoring ENOENT errors). + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1c + attributes: + - name + - + name: newset + doc: Create a new set. + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa09 + attributes: + - name + - + name: getset + doc: Get / dump sets. + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0a + attributes: + - name + reply: + value: 0xa09 + attributes: + - name + - + name: delset + doc: Delete an existing set. + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0b + attributes: + - name + - + name: destroyset + doc: Delete an existing set with destroy semantics (ignoring ENOENT errors). + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1d + attributes: + - name + - + name: newsetelem + doc: Create a new set element. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0c + attributes: + - name + - + name: getsetelem + doc: Get / dump set elements. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0d + attributes: + - name + reply: + value: 0xa0c + attributes: + - name + - + name: getsetelem-reset + doc: Get / dump set elements and reset stateful expressions. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa21 + attributes: + - name + reply: + value: 0xa0c + attributes: + - name + - + name: delsetelem + doc: Delete an existing set element. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0e + attributes: + - name + - + name: destroysetelem + doc: Delete an existing set element with destroy semantics. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1e + attributes: + - name + - + name: getgen + doc: Get / dump rule-set generation. + attribute-set: gen-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa10 + attributes: + - name + reply: + value: 0xa0f + attributes: + - name + - + name: newobj + doc: Create a new stateful object. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa12 + attributes: + - name + - + name: getobj + doc: Get / dump stateful objects. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa13 + attributes: + - name + reply: + value: 0xa12 + attributes: + - name + - + name: delobj + doc: Delete an existing stateful object. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa14 + attributes: + - name + - + name: destroyobj + doc: Delete an existing stateful object with destroy semantics. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1f + attributes: + - name + - + name: newflowtable + doc: Create a new flow table. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa16 + attributes: + - name + - + name: getflowtable + doc: Get / dump flow tables. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa17 + attributes: + - name + reply: + value: 0xa16 + attributes: + - name + - + name: delflowtable + doc: Delete an existing flow table. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa18 + attributes: + - name + - + name: destroyflowtable + doc: Delete an existing flow table with destroy semantics. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa20 + attributes: + - name + +mcast-groups: + list: + - + name: mgmt -- 2.44.0 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl 2024-03-27 18:16 [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Donald Hunter 2024-03-27 18:16 ` [PATCH net-next v1 1/2] doc/netlink/specs: Add draft nftables spec Donald Hunter @ 2024-03-27 18:17 ` Donald Hunter 2024-03-29 0:57 ` Jakub Kicinski 2024-03-27 22:45 ` [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Pablo Neira Ayuso 2 siblings, 1 reply; 14+ messages in thread From: Donald Hunter @ 2024-03-27 18:17 UTC (permalink / raw) To: netdev, Jakub Kicinski, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev Cc: donald.hunter, Donald Hunter Add a "--multi <op> <json>" command line to ynl that makes it possible to add several operations to a single netlink request payload. The --multi command line option is repeated for each operation. This is used by the nftables family for transaction batches. For example: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi batch-begin '{"res-id": 10}' \ --multi newtable '{"name": "test", "nfgen-family": 1}' \ --multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \ --multi batch-end '{"res-id": 10}' Signed-off-by: Donald Hunter <donald.hunter@gmail.com> --- tools/net/ynl/cli.py | 22 ++++++++++++++++--- tools/net/ynl/lib/ynl.py | 47 +++++++++++++++++++++++++++------------- 2 files changed, 51 insertions(+), 18 deletions(-) diff --git a/tools/net/ynl/cli.py b/tools/net/ynl/cli.py index f131e33ac3ee..1b8f87b472ba 100755 --- a/tools/net/ynl/cli.py +++ b/tools/net/ynl/cli.py @@ -19,13 +19,23 @@ class YnlEncoder(json.JSONEncoder): def main(): - parser = argparse.ArgumentParser(description='YNL CLI sample') + description = """ + YNL CLI utility - a general purpose netlink utility that uses YNL specs + to drive protocol encoding and decoding. + """ + epilog = """ + The --multi option can be repeated to include several operations + in the same netlink payload. + """ + + parser = argparse.ArgumentParser(description=description, + epilog=epilog) parser.add_argument('--spec', dest='spec', type=str, required=True) parser.add_argument('--schema', dest='schema', type=str) parser.add_argument('--no-schema', action='store_true') parser.add_argument('--json', dest='json_text', type=str) - parser.add_argument('--do', dest='do', type=str) - parser.add_argument('--dump', dest='dump', type=str) + parser.add_argument('--do', dest='do', metavar='OPERATION', type=str) + parser.add_argument('--dump', dest='dump', metavar='OPERATION', type=str) parser.add_argument('--sleep', dest='sleep', type=int) parser.add_argument('--subscribe', dest='ntf', type=str) parser.add_argument('--replace', dest='flags', action='append_const', @@ -40,6 +50,8 @@ def main(): parser.add_argument('--output-json', action='store_true') parser.add_argument('--dbg-small-recv', default=0, const=4000, action='store', nargs='?', type=int) + parser.add_argument('--multi', dest='multi', nargs=2, action='append', + metavar=('OPERATION', 'JSON_TEXT'), type=str) args = parser.parse_args() def output(msg): @@ -73,6 +85,10 @@ def main(): if args.dump: reply = ynl.dump(args.dump, attrs) output(reply) + if args.multi: + ops = [ (item[0], json.loads(item[1]), args.flags) for item in args.multi ] + reply = ynl.do_multi(ops) + output(reply) except NlError as e: print(e) exit(1) diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py index 557ef5a22b7d..cecd89db7d58 100644 --- a/tools/net/ynl/lib/ynl.py +++ b/tools/net/ynl/lib/ynl.py @@ -927,16 +927,11 @@ class YnlFamily(SpecFamily): return op['do']['request']['attributes'].copy() - def _op(self, method, vals, flags=None, dump=False): - op = self.ops[method] - + def _encode_message(self, op, vals, flags, req_seq): nl_flags = Netlink.NLM_F_REQUEST | Netlink.NLM_F_ACK for flag in flags or []: nl_flags |= flag - if dump: - nl_flags |= Netlink.NLM_F_DUMP - req_seq = random.randint(1024, 65535) msg = self.nlproto.message(nl_flags, op.req_value, 1, req_seq) if op.fixed_header: msg += self._encode_struct(op.fixed_header, vals) @@ -944,8 +939,20 @@ class YnlFamily(SpecFamily): for name, value in vals.items(): msg += self._add_attr(op.attr_set.name, name, value, search_attrs) msg = _genl_msg_finalize(msg) + return msg - self.sock.send(msg, 0) + def _ops(self, ops): + reqs_by_seq = {} + req_seq = random.randint(1024, 65535) + payload = b'' + for (method, vals, flags) in ops: + op = self.ops[method] + msg = self._encode_message(op, vals, flags, req_seq) + reqs_by_seq[req_seq] = (op, msg) + payload += msg + req_seq += 1 + + self.sock.send(payload, 0) done = False rsp = [] @@ -954,8 +961,9 @@ class YnlFamily(SpecFamily): nms = NlMsgs(reply, attr_space=op.attr_set) self._recv_dbg_print(reply, nms) for nl_msg in nms: - if nl_msg.extack: - self._decode_extack(msg, op, nl_msg.extack) + if nl_msg.extack and nl_msg.nl_seq in reqs_by_seq: + (req_op, req_msg) = reqs_by_seq[nl_msg.nl_seq] + self._decode_extack(req_msg, req_op, nl_msg.extack) if nl_msg.error: raise NlError(nl_msg) @@ -963,13 +971,15 @@ class YnlFamily(SpecFamily): if nl_msg.extack: print("Netlink warning:") print(nl_msg) + del reqs_by_seq[nl_msg.nl_seq] done = True break decoded = self.nlproto.decode(self, nl_msg) + rsp_op = self.rsp_by_value[decoded.cmd()] # Check if this is a reply to our request - if nl_msg.nl_seq != req_seq or decoded.cmd() != op.rsp_value: + if nl_msg.nl_seq not in reqs_by_seq or decoded.cmd() != rsp_op.rsp_value: if decoded.cmd() in self.async_msg_ids: self.handle_ntf(decoded) continue @@ -977,19 +987,26 @@ class YnlFamily(SpecFamily): print('Unexpected message: ' + repr(decoded)) continue - rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name) + rsp_msg = self._decode(decoded.raw_attrs, rsp_op.attr_set.name) if op.fixed_header: - rsp_msg.update(self._decode_struct(decoded.raw, op.fixed_header)) + rsp_msg.update(self._decode_struct(decoded.raw, rsp_op.fixed_header)) rsp.append(rsp_msg) if not rsp: return None - if not dump and len(rsp) == 1: + if not Netlink.NLM_F_DUMP in flags and len(rsp) == 1: return rsp[0] return rsp + def _op(self, method, vals, flags): + ops = [(method, vals, flags)] + return self._ops(ops) + def do(self, method, vals, flags=None): - return self._op(method, vals, flags) + return self._op(method, vals, flags or []) def dump(self, method, vals): - return self._op(method, vals, [], dump=True) + return self._op(method, vals, [Netlink.NLM_F_DUMP]) + + def do_multi(self, ops): + return self._ops(ops) -- 2.44.0 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl 2024-03-27 18:17 ` [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl Donald Hunter @ 2024-03-29 0:57 ` Jakub Kicinski 2024-03-29 13:37 ` Donald Hunter 0 siblings, 1 reply; 14+ messages in thread From: Jakub Kicinski @ 2024-03-29 0:57 UTC (permalink / raw) To: Donald Hunter Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter On Wed, 27 Mar 2024 18:17:00 +0000 Donald Hunter wrote: > - parser = argparse.ArgumentParser(description='YNL CLI sample') > + description = """ > + YNL CLI utility - a general purpose netlink utility that uses YNL specs YNL specs is intentional or should have been YAML? :) > + to drive protocol encoding and decoding. > + """ > + epilog = """ > + The --multi option can be repeated to include several operations > + in the same netlink payload. > + """ > + > + parser = argparse.ArgumentParser(description=description, > + epilog=epilog) > parser.add_argument('--spec', dest='spec', type=str, required=True) > parser.add_argument('--schema', dest='schema', type=str) > parser.add_argument('--no-schema', action='store_true') > parser.add_argument('--json', dest='json_text', type=str) > - parser.add_argument('--do', dest='do', type=str) > - parser.add_argument('--dump', dest='dump', type=str) > + parser.add_argument('--do', dest='do', metavar='OPERATION', type=str) > + parser.add_argument('--dump', dest='dump', metavar='OPERATION', type=str) > parser.add_argument('--sleep', dest='sleep', type=int) > parser.add_argument('--subscribe', dest='ntf', type=str) > parser.add_argument('--replace', dest='flags', action='append_const', > @@ -40,6 +50,8 @@ def main(): > parser.add_argument('--output-json', action='store_true') > parser.add_argument('--dbg-small-recv', default=0, const=4000, > action='store', nargs='?', type=int) > + parser.add_argument('--multi', dest='multi', nargs=2, action='append', > + metavar=('OPERATION', 'JSON_TEXT'), type=str) We'd only support multiple "do" requests, I wonder if we should somehow call this out. Is --multi-do unnecessary extra typing? Code itself looks pretty good! ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl 2024-03-29 0:57 ` Jakub Kicinski @ 2024-03-29 13:37 ` Donald Hunter 2024-03-29 15:43 ` Jakub Kicinski 0 siblings, 1 reply; 14+ messages in thread From: Donald Hunter @ 2024-03-29 13:37 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter Jakub Kicinski <kuba@kernel.org> writes: > On Wed, 27 Mar 2024 18:17:00 +0000 Donald Hunter wrote: >> - parser = argparse.ArgumentParser(description='YNL CLI sample') >> + description = """ >> + YNL CLI utility - a general purpose netlink utility that uses YNL specs > > YNL specs is intentional or should have been YAML? :) I'm not sure it was intentional, but YAML is definitely better :-) >> + to drive protocol encoding and decoding. >> + """ >> + epilog = """ >> + The --multi option can be repeated to include several operations >> + in the same netlink payload. >> + """ >> + >> + parser = argparse.ArgumentParser(description=description, >> + epilog=epilog) >> parser.add_argument('--spec', dest='spec', type=str, required=True) >> parser.add_argument('--schema', dest='schema', type=str) >> parser.add_argument('--no-schema', action='store_true') >> parser.add_argument('--json', dest='json_text', type=str) >> - parser.add_argument('--do', dest='do', type=str) >> - parser.add_argument('--dump', dest='dump', type=str) >> + parser.add_argument('--do', dest='do', metavar='OPERATION', type=str) >> + parser.add_argument('--dump', dest='dump', metavar='OPERATION', type=str) >> parser.add_argument('--sleep', dest='sleep', type=int) >> parser.add_argument('--subscribe', dest='ntf', type=str) >> parser.add_argument('--replace', dest='flags', action='append_const', >> @@ -40,6 +50,8 @@ def main(): >> parser.add_argument('--output-json', action='store_true') >> parser.add_argument('--dbg-small-recv', default=0, const=4000, >> action='store', nargs='?', type=int) >> + parser.add_argument('--multi', dest='multi', nargs=2, action='append', >> + metavar=('OPERATION', 'JSON_TEXT'), type=str) > > We'd only support multiple "do" requests, I wonder if we should somehow > call this out. Is --multi-do unnecessary extra typing? I prefer --multi but will update the help text to say "DO-OPERATIION" and "... several do operations". > Code itself looks pretty good! ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl 2024-03-29 13:37 ` Donald Hunter @ 2024-03-29 15:43 ` Jakub Kicinski 2024-03-29 18:57 ` Donald Hunter 0 siblings, 1 reply; 14+ messages in thread From: Jakub Kicinski @ 2024-03-29 15:43 UTC (permalink / raw) To: Donald Hunter Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter On Fri, 29 Mar 2024 13:37:31 +0000 Donald Hunter wrote: > > We'd only support multiple "do" requests, I wonder if we should somehow > > call this out. Is --multi-do unnecessary extra typing? > > I prefer --multi but will update the help text to say "DO-OPERATIION" > and "... several do operations". Alright, technically doing multi-dump should also work, but maybe there's less of a benefit there, so we can keep the multi focused on do for now. Looking at the code again, are you sure we'll process all the responses not just the first one? Shouldn't this: + del reqs_by_seq[nl_msg.nl_seq] done = True be something like: del reqs_by_seq[nl_msg.nl_seq] done = len(reqs_by_seq) == 0 ? Would be good to add an example of multi executing some get operations. My other concern is the formatting of the response. For mutli we should probably retain the indexes, e.g. 3 dos should produce an array with a length of 3, some of the entries may be None if the command only acked. Would that make sense? ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl 2024-03-29 15:43 ` Jakub Kicinski @ 2024-03-29 18:57 ` Donald Hunter 2024-03-29 21:01 ` Donald Hunter 0 siblings, 1 reply; 14+ messages in thread From: Donald Hunter @ 2024-03-29 18:57 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter Jakub Kicinski <kuba@kernel.org> writes: > On Fri, 29 Mar 2024 13:37:31 +0000 Donald Hunter wrote: >> > We'd only support multiple "do" requests, I wonder if we should somehow >> > call this out. Is --multi-do unnecessary extra typing? >> >> I prefer --multi but will update the help text to say "DO-OPERATIION" >> and "... several do operations". > > Alright, technically doing multi-dump should also work, but maybe > there's less of a benefit there, so we can keep the multi focused > on do for now. > > Looking at the code again, are you sure we'll process all the responses > not just the first one? > > Shouldn't this: > > + del reqs_by_seq[nl_msg.nl_seq] > done = True > > be something like: > > del reqs_by_seq[nl_msg.nl_seq] > done = len(reqs_by_seq) == 0 > Hmm yes, that's a good catch. I need to check the DONE semantics for these nftables batch operations. > Would be good to add an example of multi executing some get operations. I think this was a blind spot on my part because nftables doesn't support batch for get operations: https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 I'll need to try using multi for gets without any batch messages and see how everything behaves. > My other concern is the formatting of the response. For mutli we should > probably retain the indexes, e.g. 3 dos should produce an array with a > length of 3, some of the entries may be None if the command only acked. > Would that make sense? As I said, a blind spot on my part - I didn't really think there was a need to do anything for None responses but if get can work then an array of responses will be needed. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl @ 2024-03-29 21:01 ` Donald Hunter 0 siblings, 0 replies; 14+ messages in thread From: Donald Hunter @ 2024-03-29 21:01 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter On Fri, 29 Mar 2024 at 18:58, Donald Hunter <donald.hunter@gmail.com> wrote: > > Jakub Kicinski <kuba@kernel.org> writes: > > > Looking at the code again, are you sure we'll process all the responses > > not just the first one? > > > > Shouldn't this: > > > > + del reqs_by_seq[nl_msg.nl_seq] > > done = True > > > > be something like: > > > > del reqs_by_seq[nl_msg.nl_seq] > > done = len(reqs_by_seq) == 0 > > > > Hmm yes, that's a good catch. I need to check the DONE semantics for > these nftables batch operations. Well that's a problem: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi batch-begin '{"res-id": 10}' \ --multi newtable '{"name": "test", "nfgen-family": 1}' \ --multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \ --multi batch-end '{"res-id": 10}' Adding: 20778 Adding: 20779 Adding: 20780 Adding: 20781 Done: 20779 Done: 20780 There's no response for 'batch-begin' or 'batch-end'. We may need a per op spec property to tell us if a request will be acknowledged. > > Would be good to add an example of multi executing some get operations. > > I think this was a blind spot on my part because nftables doesn't > support batch for get operations: > > https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 > > I'll need to try using multi for gets without any batch messages and see how > everything behaves. Okay, so it can be made to work. Will incorporate into the next revision: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi gettable '{"name": "test", "nfgen-family": 1}' \ --multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' [{'flags': set(), 'handle': 10, 'name': 'test', 'nfgen-family': 1, 'res-id': 200, 'use': 1, 'version': 0}, {'handle': 1, 'name': 'chain', 'nfgen-family': 1, 'res-id': 200, 'table': 'test', 'use': 0, 'version': 0}] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl @ 2024-03-29 21:01 ` Donald Hunter 0 siblings, 0 replies; 14+ messages in thread From: Donald Hunter @ 2024-03-29 21:01 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter On Fri, 29 Mar 2024 at 18:58, Donald Hunter <donald.hunter@gmail.com> wrote: > > Jakub Kicinski <kuba@kernel.org> writes: > > > Looking at the code again, are you sure we'll process all the responses > > not just the first one? > > > > Shouldn't this: > > > > + del reqs_by_seq[nl_msg.nl_seq] > > done = True > > > > be something like: > > > > del reqs_by_seq[nl_msg.nl_seq] > > done = len(reqs_by_seq) == 0 > > > > Hmm yes, that's a good catch. I need to check the DONE semantics for > these nftables batch operations. Well that's a problem: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi batch-begin '{"res-id": 10}' \ --multi newtable '{"name": "test", "nfgen-family": 1}' \ --multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \ --multi batch-end '{"res-id": 10}' Adding: 20778 Adding: 20779 Adding: 20780 Adding: 20781 Done: 20779 Done: 20780 There's no response for 'batch-begin' or 'batch-end'. We may need a per op spec property to tell us if a request will be acknowledged. > > Would be good to add an example of multi executing some get operations. > > I think this was a blind spot on my part because nftables doesn't > support batch for get operations: > > https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 > > I'll need to try using multi for gets without any batch messages and see how > everything behaves. Okay, so it can be made to work. Will incorporate into the next revision: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi gettable '{"name": "test", "nfgen-family": 1}' \ --multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' [{'flags': set(), 'handle': 10, 'name': 'test', 'nfgen-family': 1, 'res-id': 200, 'use': 1, 'version': 0}, {'handle': 1, 'name': 'chain', 'nfgen-family': 1, 'res-id': 200, 'table': 'test', 'use': 0, 'version': 0}] X-sender: <netdev+bounces-83462-steffen.klassert=secunet.com@vger.kernel.org> X-Receiver: <steffen.klassert@secunet.com> ORCPT=rfc822;steffen.klassert@secunet.com NOTIFY=NEVER; X-ExtendedProps=BQAVABYAAgAAAAUAFAARAPDFCS25BAlDktII2g02frgPADUAAABNaWNyb3NvZnQuRXhjaGFuZ2UuVHJhbnNwb3J0LkRpcmVjdG9yeURhdGEuSXNSZXNvdXJjZQIAAAUAagAJAAEAAAAAAAAABQAWAAIAAAUAQwACAAAFAEYABwADAAAABQBHAAIAAAUAEgAPAGIAAAAvbz1zZWN1bmV0L291PUV4Y2hhbmdlIEFkbWluaXN0cmF0aXZlIEdyb3VwIChGWURJQk9IRjIzU1BETFQpL2NuPVJlY2lwaWVudHMvY249U3RlZmZlbiBLbGFzc2VydDY4YwUACwAXAL4AAACheZxkHSGBRqAcAp3ukbifQ049REI2LENOPURhdGFiYXNlcyxDTj1FeGNoYW5nZSBBZG1pbmlzdHJhdGl2ZSBHcm91cCAoRllESUJPSEYyM1NQRExUKSxDTj1BZG1pbmlzdHJhdGl2ZSBHcm91cHMsQ049c2VjdW5ldCxDTj1NaWNyb3NvZnQgRXhjaGFuZ2UsQ049U2VydmljZXMsQ049Q29uZmlndXJhdGlvbixEQz1zZWN1bmV0LERDPWRlBQAOABEABiAS9uuMOkqzwmEZDvWNNQUAHQAPAAwAAABtYngtZXNzZW4tMDIFADwAAgAADwA2AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5NYWlsUmVjaXBpZW50LkRpc3BsYXlOYW1lDwARAAAAS2xhc3NlcnQsIFN0ZWZmZW4FAAwAAgAABQBsAAIAAAUAWAAXAEoAAADwxQktuQQJQ5LSCNoNNn64Q049S2xhc3NlcnQgU3RlZmZlbixPVT1Vc2VycyxPVT1NaWdyYXRpb24sREM9c2VjdW5ldCxEQz1kZQUAJgACAAEFACIADwAxAAAAQXV0b1Jlc3BvbnNlU3VwcHJlc3M6IDANClRyYW5zbWl0SGlzdG9yeTogRmFsc2UNCg8ALwAAAE1pY3Jvc29mdC5FeGNoYW5nZS5UcmFuc3BvcnQuRXhwYW5zaW9uR3JvdXBUeXBlDwAVAAAATWVtYmVyc0dyb3VwRXhwYW5zaW9uBQAjAAIAAQ== X-CreatedBy: MSExchange15 X-HeloDomain: b.mx.secunet.com X-ExtendedProps: BQBjAAoAxkamlidQ3AgFAGEACAABAAAABQA3AAIAAA8APAAAAE1pY3Jvc29mdC5FeGNoYW5nZS5UcmFuc3BvcnQuTWFpbFJlY2lwaWVudC5Pcmdhbml6YXRpb25TY29wZREAAAAAAAAAAAAAAAAAAAAAAAUASQACAAEFAAQAFCABAAAAHAAAAHN0ZWZmZW4ua2xhc3NlcnRAc2VjdW5ldC5jb20FAAYAAgABBQApAAIAAQ8ACQAAAENJQXVkaXRlZAIAAQUAAgAHAAEAAAAFAAMABwAAAAAABQAFAAIAAQUAYgAKABgAAADMigAABQBkAA8AAwAAAEh1Yg== X-Source: SMTP:Default MBX-ESSEN-02 X-SourceIPAddress: 62.96.220.37 X-EndOfInjectedXHeaders: 16461 Received: from cas-essen-02.secunet.de (10.53.40.202) by mbx-essen-02.secunet.de (10.53.40.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.37; Fri, 29 Mar 2024 22:01:56 +0100 Received: from b.mx.secunet.com (62.96.220.37) by cas-essen-02.secunet.de (10.53.40.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Fri, 29 Mar 2024 22:01:56 +0100 Received: from localhost (localhost [127.0.0.1]) by b.mx.secunet.com (Postfix) with ESMTP id 3E45A2032C for <steffen.klassert@secunet.com>; Fri, 29 Mar 2024 22:01:56 +0100 (CET) X-Virus-Scanned: by secunet X-Spam-Flag: NO X-Spam-Score: -2.749 X-Spam-Level: X-Spam-Status: No, score=-2.749 tagged_above=-999 required=2.1 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: a.mx.secunet.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from b.mx.secunet.com ([127.0.0.1]) by localhost (a.mx.secunet.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fLkgBUKcTaam for <steffen.klassert@secunet.com>; Fri, 29 Mar 2024 22:01:55 +0100 (CET) Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=147.75.48.161; helo=sy.mirrors.kernel.org; envelope-from=netdev+bounces-83462-steffen.klassert=secunet.com@vger.kernel.org; receiver=steffen.klassert@secunet.com DKIM-Filter: OpenDKIM Filter v2.11.0 b.mx.secunet.com CEC86200BB Authentication-Results: b.mx.secunet.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ho+OjBNb" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [147.75.48.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by b.mx.secunet.com (Postfix) with ESMTPS id CEC86200BB for <steffen.klassert@secunet.com>; Fri, 29 Mar 2024 22:01:54 +0100 (CET) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 19968B2427F for <steffen.klassert@secunet.com>; Fri, 29 Mar 2024 21:01:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9F65213791F; Fri, 29 Mar 2024 21:01:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ho+OjBNb" X-Original-To: netdev@vger.kernel.org Received: from mail-oa1-f45.google.com (mail-oa1-f45.google.com [209.85.160.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92C8954BCC for <netdev@vger.kernel.org>; Fri, 29 Mar 2024 21:01:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711746083; cv=none; b=UYQIxJbF0iMH3ZsdAKsR2WhSOeVLknz6XBqGSZdIqglT4g0x+m1+iQAJgbpeDx4K/GP6IKrlLH6vUPW6kiO0E+mH+NyxvPsUhWQ2SDOSQlNMqnuxKt6p8AImZ9ZLnPja0WglzYtRgaxLdCFXDb/++aF693jqKAp/aq43EgHoYI0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711746083; c=relaxed/simple; bh=1F336ycsZ/DeFmR3ip4gKdMLUxslUjLhibn8LWK2nsQ=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=dqczGxbj+vc5Ex8+YOql2W5MFQyEyJDvs+VvKZxCZX/YbWFRDEeuR5cJnie4QDZZdUKkWF4hgxcR/Gn3m5DexMqEqSVK1Agq4Gr2DSCmglk4vO609sLx9zCF9KTbdIqb0rfTy0Qw8GBCP3KZDb0Qs3tYWRyoxcTbXU+z3FW17iA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ho+OjBNb; arc=none smtp.client-ip=209.85.160.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-oa1-f45.google.com with SMTP id 586e51a60fabf-2218a0f55e1so1281980fac.1 for <netdev@vger.kernel.org>; Fri, 29 Mar 2024 14:01:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711746080; x=1712350880; darn=vger.kernel.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=yf52Uh55Alu6R/Ag5oOPyX9j+A5vWytrrN0jiumVZ90=; b=ho+OjBNbnc3h384Al3xJv2eDnLkMbeh8PklqJBE+HkNQYmq+u3NA3WEUXY6ntS1oum BbDLftW/w1L3bnRUkzfKGFj6NOD91Ao5B7w8TbHmTTWygGGJMaI5oluhe6pXWRKlcwQE eal66+QsD61hUSz7DafBrrpV7r44uiME3jzCun3upnTJCiz+CzfvihUtTLxPyGFkXtzB 2XpAMIxS+WY4coIS9d3qLyHgN1MUdr9S9ZepzVwPYxull2EILd3FX8/YV0IsDk1Olzh8 4VhId5UWqiDJuZFox6GjK+EahBBbT5idHwBSumTfjPybPSAbMGWWeY+vfuxETCQO+JPu 21+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711746080; x=1712350880; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yf52Uh55Alu6R/Ag5oOPyX9j+A5vWytrrN0jiumVZ90=; b=mmXVo9v3DV+2/oxtG3OebzaU3GXy2AOrXBmmxq3it5M9egIWB6464eze8fJefK6BIP 4pVPO0njIslLwBaciyjuicRAS0q5IO2vziESOt8OJMs7B+cupw3i+uzxiQ07vrsp/ok0 KgXjIm7hYGtVo/vW2SYeLFBgtjr8wsYTLK30/iJ8DKvuIO07W4Vy215kpZfuDl1NGbWb zuTwYiVaKaQ1nwPZCV5BHV9cEVqsFYmN0p892Jpnt2Hrtpd8GRxNXMIWRGoy3UPYuE5P crdfjhOsuna/vY51d/Vdscks5gpZjbvLR3v/IZZ2W4enAtVGjnilS3Up6N69FPesoi6d 9hFQ== X-Gm-Message-State: AOJu0Yxy1Z1OQwgYOlgx6qQOrRpezxRv9fa1TVH11l25zaM22mXmBbSm AeQFFbIL2e6QZxC+BatQg1inIxrP8xE3ArNrMah7H0eVgXvXHFwIXqQ0/+4c6/0T85Bl/6xNwxN aa6cs7TpBTZPmb37LWn9+9ETHiq3WJK4j0cHKkg== X-Google-Smtp-Source: AGHT+IFNOkfSs1BURB1rlpqe3TL2grutEAtJuWGFXQ3+Z6VLEkc3CSL1O9fqzu1ibdOxjluwMoca7IZvOR8xkQpFSqc= X-Received: by 2002:a05:6870:6124:b0:229:eb17:3c19 with SMTP id s36-20020a056870612400b00229eb173c19mr3037666oae.35.1711746080466; Fri, 29 Mar 2024 14:01:20 -0700 (PDT) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: <netdev.vger.kernel.org> List-Subscribe: <mailto:netdev+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:netdev+unsubscribe@vger.kernel.org> MIME-Version: 1.0 References: <20240327181700.77940-1-donald.hunter@gmail.com> <20240327181700.77940-3-donald.hunter@gmail.com> <20240328175729.15208f4a@kernel.org> <m234s9jh0k.fsf@gmail.com> <20240329084346.7a744d1e@kernel.org> <m2plvcj27b.fsf@gmail.com> In-Reply-To: <m2plvcj27b.fsf@gmail.com> From: Donald Hunter <donald.hunter@gmail.com> Date: Fri, 29 Mar 2024 21:01:09 +0000 Message-ID: <CAD4GDZw0RW3B2n5vC-q-XLpQ_bCg0iP13qvOa=cjK37CPLJsKg@mail.gmail.com> Subject: Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl To: Jakub Kicinski <kuba@kernel.org> Cc: netdev@vger.kernel.org, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Paolo Abeni <pabeni@redhat.com>, Jiri Pirko <jiri@resnulli.us>, Jacob Keller <jacob.e.keller@intel.com>, Stanislav Fomichev <sdf@google.com>, donald.hunter@redhat.com Content-Type: text/plain; charset="UTF-8" Return-Path: netdev+bounces-83462-steffen.klassert=secunet.com@vger.kernel.org X-MS-Exchange-Organization-OriginalArrivalTime: 29 Mar 2024 21:01:56.2893 (UTC) X-MS-Exchange-Organization-Network-Message-Id: 474836ca-b8ec-45fc-ee52-08dc503377c7 X-MS-Exchange-Organization-OriginalClientIPAddress: 62.96.220.37 X-MS-Exchange-Organization-OriginalServerIPAddress: 10.53.40.202 X-MS-Exchange-Organization-Cross-Premises-Headers-Processed: cas-essen-02.secunet.de X-MS-Exchange-Organization-OrderedPrecisionLatencyInProgress: LSRV=mbx-essen-02.secunet.de:TOTAL-HUB=0.213|SMR=0.137(SMRDE=0.005|SMRC=0.131(SMRCL=0.102|X-SMRCR=0.131))|CAT=0.075(CATOS=0.001 |CATRESL=0.026(CATRESLP2R=0.018)|CATORES=0.045(CATRS=0.045(CATRS-Index Routing Agent=0.043 )));2024-03-29T21:01:56.520Z X-MS-Exchange-Forest-ArrivalHubServer: mbx-essen-02.secunet.de X-MS-Exchange-Organization-AuthSource: cas-essen-02.secunet.de X-MS-Exchange-Organization-AuthAs: Anonymous X-MS-Exchange-Organization-FromEntityHeader: Internet X-MS-Exchange-Organization-OriginalSize: 10422 X-MS-Exchange-Organization-HygienePolicy: Standard X-MS-Exchange-Organization-MessageLatency: SRV=cas-essen-02.secunet.de:TOTAL-FE=0.017|SMR=0.007(SMRPI=0.004(SMRPI-FrontendProxyAgent=0.004))|SMS=0.011 X-MS-Exchange-Organization-Recipient-Limit-Verified: True X-MS-Exchange-Organization-TotalRecipientCount: 1 X-MS-Exchange-Organization-Rules-Execution-History: 0b0cf904-14ac-4724-8bdf-482ee6223cf2%%%fd34672d-751c-45ae-a963-ed177fcabe23%%%d8080257-b0c3-47b4-b0db-23bc0c8ddb3c%%%95e591a2-5d7d-4afa-b1d0-7573d6c0a5d9%%%f7d0f6bc-4dcc-4876-8c5d-b3d6ddbb3d55%%%16355082-c50b-4214-9c7d-d39575f9f79b X-MS-Exchange-Forest-RulesExecuted: mbx-essen-02 X-MS-Exchange-Organization-RulesExecuted: mbx-essen-02 X-MS-Exchange-Forest-IndexAgent-0: AQ0CZW4AAV0FAAAPAAADH4sIAAAAAAAEAL1UbW8bRRDei31+ucRNhU DwCUZB6Frh2G4EahK1pUgFtVCoBJXyoVTR+m5sL77bde/26lhV/x0/ jJld27HbUFEqYZ3vZmbn5Zm3/ev6Ew0/FqoLRyfwiyzgaHD0DUgLt4 5Pvz3uwgOjZZbCw0pbLOBO6tjexLH3x7lUWS8x+T2YF8biaSe6Rw/8 JKfVEH5WidLlVMEd4uT9KRYas54pxqytLJYr9Xvw2Jip0mOOaycIiU kR5Fgq3QVZICxMBWVFxBzjLINZYRIsS5BEs3qB5czoEkvvTBsLf1al dzVSBVFG43fu0Gv8PjFVluqYVZSDsTr5Gq74pZhRjBfl+XBxXuKLZz o7z8txjz7EPfeGV/6oWgh34WlR4UaMIUJpcqTQlHGmprgJ4EMjL0Nm qG9sGN6Eu3dhsArD34d5Dgssu1QBaWOqJYyNSSGRNpn04BFoxBSsgW SCydRV8sGTX3+AEnOprUpKGJmC/dBJiaBHVg4zLGHI9mBmWEirqCe9 TtSJztA1ahmHukeq+Smf9PrWmKzsa7T9hc76SaZ6swX80YlcMoeH5Q wTmsGkylFb55J1M6WnfT4iy2Xk3kLm2YZhXmVWeTiHQxwrDfGrAxqU Q5UenMKtwev4LWWNc+eLNbXMkfQOaErtQRcO9GiM+nAkc5Ut2P5q82 QifaCVuROwvXP8ng49eNTpW9A70fdpStNzStt6+/bxNnuyxR4Pttlb nYhWGteqa4YVO9HTCRZIbdJmvVXcaYg3KhnDpYDQxT04Q8jlws+M7E TUfRoBcL2jbhNrFzxLluegKkGNaAxoOiuqBcwVCWklZDLVZp5hOsbU TQ3P9hnvKZ+64SQXMqUIGvBC5jPqlBmBrxVeYFJZXiheLRij3R5CN/ OPeN311C09zCUP45BGKSWkhu8IyBcwk4WlgImsNsc6NVjSdcFOymo2 M6zjJp1rsx1sdalNrJ2Vp/0+ZupCFb2hMTy2fFn26Vtd9DPJs9AvTV Uk6DaA/iOV0cVKU33uI5/LmeolXz4+GZwcrbLgK3C1nrZYUEU5b1+H JR5KT1m65CwVa7GEmtOVKceUjOSMEWFi5uwPX2Kx8JfRECfyJfqlfT KViy5VE5SlW0FzE3JJ1zIFnZtiSk3nximdmILqQbkQzYDoptB4Yam/ L1VJBflf9pxS/pDVJfMPWt1O9OxVPMrkuIxPqbT2xs0uR4gnVOoMY1 5aL2DnxMbsKl6KNryxppf6dY95M5emNI+Xx9Qyri0JBq9Z8moj1HYk l8Z7hnIZvwHThx+8Hf4591eIHVGriXAnELv0MF2vM1sLhQhFs8FPqx GITx0digYph6JDREtE9UDsi5DkNX5fb4rWzspDyML6im7QkWfbYt8L 6e287bfJj/iKhQwg9B4IT9sT/P6iLpp1sU+0U/78ncqf1V3oS4WgFQ gRiI4DwGgDseMReoJ0HBJGRf5rLs2WaNM7Evv1lUlTXAsD8bHT9M+O 6NBpJxA3xTVK0FsRTiJCjt4IRbsWiAYjpChU6n2ns1cXLXpIZ080Gk Hbwbu2Pgo5dPtKhZAzJQDNbbek01r265+JetDYJtq+X4SZe7oE3Nyh viyjNNbdDFblJZr0PUFvelxlWp7dE5+Ezqq+bo2o+Sgkd73jvkfiI7 IiiWfXtHdIBJm0GN6ur6qvNlkRjt9oGkXTjxMduflsurmKyNbTziFV MlzD4NKtWry2dYn7VjYjjkUmDY9hnax7M7Zd59B7aFEf//Uk/Nd2k5 xw7hLrWtxyTna3dS7HoCZEjYet4eXNIPJO1pou0N67zR2M3Ted/A02 dvMXaQwAAAEKzwI8P3htbCB2ZXJzaW9uPSIxLjAiIGVuY29kaW5nPS J1dGYtMTYiPz4NCjxFbWFpbFNldD4NCiAgPFZlcnNpb24+MTUuMC4w LjA8L1ZlcnNpb24+DQogIDxFbWFpbHM+DQogICAgPEVtYWlsIFN0YX J0SW5kZXg9IjQ1IiBQb3NpdGlvbj0iU2lnbmF0dXJlIj4NCiAgICAg IDxFbWFpbFN0cmluZz5kb25hbGQuaHVudGVyQGdtYWlsLmNvbTwvRW 1haWxTdHJpbmc+DQogICAgPC9FbWFpbD4NCiAgICA8RW1haWwgU3Rh cnRJbmRleD0iOTkiPg0KICAgICAgPEVtYWlsU3RyaW5nPmt1YmFAa2 VybmVsLm9yZzwvRW1haWxTdHJpbmc+DQogICAgPC9FbWFpbD4NCiAg PC9FbWFpbHM+DQo8L0VtYWlsU2V0PgELgQM8P3htbCB2ZXJzaW9uPS IxLjAiIGVuY29kaW5nPSJ1dGYtMTYiPz4NCjxVcmxTZXQ+DQogIDxW ZXJzaW9uPjE1LjAuMC4wPC9WZXJzaW9uPg0KICA8VXJscz4NCiAgIC A8VXJsIFN0YXJ0SW5kZXg9IjMwNiIgVHlwZT0iVXJsIj4NCiAgICAg IDxVcmxTdHJpbmc+bmxfbXNnLm5sPC9VcmxTdHJpbmc+DQogICAgPC 9Vcmw+DQogICAgPFVybCBTdGFydEluZGV4PSIxMzY3IiBQb3NpdGlv bj0iT3RoZXIiIFR5cGU9IlVybCI+DQogICAgICA8VXJsU3RyaW5nPm h0dHBzOi8vZWxpeGlyLmJvb3RsaW4uY29tL2xpbnV4L2xhdGVzdC9z b3VyY2UvbmV0L25ldGZpbHRlci9uZl90YWJsZXNfYXBpLmMjTDkwOT I8L1VybFN0cmluZz4NCiAgICA8L1VybD4NCiAgPC9VcmxzPg0KPC9V cmxTZXQ+AQydBzw/eG1sIHZlcnNpb249IjEuMCIgZW5jb2Rpbmc9In V0Zi0xNiI/Pg0KPENvbnRhY3RTZXQ+DQogIDxWZXJzaW9uPjE1LjAu MC4wPC9WZXJzaW9uPg0KICA8Q29udGFjdHM+DQogICAgPENvbnRhY3 QgU3RhcnRJbmRleD0iMzAiIFBvc2l0aW9uPSJTaWduYXR1cmUiPg0K ICAgICAgPFBlcnNvbiBTdGFydEluZGV4PSIzMCIgUG9zaXRpb249Il NpZ25hdHVyZSI+DQogICAgICAgIDxQZXJzb25TdHJpbmc+RG9uYWxk IEh1bnRlcjwvUGVyc29uU3RyaW5nPg0KICAgICAgPC9QZXJzb24+DQ ogICAgICA8RW1haWxzPg0KICAgICAgICA8RW1haWwgU3RhcnRJbmRl eD0iNDUiIFBvc2l0aW9uPSJTaWduYXR1cmUiPg0KICAgICAgICAgID xFbWFpbFN0cmluZz5kb25hbGQuaHVudGVyQGdtYWlsLmNvbTwvRW1h aWxTdHJpbmc+DQogICAgICAgIDwvRW1haWw+DQogICAgICA8L0VtYW lscz4NCiAgICAgIDxDb250YWN0U3RyaW5nPkRvbmFsZCBIdW50ZXIg Jmx0O2RvbmFsZC5odW50ZXJAZ21haWwuY29tPC9Db250YWN0U3RyaW 5nPg0KICAgIDwvQ29udGFjdD4NCiAgICA8Q29udGFjdCBTdGFydElu ZGV4PSI4MyI+DQogICAgICA8UGVyc29uIFN0YXJ0SW5kZXg9IjgzIj 4NCiAgICAgICAgPFBlcnNvblN0cmluZz5KYWt1YiBLaWNpbnNraTwv UGVyc29uU3RyaW5nPg0KICAgICAgPC9QZXJzb24+DQogICAgICA8RW 1haWxzPg0KICAgICAgICA8RW1haWwgU3RhcnRJbmRleD0iOTkiPg0K ICAgICAgICAgIDxFbWFpbFN0cmluZz5rdWJhQGtlcm5lbC5vcmc8L0 VtYWlsU3RyaW5nPg0KICAgICAgICA8L0VtYWlsPg0KICAgICAgPC9F bWFpbHM+DQogICAgICA8Q29udGFjdFN0cmluZz5KYWt1YiBLaWNpbn NraSAmbHQ7a3ViYUBrZXJuZWwub3JnPC9Db250YWN0U3RyaW5nPg0K ICAgIDwvQ29udGFjdD4NCiAgPC9Db250YWN0cz4NCjwvQ29udGFjdF NldD4BDs8BUmV0cmlldmVyT3BlcmF0b3IsMTAsMjtSZXRyaWV2ZXJP cGVyYXRvciwxMSwyO1Bvc3REb2NQYXJzZXJPcGVyYXRvciwxMCwxO1 Bvc3REb2NQYXJzZXJPcGVyYXRvciwxMSwwO1Bvc3RXb3JkQnJlYWtl ckRpYWdub3N0aWNPcGVyYXRvciwxMCwzO1Bvc3RXb3JkQnJlYWtlck RpYWdub3N0aWNPcGVyYXRvciwxMSwwO1RyYW5zcG9ydFdyaXRlclBy b2R1Y2VyLDIwLDE3 X-MS-Exchange-Forest-IndexAgent: 1 3252 X-MS-Exchange-Forest-EmailMessageHash: 130014E9 X-MS-Exchange-Forest-Language: en X-MS-Exchange-Organization-Processed-By-Journaling: Journal Agent On Fri, 29 Mar 2024 at 18:58, Donald Hunter <donald.hunter@gmail.com> wrote: > > Jakub Kicinski <kuba@kernel.org> writes: > > > Looking at the code again, are you sure we'll process all the responses > > not just the first one? > > > > Shouldn't this: > > > > + del reqs_by_seq[nl_msg.nl_seq] > > done = True > > > > be something like: > > > > del reqs_by_seq[nl_msg.nl_seq] > > done = len(reqs_by_seq) == 0 > > > > Hmm yes, that's a good catch. I need to check the DONE semantics for > these nftables batch operations. Well that's a problem: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi batch-begin '{"res-id": 10}' \ --multi newtable '{"name": "test", "nfgen-family": 1}' \ --multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \ --multi batch-end '{"res-id": 10}' Adding: 20778 Adding: 20779 Adding: 20780 Adding: 20781 Done: 20779 Done: 20780 There's no response for 'batch-begin' or 'batch-end'. We may need a per op spec property to tell us if a request will be acknowledged. > > Would be good to add an example of multi executing some get operations. > > I think this was a blind spot on my part because nftables doesn't > support batch for get operations: > > https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 > > I'll need to try using multi for gets without any batch messages and see how > everything behaves. Okay, so it can be made to work. Will incorporate into the next revision: ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/nftables.yaml \ --multi gettable '{"name": "test", "nfgen-family": 1}' \ --multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' [{'flags': set(), 'handle': 10, 'name': 'test', 'nfgen-family': 1, 'res-id': 200, 'use': 1, 'version': 0}, {'handle': 1, 'name': 'chain', 'nfgen-family': 1, 'res-id': 200, 'table': 'test', 'use': 0, 'version': 0}] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl @ 2024-03-29 21:46 ` Jakub Kicinski 0 siblings, 0 replies; 14+ messages in thread From: Jakub Kicinski @ 2024-03-29 21:46 UTC (permalink / raw) To: Donald Hunter, Pablo Neira Ayuso Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter, fw On Fri, 29 Mar 2024 21:01:09 +0000 Donald Hunter wrote: > There's no response for 'batch-begin' or 'batch-end'. We may need a > per op spec property to tell us if a request will be acknowledged. :( Pablo, could we possibly start processing the ACK flags on those messages? Maybe the existing user space doesn't set ACK so nobody would notice? I don't think the messages are otherwise marked as special from the "netlink layer" perspective. > > I think this was a blind spot on my part because nftables doesn't > > support batch for get operations: > > > > https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 > > > > I'll need to try using multi for gets without any batch messages and see how > > everything behaves. > > Okay, so it can be made to work. Will incorporate into the next revision: Great! ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl @ 2024-03-29 21:46 ` Jakub Kicinski 0 siblings, 0 replies; 14+ messages in thread From: Jakub Kicinski @ 2024-03-29 21:46 UTC (permalink / raw) To: Donald Hunter, Pablo Neira Ayuso Cc: netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter, fw On Fri, 29 Mar 2024 21:01:09 +0000 Donald Hunter wrote: > There's no response for 'batch-begin' or 'batch-end'. We may need a > per op spec property to tell us if a request will be acknowledged. :( Pablo, could we possibly start processing the ACK flags on those messages? Maybe the existing user space doesn't set ACK so nobody would notice? I don't think the messages are otherwise marked as special from the "netlink layer" perspective. > > I think this was a blind spot on my part because nftables doesn't > > support batch for get operations: > > > > https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 > > > > I'll need to try using multi for gets without any batch messages and see how > > everything behaves. > > Okay, so it can be made to work. Will incorporate into the next revision: Great! X-sender: <netdev+bounces-83467-peter.schumann=secunet.com@vger.kernel.org> X-Receiver: <peter.schumann@secunet.com> ORCPT=rfc822;peter.schumann@secunet.com X-CreatedBy: MSExchange15 X-HeloDomain: mbx-dresden-01.secunet.de X-ExtendedProps: BQBjAAoA+UemlidQ3AgFADcAAgAADwA8AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5NYWlsUmVjaXBpZW50Lk9yZ2FuaXphdGlvblNjb3BlEQAAAAAAAAAAAAAAAAAAAAAADwA/AAAATWljcm9zb2Z0LkV4Y2hhbmdlLlRyYW5zcG9ydC5EaXJlY3RvcnlEYXRhLk1haWxEZWxpdmVyeVByaW9yaXR5DwADAAAATG93 X-Source: SMTP:Default MBX-ESSEN-02 X-SourceIPAddress: 10.53.40.199 X-EndOfInjectedXHeaders: 8081 Received: from mbx-dresden-01.secunet.de (10.53.40.199) by mbx-essen-02.secunet.de (10.53.40.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.37; Fri, 29 Mar 2024 22:46:52 +0100 Received: from b.mx.secunet.com (62.96.220.37) by cas-essen-02.secunet.de (10.53.40.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Fri, 29 Mar 2024 22:46:52 +0100 Received: from localhost (localhost [127.0.0.1]) by b.mx.secunet.com (Postfix) with ESMTP id 2117320396 for <peter.schumann@secunet.com>; Fri, 29 Mar 2024 22:46:52 +0100 (CET) X-Virus-Scanned: by secunet X-Spam-Flag: NO X-Spam-Score: -3.099 X-Spam-Level: X-Spam-Status: No, score=-3.099 tagged_above=-999 required=2.1 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.099, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, MAILING_LIST_MULTI=-1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: a.mx.secunet.com (amavisd-new); dkim=pass (2048-bit key) header.d=kernel.org Received: from b.mx.secunet.com ([127.0.0.1]) by localhost (a.mx.secunet.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lWTysdRw2TI3 for <peter.schumann@secunet.com>; Fri, 29 Mar 2024 22:46:51 +0100 (CET) Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=147.75.48.161; helo=sy.mirrors.kernel.org; envelope-from=netdev+bounces-83467-peter.schumann=secunet.com@vger.kernel.org; receiver=peter.schumann@secunet.com DKIM-Filter: OpenDKIM Filter v2.11.0 b.mx.secunet.com 6BFD82032C Authentication-Results: b.mx.secunet.com; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bI+U+9iL" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [147.75.48.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by b.mx.secunet.com (Postfix) with ESMTPS id 6BFD82032C for <peter.schumann@secunet.com>; Fri, 29 Mar 2024 22:46:50 +0100 (CET) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 20389B21C05 for <peter.schumann@secunet.com>; Fri, 29 Mar 2024 21:46:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0A40D13BC09; Fri, 29 Mar 2024 21:46:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bI+U+9iL" X-Original-To: netdev@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA80785926 for <netdev@vger.kernel.org>; Fri, 29 Mar 2024 21:46:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711748801; cv=none; b=hLKYXnQEay5o+2wB5f0ryqS+rZ4ZW/pWleHMXwjhbqTEO9laLXYaP0C6ZTYGVrNw+Tt5OVQ/RQaNUat82Rt+EhBBWWqzcvErd7KDsFj0u4E1bDi1tepghJvI1eyyM+7gjw9B2Jl5hWUWRNj3KHwymj9hNAeWQKdXyYcIelmzd6g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711748801; c=relaxed/simple; bh=WYrLnd5VJ+w8tGoOoxXgy3WX373NxHkTc+nBr5+yjG4=; h=Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Da0PmiKj4OjIEt8CupTNYdDoBOgmm/fmvFKxeKWlbCItOgbA0gtxxxkd4hPTd3TIzWfN6pUedIOpwdyyZE0XgLyjFKerPpbHIyQsmI5+UNZ2pzKJU7SOGYeO/z+jep+WWfr4R1gtnmwquWEj0SLan7cV63m1nEY70J0ZSUYgjHc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bI+U+9iL; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BAFCC433C7; Fri, 29 Mar 2024 21:46:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711748800; bh=WYrLnd5VJ+w8tGoOoxXgy3WX373NxHkTc+nBr5+yjG4=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=bI+U+9iL3hiAq1i9X3EfgmszunTmib3XIbrxUXRflu3eiFNobwQTe80Nf79MnqiG8 t3e98Rgu7Jq4Mk3ZE4Fdq4v43fw9On7zw5k3qEWPs63RVIGvHxkkIqzruRT0wC46jl 9q2y63dRk0GsdQxnFxFqQ44B8lisQgn22oS7gpVRPtSnINNdrbZMtXLzpz7n7rYzER WwmL/vzkFrdHEj9I9WpPRcCiHv3J4pPbcwn/oyA4gO058KPdy3NpRp992LZOHjmd3M ax2/pQcD7XZi71fGhCFm+oewMT3YFpCaojmdBqcEOVTku7zALR43tPlo8iANKnkNbS 495lKCzcWHj0w== Date: Fri, 29 Mar 2024 14:46:39 -0700 From: Jakub Kicinski <kuba@kernel.org> To: Donald Hunter <donald.hunter@gmail.com>, Pablo Neira Ayuso <pablo@netfilter.org> Cc: netdev@vger.kernel.org, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Paolo Abeni <pabeni@redhat.com>, Jiri Pirko <jiri@resnulli.us>, Jacob Keller <jacob.e.keller@intel.com>, Stanislav Fomichev <sdf@google.com>, donald.hunter@redhat.com, fw@strlen.de Subject: Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl Message-ID: <20240329144639.0b42dc19@kernel.org> In-Reply-To: <CAD4GDZw0RW3B2n5vC-q-XLpQ_bCg0iP13qvOa=cjK37CPLJsKg@mail.gmail.com> References: <20240327181700.77940-1-donald.hunter@gmail.com> <20240327181700.77940-3-donald.hunter@gmail.com> <20240328175729.15208f4a@kernel.org> <m234s9jh0k.fsf@gmail.com> <20240329084346.7a744d1e@kernel.org> <m2plvcj27b.fsf@gmail.com> <CAD4GDZw0RW3B2n5vC-q-XLpQ_bCg0iP13qvOa=cjK37CPLJsKg@mail.gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: <netdev.vger.kernel.org> List-Subscribe: <mailto:netdev+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:netdev+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Return-Path: netdev+bounces-83467-peter.schumann=secunet.com@vger.kernel.org X-MS-Exchange-Organization-OriginalArrivalTime: 29 Mar 2024 21:46:52.1893 (UTC) X-MS-Exchange-Organization-Network-Message-Id: e724de8b-c62a-42f5-a116-08dc5039bea9 X-MS-Exchange-Organization-OriginalClientIPAddress: 62.96.220.37 X-MS-Exchange-Organization-OriginalServerIPAddress: 10.53.40.202 X-MS-Exchange-Organization-Cross-Premises-Headers-Processed: cas-essen-02.secunet.de X-MS-Exchange-Organization-OrderedPrecisionLatencyInProgress: LSRV=cas-essen-02.secunet.de:TOTAL-FE=0.008|SMR=0.008(SMRPI=0.006(SMRPI-FrontendProxyAgent=0.006));2024-03-29T21:46:52.198Z X-MS-Exchange-Forest-ArrivalHubServer: mbx-essen-02.secunet.de X-MS-Exchange-Organization-AuthSource: cas-essen-02.secunet.de X-MS-Exchange-Organization-AuthAs: Anonymous X-MS-Exchange-Organization-OriginalSize: 7533 X-MS-Exchange-Organization-Transport-Properties: DeliveryPriority=Low X-MS-Exchange-Organization-Prioritization: 2:ShadowRedundancy X-MS-Exchange-Organization-IncludeInSla: False:ShadowRedundancy On Fri, 29 Mar 2024 21:01:09 +0000 Donald Hunter wrote: > There's no response for 'batch-begin' or 'batch-end'. We may need a > per op spec property to tell us if a request will be acknowledged. :( Pablo, could we possibly start processing the ACK flags on those messages? Maybe the existing user space doesn't set ACK so nobody would notice? I don't think the messages are otherwise marked as special from the "netlink layer" perspective. > > I think this was a blind spot on my part because nftables doesn't > > support batch for get operations: > > > > https://elixir.bootlin.com/linux/latest/source/net/netfilter/nf_tables_api.c#L9092 > > > > I'll need to try using multi for gets without any batch messages and see how > > everything behaves. > > Okay, so it can be made to work. Will incorporate into the next revision: Great! ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl 2024-03-29 21:46 ` Jakub Kicinski (?) @ 2024-03-29 22:12 ` Pablo Neira Ayuso -1 siblings, 0 replies; 14+ messages in thread From: Pablo Neira Ayuso @ 2024-03-29 22:12 UTC (permalink / raw) To: Jakub Kicinski Cc: Donald Hunter, netdev, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter, fw, netfilter-devel On Fri, Mar 29, 2024 at 02:46:39PM -0700, Jakub Kicinski wrote: > On Fri, 29 Mar 2024 21:01:09 +0000 Donald Hunter wrote: > > There's no response for 'batch-begin' or 'batch-end'. We may need a > > per op spec property to tell us if a request will be acknowledged. > > :( > > Pablo, could we possibly start processing the ACK flags on those > messages? Maybe the existing user space doesn't set ACK so nobody > would notice? > > I don't think the messages are otherwise marked as special from > the "netlink layer" perspective. It is possible to explore this. I don't have a use-case for NLM_F_ACK and the begin marker message at this stage. Thanks. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages 2024-03-27 18:16 [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Donald Hunter 2024-03-27 18:16 ` [PATCH net-next v1 1/2] doc/netlink/specs: Add draft nftables spec Donald Hunter 2024-03-27 18:17 ` [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl Donald Hunter @ 2024-03-27 22:45 ` Pablo Neira Ayuso 2024-03-28 15:33 ` Donald Hunter 2 siblings, 1 reply; 14+ messages in thread From: Pablo Neira Ayuso @ 2024-03-27 22:45 UTC (permalink / raw) To: Donald Hunter Cc: netdev, Jakub Kicinski, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter Please, Cc netfilter-devel@vger.kernel.org for netfilter related stuff. On Wed, Mar 27, 2024 at 06:16:58PM +0000, Donald Hunter wrote: > This series adds a ynl spec for nftables and extends ynl with a --multi > command line option that makes it possible to send transactional batches > for nftables. > > An example of usage is: > > ./tools/net/ynl/cli.py \ > --spec Documentation/netlink/specs/nftables.yaml \ > --multi batch-begin '{"res-id": 10}' \ > --multi newtable '{"name": "test", "nfgen-family": 1}' \ > --multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \ > --multi batch-end '{"res-id": 10}' > > Donald Hunter (2): > doc/netlink/specs: Add draft nftables spec > tools/net/ynl: Add multi message support to ynl > > Documentation/netlink/specs/nftables.yaml | 1264 +++++++++++++++++++++ > tools/net/ynl/cli.py | 22 +- > tools/net/ynl/lib/ynl.py | 47 +- > 3 files changed, 1315 insertions(+), 18 deletions(-) > create mode 100644 Documentation/netlink/specs/nftables.yaml > > -- > 2.44.0 > > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages 2024-03-27 22:45 ` [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Pablo Neira Ayuso @ 2024-03-28 15:33 ` Donald Hunter 0 siblings, 0 replies; 14+ messages in thread From: Donald Hunter @ 2024-03-28 15:33 UTC (permalink / raw) To: Pablo Neira Ayuso Cc: netdev, Jakub Kicinski, David S. Miller, Eric Dumazet, Paolo Abeni, Jiri Pirko, Jacob Keller, Stanislav Fomichev, donald.hunter Pablo Neira Ayuso <pablo@netfilter.org> writes: > Please, Cc netfilter-devel@vger.kernel.org for netfilter related stuff. Okay, should I resend then? ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2024-03-31 16:43 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-03-27 18:16 [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Donald Hunter 2024-03-27 18:16 ` [PATCH net-next v1 1/2] doc/netlink/specs: Add draft nftables spec Donald Hunter 2024-03-27 18:17 ` [PATCH net-next v1 2/2] tools/net/ynl: Add multi message support to ynl Donald Hunter 2024-03-29 0:57 ` Jakub Kicinski 2024-03-29 13:37 ` Donald Hunter 2024-03-29 15:43 ` Jakub Kicinski 2024-03-29 18:57 ` Donald Hunter 2024-03-29 21:01 ` Donald Hunter 2024-03-29 21:01 ` Donald Hunter 2024-03-29 21:46 ` Jakub Kicinski 2024-03-29 21:46 ` Jakub Kicinski 2024-03-29 22:12 ` Pablo Neira Ayuso 2024-03-27 22:45 ` [PATCH net-next v1 0/2] netlink: Add nftables spec w/ multi messages Pablo Neira Ayuso 2024-03-28 15:33 ` Donald Hunter
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.