From: David Wei <dw@davidwei.uk>
To: Jiri Pirko <jiri@resnulli.us>
Cc: Jakub Kicinski <kuba@kernel.org>,
Sabrina Dubroca <sd@queasysnail.net>,
netdev@vger.kernel.org, "David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Paolo Abeni <pabeni@redhat.com>
Subject: Re: [PATCH net-next v5 3/5] netdevsim: forward skbs from one connected port to another
Date: Wed, 3 Jan 2024 14:36:36 -0800 [thread overview]
Message-ID: <bf4760df-a78f-431d-8c33-b7a2f7fb393d@davidwei.uk> (raw)
In-Reply-To: <ZZPv42K9VRTao735@nanopsycho>
On 2024-01-02 03:13, Jiri Pirko wrote:
> Thu, Dec 28, 2023 at 02:46:31AM CET, dw@davidwei.uk wrote:
>> Forward skbs sent from one netdevsim port to its connected netdevsim
>> port using dev_forward_skb, in a spirit similar to veth.
>>
>> Add a tx_dropped variable to struct netdevsim, tracking the number of
>> skbs that could not be forwarded using dev_forward_skb().
>>
>> The xmit() function accessing the peer ptr is protected by an RCU read
>> critical section. The rcu_read_lock() is functionally redundant as since
>> v5.0 all softirqs are implicitly RCU read critical sections; but it is
>> useful for human readers.
>>
>> If another CPU is concurrently in nsim_destroy(), then it will first set
>> the peer ptr to NULL. This does not affect any existing readers that
>> dereferenced a non-NULL peer. Then, in unregister_netdevice(), there is
>> a synchronize_rcu() before the netdev is actually unregistered and
>> freed. This ensures that any readers i.e. xmit() that got a non-NULL
>> peer will complete before the netdev is freed.
>>
>> Any readers after the RCU_INIT_POINTER() but before synchronize_rcu()
>> will dereference NULL, making it safe.
>>
>> The codepath to nsim_destroy() and nsim_create() takes both the newly
>> added nsim_dev_list_lock and rtnl_lock. This makes it safe with
>
> I don't see the rtnl_lock take in those functions.
>
>
> Otherwise, this patch looks fine to me.
For nsim_create(), rtnl_lock is taken in nsim_init_netdevsim(). For
nsim_destroy(), rtnl_lock is taken directly in the function.
What I mean here is, in the netdevsim device modification paths locks
are taken in this order:
devl_lock -> rtnl_lock
nsim_dev_list_lock is taken outside (not nested) of these.
In nsim_dev_peer_write() where two ports are linked, locks are taken in
this order:
nsim_dev_list_lock -> devl_lock -> rtnl_lock
This will not cause deadlocks and ensures that two ports being linked
are both valid.
>
>
>> concurrent calls to linking two netdevsims together.
>>
>> Signed-off-by: David Wei <dw@davidwei.uk>
>> ---
>> drivers/net/netdevsim/netdev.c | 21 ++++++++++++++++++---
>> drivers/net/netdevsim/netdevsim.h | 1 +
>> 2 files changed, 19 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c
>> index 434322f6a565..0009d0f1243f 100644
>> --- a/drivers/net/netdevsim/netdev.c
+++ b/drivers/net/netdevsim/netdev.c
>> @@ -29,19 +29,34 @@
>> static netdev_tx_t nsim_start_xmit(struct sk_buff *skb, struct net_device *dev)
>> {
>> struct netdevsim *ns = netdev_priv(dev);
>> + struct netdevsim *peer_ns;
>> + int ret = NETDEV_TX_OK;
>>
>> if (!nsim_ipsec_tx(ns, skb))
>> goto out;
>>
>> + rcu_read_lock();
>> + peer_ns = rcu_dereference(ns->peer);
>> + if (!peer_ns)
>> + goto out_stats;
>> +
>> + skb_tx_timestamp(skb);
>> + if (unlikely(dev_forward_skb(peer_ns->netdev, skb) == NET_RX_DROP))
>> + ret = NET_XMIT_DROP;
>> +
>> +out_stats:
>> + rcu_read_unlock();
>> u64_stats_update_begin(&ns->syncp);
>> ns->tx_packets++;
>> ns->tx_bytes += skb->len;
>> + if (ret == NET_XMIT_DROP)
>> + ns->tx_dropped++;
>> u64_stats_update_end(&ns->syncp);
>> + return ret;
>>
>> out:
>> dev_kfree_skb(skb);
>> -
>> - return NETDEV_TX_OK;
>> + return ret;
>> }
>>
>> static void nsim_set_rx_mode(struct net_device *dev)
>> @@ -70,6 +85,7 @@ nsim_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
>> start = u64_stats_fetch_begin(&ns->syncp);
>> stats->tx_bytes = ns->tx_bytes;
>> stats->tx_packets = ns->tx_packets;
>> + stats->tx_dropped = ns->tx_dropped;
>> } while (u64_stats_fetch_retry(&ns->syncp, start));
>> }
>>
>> @@ -302,7 +318,6 @@ static void nsim_setup(struct net_device *dev)
>> eth_hw_addr_random(dev);
>>
>> dev->tx_queue_len = 0;
>> - dev->flags |= IFF_NOARP;
>> dev->flags &= ~IFF_MULTICAST;
>> dev->priv_flags |= IFF_LIVE_ADDR_CHANGE |
>> IFF_NO_QUEUE;
>> diff --git a/drivers/net/netdevsim/netdevsim.h b/drivers/net/netdevsim/netdevsim.h
>> index 24fc3fbda791..083b1ee7a1a2 100644
>> --- a/drivers/net/netdevsim/netdevsim.h
>> +++ b/drivers/net/netdevsim/netdevsim.h
>> @@ -98,6 +98,7 @@ struct netdevsim {
>>
>> u64 tx_packets;
>> u64 tx_bytes;
>> + u64 tx_dropped;
>> struct u64_stats_sync syncp;
>>
>> struct nsim_bus_dev *nsim_bus_dev;
>> --
>> 2.39.3
>>
next prev parent reply other threads:[~2024-01-03 22:36 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-28 1:46 [PATCH net-next v5 0/5] netdevsim: link and forward skbs between ports David Wei
2023-12-28 1:46 ` [PATCH net-next v5 1/5] netdevsim: maintain a list of probed netdevsims David Wei
2024-01-02 11:04 ` Jiri Pirko
2024-01-03 21:48 ` David Wei
2023-12-28 1:46 ` [PATCH net-next v5 2/5] netdevsim: allow two netdevsim ports to be connected David Wei
2024-01-02 11:11 ` Jiri Pirko
2024-01-03 21:56 ` David Wei
2024-01-04 9:30 ` Jiri Pirko
2024-01-04 1:39 ` Jakub Kicinski
2024-01-09 16:57 ` David Wei
2024-01-10 1:53 ` Jakub Kicinski
2023-12-28 1:46 ` [PATCH net-next v5 3/5] netdevsim: forward skbs from one connected port to another David Wei
2024-01-02 11:13 ` Jiri Pirko
2024-01-03 22:36 ` David Wei [this message]
2024-01-04 9:31 ` Jiri Pirko
2024-01-09 16:58 ` David Wei
2024-01-02 11:20 ` Eric Dumazet
2024-01-03 21:57 ` David Wei
2023-12-28 1:46 ` [PATCH net-next v5 4/5] netdevsim: add selftest for forwarding skb between connected ports David Wei
2023-12-28 1:46 ` [PATCH net-next v5 5/5] netdevsim: add Makefile for selftests David Wei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bf4760df-a78f-431d-8c33-b7a2f7fb393d@davidwei.uk \
--to=dw@davidwei.uk \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=jiri@resnulli.us \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=sd@queasysnail.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).