From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ferruh Yigit Subject: Re: [PATCH 2/2] examples/kni: stop lcores while doing kni ops Date: Fri, 20 Oct 2017 17:42:13 -0700 Message-ID: <06faf3fd-ff00-bcab-e462-0ddd2a35728d@intel.com> References: <1508154348-10988-1-git-send-email-tdu@semihalf.com> <1508154348-10988-3-git-send-email-tdu@semihalf.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit To: Tomasz Duszynski , dev@dpdk.org Return-path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 72B8A1B294 for ; Sat, 21 Oct 2017 02:42:15 +0200 (CEST) In-Reply-To: <1508154348-10988-3-git-send-email-tdu@semihalf.com> Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 10/16/2017 4:45 AM, Tomasz Duszynski wrote: > Since the transmit and receive functions should not be invoked when > the device is stopped, stop lcores during kni ops and restart them > after device is started once again. Hi Tomasz, Are you observing any error or unexpected behavior because of rx/tx functions? I am not sure about the patch, please check below logs, and trying to understand scope of the patch, can you please give more details what happens if this patch is missing? > > Signed-off-by: Tomasz Duszynski > --- > examples/kni/main.c | 28 ++++++++++++++++++++++++++++ > 1 file changed, 28 insertions(+) > > diff --git a/examples/kni/main.c b/examples/kni/main.c > index cb48fb5..5c50448 100644 > --- a/examples/kni/main.c > +++ b/examples/kni/main.c > @@ -166,6 +166,23 @@ static int kni_change_mtu(uint16_t port_id, unsigned int new_mtu); > static int kni_config_network_interface(uint16_t port_id, uint8_t if_up); > > static rte_atomic32_t kni_stop = RTE_ATOMIC32_INIT(0); > +static rte_atomic32_t kni_restart = RTE_ATOMIC32_INIT(0); > + > +static void > +kni_stop_lcores(void) > +{ > + unsigned int i; > + > + rte_atomic32_inc(&kni_restart); > + rte_atomic32_inc(&kni_stop); > + > + RTE_LCORE_FOREACH(i) { > + if (i == rte_lcore_id()) > + continue; This function called by port Rx core [1], and since the thread can't wait itself to finish, specially if nb_kni > 1, the Rx core still can do some work even after exit from this function. > + > + rte_eal_wait_lcore(i); The API documentation says: "To be executed on the MASTER lcore only." Not sure what happens when called from slave core, as we did here. > + } > +} > > /* Print out statistics on packets handled */ > static void > @@ -712,6 +729,7 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu) > > RTE_LOG(INFO, APP, "Change MTU of port %d to %u\n", port_id, new_mtu); > > + kni_stop_lcores(); > /* Stop specific port */ > rte_eth_dev_stop(port_id); > > @@ -755,6 +773,8 @@ kni_config_network_interface(uint16_t port_id, uint8_t if_up) > RTE_LOG(INFO, APP, "Configure network interface of %d %s\n", > port_id, if_up ? "up" : "down"); > > + kni_stop_lcores(); > + > if (if_up != 0) { /* Configure network interface up */ > rte_eth_dev_stop(port_id); > ret = rte_eth_dev_start(port_id); > @@ -911,6 +931,7 @@ main(int argc, char** argv) > } > check_all_ports_link_status(nb_sys_ports, ports_mask); > > +restart: > /* Launch per-lcore function on every lcore */ > rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER); > RTE_LCORE_FOREACH_SLAVE(i) { > @@ -918,6 +939,13 @@ main(int argc, char** argv) > return -1; > } > > + if (rte_atomic32_read(&kni_restart)) { > + rte_atomic32_dec(&kni_stop); > + rte_atomic32_dec(&kni_restart); kni_stop_lcores() called per port, so it is possible that kni_stop and kni_restart increased parallel, many times. But this decrement is per application, so they will be decremented sequentially, casing app stop - start unnecessarily. > + > + goto restart; This will cause assigning tasks to cores again, and will produce all related logs again, if you enable debug logs you will see it [2]. I believe confusing to have those logs every time mtu updated etc... > + } > + > /* Release resources */ > for (port = 0; port < nb_sys_ports; port++) { > if (!(ports_mask & (1 << port))) > -- > 2.7.4 > [1] main_loop kni_ingress rte_kni_handle_request kni_change_mtu || kni_config_network_interface kni_stop_lcores [2] APP: Change MTU of port 0 to 1402 PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=0). PMD: ixgbe_dev_link_status_print(): Port 0: Link Down PMD: ixgbe_dev_link_status_print(): PCI Address: 0000:08:00.1 APP: Lcore 1 is reading from port 0 APP: Lcore 2 is writing to port 0 APP: Lcore 3 is reading from port 1 APP: Lcore 4 is writing to port 1 APP: Lcore 5 has nothing to do APP: Lcore 6 has nothing to do APP: Lcore 7 has nothing to do APP: Lcore 8 has nothing to do APP: Lcore 9 has nothing to do APP: Lcore 10 has nothing to do APP: Lcore 11 has nothing to do APP: Lcore 12 has nothing to do APP: Lcore 13 has nothing to do APP: Lcore 14 has nothing to do APP: Lcore 15 has nothing to do APP: Lcore 16 has nothing to do APP: Lcore 17 has nothing to do APP: Lcore 18 has nothing to do APP: Lcore 19 has nothing to do APP: Lcore 20 has nothing to do APP: Lcore 21 has nothing to do APP: Lcore 22 has nothing to do APP: Lcore 23 has nothing to do APP: Lcore 24 has nothing to do APP: Lcore 25 has nothing to do APP: Lcore 26 has nothing to do APP: Lcore 27 has nothing to do APP: Lcore 28 has nothing to do APP: Lcore 29 has nothing to do APP: Lcore 30 has nothing to do APP: Lcore 31 has nothing to do APP: Lcore 0 has nothing to do