* Any ideas how to stop DPDK from banning me from the box.
@ 2013-07-18 15:42 Dmitry Vyal
[not found] ` <51E80CF9.6030005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
0 siblings, 1 reply; 7+ messages in thread
From: Dmitry Vyal @ 2013-07-18 15:42 UTC (permalink / raw)
To: dev-VfR2kkLFssw
Greetings!
I've been playing with dpdk on my desktop for some time and decided to
finally test in on a server. It has a plenty of nics:
dev@box:~$ lspci |grep Ether
01:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
03:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
0c:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
0c:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
0e:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
0e:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
10:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
10:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
12:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
Connection
82:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
82:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
88:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
88:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
8a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
8a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
I connect to the server using ssh through one of gigabit ports. The
problem is that no matter which cards I blacklist with -b option then
starting my dpdk application it takes away all the interfaces from the
kernel and later returns blacklisted ones. This disrupts my ssh session
and I can no longer connect to the server.
Any ideas how to overcome this?
Best regards,
Dmitry Vyal.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any ideas how to stop DPDK from banning me from the box.
[not found] ` <51E80CF9.6030005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
@ 2013-07-18 15:53 ` Patrick Mahan
2013-07-18 15:59 ` Marco Chiappero
2013-07-19 14:30 ` Antti Kantee
2 siblings, 0 replies; 7+ messages in thread
From: Patrick Mahan @ 2013-07-18 15:53 UTC (permalink / raw)
To: Dmitry Vyal; +Cc: dev-VfR2kkLFssw
Dmitry,
Please post the exact blacklist option. I use the '-b' option all the time with my test box (Dell T620).
Thanks,
Patrick
Sent from my iPad
On Jul 18, 2013, at 8:42 AM, Dmitry Vyal <dmitryvyal-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Greetings!
>
> I've been playing with dpdk on my desktop for some time and decided to finally test in on a server. It has a plenty of nics:
>
> dev@box:~$ lspci |grep Ether
> 01:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 03:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 03:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 0c:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> 0c:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> 0e:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> 0e:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> 10:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> 10:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> 12:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
> 82:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 82:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 88:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 88:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 8a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 8a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
>
> I connect to the server using ssh through one of gigabit ports. The problem is that no matter which cards I blacklist with -b option then starting my dpdk application it takes away all the interfaces from the kernel and later returns blacklisted ones. This disrupts my ssh session and I can no longer connect to the server.
>
> Any ideas how to overcome this?
>
> Best regards,
> Dmitry Vyal.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any ideas how to stop DPDK from banning me from the box.
[not found] ` <51E80CF9.6030005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-07-18 15:53 ` Patrick Mahan
@ 2013-07-18 15:59 ` Marco Chiappero
[not found] ` <51E810C4.1090207-YK5v5TwyeXionA0d6jMUrA@public.gmane.org>
2013-07-19 14:30 ` Antti Kantee
2 siblings, 1 reply; 7+ messages in thread
From: Marco Chiappero @ 2013-07-18 15:59 UTC (permalink / raw)
To: dev-VfR2kkLFssw
Il 18/07/2013 17:42, Dmitry Vyal ha scritto:
> Greetings!
Hi
> [...]
> I connect to the server using ssh through one of gigabit ports.
I do the very same thing and I have no such problem.
> The
> problem is that no matter which cards I blacklist with -b option then
> starting my dpdk application it takes away all the interfaces from the
> kernel and later returns blacklisted ones. This disrupts my ssh session
> and I can no longer connect to the server.
Maybe you are using wrong IDs, verify that.
Here a working example:
pino:~/dpdk-1.2.3r4 ~> lspci | grep -i ethernet
02:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN
10GigE, PCIe 2.0 5GT/s] (rev b0)
03:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
04:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
pino:~/dpdk-1.2.3r4 ~> sudo build/app/testpmd -c 0xf -n 3 -r 1 -b
0000:05:00.0 -b 0000:05:00.1
EAL: coremask set to f
...
Initializing port 0... done: Link Down
Initializing port 1... done: Link Down
Initializing port 2... done: Link Up - speed 10000 Mbps - full-duplex
Initializing port 3... done: Link Up - speed 10000 Mbps - full-duplex
No commandline core given, start packet forwarding
io packet forwarding - CRC stripping disabled - packets/burst=16
nb forwarding cores=1 - nb forwarding ports=4
RX queues=1 - RX desc=128 - RX free threshold=0
RX threshold registers: pthresh=8 hthresh=8 wthresh=4
TX queues=1 - TX desc=512 - TX free threshold=0
TX threshold registers: pthresh=36 hthresh=0 wthresh=0
TX RS bit threshold=0
Press enter to exit
[ssh is still up]
Regards,
Marco
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any ideas how to stop DPDK from banning me from the box.
[not found] ` <51E810C4.1090207-YK5v5TwyeXionA0d6jMUrA@public.gmane.org>
@ 2013-07-19 7:11 ` Dmitry Vyal
[not found] ` <51E8E6B0.8050602-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
0 siblings, 1 reply; 7+ messages in thread
From: Dmitry Vyal @ 2013-07-19 7:11 UTC (permalink / raw)
To: dev-VfR2kkLFssw
Hi Marco,
thanks for posting the example I carefully composed mine using your as a
basis and, guess, all works I don't have the exact cmdline I issued
yesterday. Looks like bash didn't have a chance to save history But I
guess I put all the -b options after the -- delimiter so they weren't
parsed by rte_eal_init. You saved me a day!
>
> pino:~/dpdk-1.2.3r4 ~> sudo build/app/testpmd -c 0xf -n 3 -r 1 -b
> 0000:05:00.0 -b 0000:05:00.1
> EAL: coremask set to f
> ...
> Initializing port 0... done: Link Down
> Initializing port 1... done: Link Down
> Initializing port 2... done: Link Up - speed 10000 Mbps - full-duplex
> Initializing port 3... done: Link Up - speed 10000 Mbps - full-duplex
> No commandline core given, start packet forwarding
> io packet forwarding - CRC stripping disabled - packets/burst=16
> nb forwarding cores=1 - nb forwarding ports=4
> RX queues=1 - RX desc=128 - RX free threshold=0
> RX threshold registers: pthresh=8 hthresh=8 wthresh=4
> TX queues=1 - TX desc=512 - TX free threshold=0
> TX threshold registers: pthresh=36 hthresh=0 wthresh=0
> TX RS bit threshold=0
> Press enter to exit
>
> [ssh is still up]
>
>
> Regards,
> Marco
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any ideas how to stop DPDK from banning me from the box.
[not found] ` <51E8E6B0.8050602-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
@ 2013-07-19 8:34 ` Marco Chiappero
0 siblings, 0 replies; 7+ messages in thread
From: Marco Chiappero @ 2013-07-19 8:34 UTC (permalink / raw)
To: dev-VfR2kkLFssw
Il 19/07/2013 09:11, Dmitry Vyal ha scritto:
> Hi Marco,
>
> thanks for posting the example I carefully composed mine using your as a
> basis and, guess, all works I don't have the exact cmdline I issued
> yesterday. Looks like bash didn't have a chance to save history But I
> guess I put all the -b options after the -- delimiter so they weren't
> parsed by rte_eal_init. You saved me a day!
Hi Dmitry,
I'm glad I could help!
By the way, you can also avoid using the PMD igb driver entirely if you
don't need it (I don't as I just care about 10Gbit interfaces, but since
you have lots of 82576 NICs I'm not that sure); just change your compile
configuration, either the default one or a custom one (depending on what
you are using):
pino:~/dpdk-1.2.3r4 -> grep IGB config/defconfig_x86_64-marco-linuxapp-gcc
# Compile burst-oriented IGB PMD driver
CONFIG_RTE_LIBRTE_IGB_PMD=n
CONFIG_RTE_LIBRTE_IGB_DEBUG_INIT=n
CONFIG_RTE_LIBRTE_IGB_DEBUG_RX=n
CONFIG_RTE_LIBRTE_IGB_DEBUG_TX=n
CONFIG_RTE_LIBRTE_IGB_DEBUG_TX_FREE=n
CONFIG_RTE_LIBRTE_IGB_DEBUG_DRIVER=n
As you can see here I keep using the kernel space igb driver:
pino:~/dpdk-1.2.3r4 -> lspci -kD | grep -A 2 -i ether
0000:02:00.0 Ethernet controller: Mellanox Technologies MT26448
[ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
Subsystem: Mellanox Technologies Device 0016
Kernel driver in use: mlx4_core
0000:03:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
Kernel driver in use: igb_uio
0000:03:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
Kernel driver in use: igb_uio
0000:04:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
Kernel driver in use: igb_uio
0000:04:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
Kernel driver in use: igb_uio
0000:05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
Subsystem: Super Micro Computer Inc Device 10c9
Kernel driver in use: igb
0000:05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
Subsystem: Super Micro Computer Inc Device 10c9
Kernel driver in use: igb
Regards,
Marco
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any ideas how to stop DPDK from banning me from the box.
[not found] ` <51E80CF9.6030005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-07-18 15:53 ` Patrick Mahan
2013-07-18 15:59 ` Marco Chiappero
@ 2013-07-19 14:30 ` Antti Kantee
[not found] ` <51E94D83.5040603-X3B1VOXEql0@public.gmane.org>
2 siblings, 1 reply; 7+ messages in thread
From: Antti Kantee @ 2013-07-19 14:30 UTC (permalink / raw)
To: dev-VfR2kkLFssw
Semi-related: why does DPDK take a blacklist instead of a list of
devices that it should use (perhaps with a special value "all" in case
someone really wants that behavior)? Historical reasons?
(yes, I've also bricked my ssh connections several times)
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any ideas how to stop DPDK from banning me from the box.
[not found] ` <51E94D83.5040603-X3B1VOXEql0@public.gmane.org>
@ 2013-07-19 16:35 ` Vincent JARDIN
0 siblings, 0 replies; 7+ messages in thread
From: Vincent JARDIN @ 2013-07-19 16:35 UTC (permalink / raw)
To: dev-VfR2kkLFssw
On 19/07/2013 16:30, Antti Kantee wrote:
> Semi-related: why does DPDK take a blacklist instead of a list of
> devices that it should use (perhaps with a special value "all" in case
> someone really wants that behavior)? Historical reasons?
>
> (yes, I've also bricked my ssh connections several times)
Antti, you are right. Should you need to make it better with a device
name, your patch can be included.
About such ssh connection, we did not feel the pain at 6WIND since we
have a stack that manages such issues so it does not break, but your
request is valid.
Looking for any patch that would help,
Vincent
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2013-07-19 16:35 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-18 15:42 Any ideas how to stop DPDK from banning me from the box Dmitry Vyal
[not found] ` <51E80CF9.6030005-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-07-18 15:53 ` Patrick Mahan
2013-07-18 15:59 ` Marco Chiappero
[not found] ` <51E810C4.1090207-YK5v5TwyeXionA0d6jMUrA@public.gmane.org>
2013-07-19 7:11 ` Dmitry Vyal
[not found] ` <51E8E6B0.8050602-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-07-19 8:34 ` Marco Chiappero
2013-07-19 14:30 ` Antti Kantee
[not found] ` <51E94D83.5040603-X3B1VOXEql0@public.gmane.org>
2013-07-19 16:35 ` Vincent JARDIN
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.