From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vincent Li Subject: Re: virtio pmd failed in pci probing Date: Fri, 13 May 2016 10:18:06 -0700 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 To: dev@dpdk.org Return-path: Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) by dpdk.org (Postfix) with ESMTP id B68DB9A91 for ; Fri, 13 May 2016 19:18:06 +0200 (CEST) Received: by mail-lf0-f53.google.com with SMTP id j8so91245456lfd.2 for ; Fri, 13 May 2016 10:18:06 -0700 (PDT) In-Reply-To: List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" sorry for the noise, it turned out I need to add the virtio pmd in the mTCP app Makefile as below LIBS += -m64 -g -O3 -pthread -lrt -march=native -Wl,-export-dynamic ${MTCP_FLD}/lib/libmtcp.a -L../../dpdk/lib -Wl,-lnuma -Wl,-lmtcp -Wl,-lpthread -Wl,-lrt -Wl,-ldl -Wl,--whole-archive -Wl,-lrte_distributor -Wl,-lrte_kni -Wl,-lrte_pipeline -Wl,-lrte_table -Wl,-lrte_port -Wl,-lrte_timer -Wl,-lrte_hash -Wl,-lrte_lpm -Wl,-lrte_power -Wl,-lrte_acl -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm -Wl,-lrt -Wl,--start-group -Wl,-lrte_kvargs -Wl,-lrte_mbuf -Wl,-lrte_ip_frag -Wl,-lethdev -Wl,-lrte_mempool -Wl,-lrte_ring -Wl,-lrte_eal -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond -Wl,-lrte_pmd_vmxnet3_uio -Wl,-lrte_pmd_i40e -Wl,-lrte_pmd_ixgbe -Wl,-lrte_pmd_e1000 -Wl,-lrte_pmd_virtio -Wl,-lrte_pmd_ring -Wl,-lrt -Wl,-lm -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive On Thu, May 12, 2016 at 1:10 PM, Vincent Li wrote: > I add a debug log line in dpdk-16.04/lib/librte_ether/rte_ethdev.c > rte_eth_driver_register > > void > rte_eth_driver_register(struct eth_driver *eth_drv) > { > eth_drv->pci_drv.devinit = rte_eth_dev_init; > eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; > rte_eal_pci_register(ð_drv->pci_drv); > RTE_LOG(DEBUG, EAL, " register pmd driver %s\n", > eth_drv->pci_drv.name); > } > > then run the mTCP app, rte_virtio_pmd is missing: > > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > unreliable clock cycles ! > EAL: Master lcore 0 is ready (tid=6ec31900;cpuset=[0]) > EAL: register pmd driver rte_vmxnet3_pmd > EAL: register pmd driver rte_i40e_pmd > EAL: register pmd driver rte_i40evf_pmd > EAL: register pmd driver rte_ixgbe_pmd > EAL: register pmd driver rte_ixgbevf_pmd > EAL: register pmd driver rte_igb_pmd > EAL: register pmd driver rte_igbvf_pmd > EAL: register pmd driver rte_em_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd > pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted > EAL: Error - exiting with code: 1 > Cause: No Ethernet port! > > any clue? > > > On Thu, May 12, 2016 at 9:34 AM, Vincent Li wrote: >> Hi >> >> I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04 >> on KVM guest and it appears the virtio pmd fail to load, detail info >> below: >> >> >> # ./tools/dpdk_nic_bind.py --status >> >> Network devices using DPDK-compatible driver >> ============================================ >> 0000:00:07.0 'Virtio network device' drv=igb_uio unused= >> >> Network devices using kernel driver >> =================================== >> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio >> 0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio >> >> in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it >> easy to test for me): >> >> >> >> sprintf(whitelist, "%s", "0000:00:07.0"); >> /* initialize the rte env first, what a waste of >> implementation effort! */ >> char *argv[] = {"", >> "-c", >> cpumaskbuf, >> "-w", >> whitelist, >> "-n", >> mem_channels, >> "--proc-type=auto", >> "" >> }; >> const int argc = 8; >> >> /* >> * re-set getopt extern variable optind. >> * this issue was a bitch to debug >> * rte_eal_init() internally uses getopt() syscall >> * mtcp applications that also use an `external' getopt >> * will cause a violent crash if optind is not reset to zero >> * prior to calling the func below... >> * see man getopt(3) for more details >> */ >> optind = 0; >> >> /* initialize the dpdk eal env */ >> ret = rte_eal_init(argc, argv); >> if (ret < 0) >> rte_exit(EXIT_FAILURE, "Invalid EAL args!\n"); >> /* give me the count of 'detected' ethernet ports */ >> num_devices = rte_eth_dev_count(); >> if (num_devices == 0) { >> rte_exit(EXIT_FAILURE, "No Ethernet port!\n"); >> } >> >> in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c >> rte_eal_pci_probe_one_driver, I added debug line below: >> >> /* >> * If vendor/device ID match, call the devinit() function of the >> * driver. >> */ >> static int >> rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct >> rte_pci_device *dev) >> { >> int ret; >> const struct rte_pci_id *id_table; >> >> RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id >> dr->name %x:%x %s\n", dev->id.vendor_id, >> dev->id.device_id, dr->name); >> >> >> when I run mTCP app as below, it says "No Ethernet port!", please note >> that the debug line did not show rte_virtio_pmd, why? is it because >> virtio pmd lack of vendor_id/device_id implementation? >> >> >> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using >> unreliable clock cycles ! >> EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0]) >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd >> pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted >> EAL: Error - exiting with code: 1 >> Cause: No Ethernet port! >> >> >> dpdk testpmd app detects the ethernet port fine, here is the testpmd >> output (Note here the debug line shows rte_virtio_pmd): >> >> ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i >> >> EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0]) >> EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3]) >> EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2]) >> EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1]) >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 >> rte_virtio_pmd >> EAL: PCI device 0000:00:07.0 on NUMA socket -1 >> EAL: probe driver: 1af4:1000 rte_virtio_pmd >> EAL: PCI memory mapped at 0x7fabe8a00000 >> >> >> I am able to workaround this by compiling dpdk as shared library for >> mTCP and use '-d' to load librte_pmd_virtio.so explicitly. >> >> it looks to me there isn't much difference between how testpmd and >> mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and >> rte_eth_dev_count() >> >> is there any potential bug in the implementation of virtio pmd in pci probing ? >> >> Thanks!