Re: Reg: promiscuous mode on VF

[bharath paulraj <bharathpaul@gmail.com>]

Hi Greg,

Thanks for your response. I am using the old kernel
(2.6.32-573.22.1.el6.x86_64).
First I have to update the kernel to version-3.18.4 and will try what you
have specified.
I will keep you posted with the outcomes.

Thanks,
Bharath

On Sat, Apr 9, 2016 at 2:36 AM, Rose, Gregory V <gregory.v.rose@intel.com>
wrote:

> Bharath,
>
>
>
> Please try the following:
>
>
>
> In this example we are directing the traffic to queue 1 of VF ID 5.  First
> you must turn on ntuple.
>
>
>
> $ ethtool –K p5p2 ntuple on
>
>
>
> Then you can set the FD rule:
>
>
>
> $ ethtool –N p5p2 flow-type udp4 dst-port 4790 action 1 user-def 5
>
>
>
> When I do that this is the rule definition result:
>
>
>
> Filter: 2045
>
>         Rule Type: UDP over IPv4
>
>         Src IP addr: 0.0.0.0 mask: 255.255.255.255
>
>         Dest IP addr: 0.0.0.0 mask: 255.255.255.255
>
>         TOS: 0x0 mask: 0xff
>
>         Src port: 0 mask: 0xffff
>
>         Dest port: 4790 mask: 0x0
>
>         VLAN EtherType: 0x0 mask: 0xffff
>
>         VLAN: 0x0 mask: 0xffff
>
>         User-defined: 0x5 mask: 0xffffffffffffff00
>
>         Action: Direct to queue 1
>
>
>
> Be sure to use a version of ethtool 3.0 or greater and a recent kernel – I
> tested on ethtool version 3.15 with a 3.18.4 Linux kernel from kernel.org
> and using the most recent ixgbe driver release.
>
>
>
> Let me know if this works for you.
>
>
>
> Thanks,
>
>
>
> - Greg
>
>
>
> *From:* Rose, Gregory V
> *Sent:* Thursday, April 7, 2016 4:43 PM
> *To:* bharath paulraj <bharathpaul@gmail.com>; Qiu, Michael <
> michael.qiu@intel.com>
> *Cc:* Zhang, Helin <helin.zhang@intel.com>; Lu, Wenzhuo <
> wenzhuo.lu@intel.com>; Rowden, Aaron F <aaron.f.rowden@intel.com>;
> dev@dpdk.org; Jayakumar, Muthurajan <muthurajan.jayakumar@intel.com>
>
> *Subject:* RE: [dpdk-dev] Reg: promiscuous mode on VF
>
>
>
> Bharath,
>
>
>
> I’m sorry for the delay but I am working on a response to this.  There is
> a way to do what you need to do and I will get together some instructions
> and respond by tomorrow.
>
>
>
> Thanks and regards,
>
>
>
> - Greg
>
>
>
> *From:* bharath paulraj [mailto:bharathpaul@gmail.com
> <bharathpaul@gmail.com>]
> *Sent:* Thursday, April 7, 2016 3:40 AM
> *To:* Qiu, Michael <michael.qiu@intel.com>
> *Cc:* Rose, Gregory V <gregory.v.rose@intel.com>; Zhang, Helin <
> helin.zhang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Rowden, Aaron
> F <aaron.f.rowden@intel.com>; dev@dpdk.org; Jayakumar, Muthurajan <
> muthurajan.jayakumar@intel.com>
> *Subject:* Re: [dpdk-dev] Reg: promiscuous mode on VF
>
>
>
> Hi Team,
>
>
>
> May I have some update on my previous mail? I am here stuck in flow
> creation.
>
>
>
> Thanks,
>
> Bharath
>
>
>
> On Thu, Mar 31, 2016 at 4:13 PM, bharath paulraj <bharathpaul@gmail.com>
> wrote:
>
> Hi Michael and All,
>
>
>          I am unable to set the rule to receive the packet on the VF.
> Below is my setup.
>
> 1. Creating one virtual function with one queue, in one of my port, p2p1.
> *    modprobe ixgbe MQ=1 max_vfs=1 RSS=1 allow_unsupported_sfp=1 *
> 2. Below is the interface status after creating one virtual function.
> [root@XXXX sriov]# ifconfig p2p1
> p2p1      Link encap:Ethernet  HWaddr A0:36:9F:86:C2:74
>           inet6 addr: fe80::a236:9fff:fe86:c274/64 Scope:Link
>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500 Metric:1
>           RX packets:2540 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:157456 (153.7 KiB)  TX bytes:258 (258.0 b)
>
> [root@XXXX sriov]# ifconfig p2p1_0
> p2p1_0    Link encap:Ethernet  HWaddr DA:61:95:CD:AF:35
>           inet6 addr: fe80::d861:95ff:fecd:af35/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:12 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:360 (360.0 b)  TX bytes:740 (740.0 b)
> 3. Next I am enable ntuple
> *ethtool -K p2p1  ntuple on *
> 4. Now I am adding below rule
>
> *ethtool -N p2p1   flow-type udp4 dst-port 4789 action 0x100000000 --> VF
> 0, queue 0 ethtool -N p2p1   flow-type udp4 dst-port 4790 action
> 0x000000000 --> PF queue 0 *
> 5. [root@XXX sriov]# ethtool -n p2p1
> 1 RX rings available
> Total 2 rules
>
> Filter: 2044
>         Rule Type: UDP over IPv4
>         Src IP addr: 0.0.0.0 mask: 255.255.255.255
>         Dest IP addr: 0.0.0.0 mask: 255.255.255.255
>         TOS: 0x0 mask: 0xff
>         Src port: 0 mask: 0xffff
>         Dest port: 4790 mask: 0x0
>         VLAN EtherType: 0x0 mask: 0xffff
>         VLAN: 0x0 mask: 0xffff
>         User-defined: 0x0 mask: 0xffffffffffffffff
>         Action: Direct to queue 0
>
> Filter: 2045
>         Rule Type: UDP over IPv4
>         Src IP addr: 0.0.0.0 mask: 255.255.255.255
>         Dest IP addr: 0.0.0.0 mask: 255.255.255.255
>         TOS: 0x0 mask: 0xff
>         Src port: 0 mask: 0xffff
>         Dest port: 4789 mask: 0x0
>         VLAN EtherType: 0x0 mask: 0xffff
>         VLAN: 0x0 mask: 0xffff
>         User-defined: 0x0 mask: 0xffffffffffffffff
>         Action: Direct to queue 0
> *    >> Won't it show the VF queue numbers here?*
>
> 6. Start the VM over p2p1_0
> 7. Below is the Packet I am sending
> a) Dest MAC - VF Mac, Src MAC - any untagged, src ip - 1.1.1.1 dest ip -
> 2.2.2.2 src port - 100 dest port - 4789
> b) Dest MAC - VF Mac, Src MAC - any, untagged, src ip - 1.1.1.1 dest ip -
> 2.2.2.2 src port - 100 dest port - 4790
> c) Dest MAC - VF Mac, untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src
> port - 100 dest port - 4791
>
> All the above testing is done on centOs-6.7 with ixgbe version - 4.3.13
> with patch you mentioned on 82599 Ethernet controller
> Linux XXX 2.6.32-573.22.1.el6.x86_64 #1 SMP Wed Mar 23 03:35:39 UTC 2016
> x86_64 x86_64 x86_64 GNU/Linux
>
>
> *Observation: *
> If the packet matches the rule, I am not able to see the packet in the VF,
> instead I am able to see the packet in PF.
> for the packets a) and b), I am able to see te packet only in PF. Even if
> the packet destination MAC is VF's MAC,
> I am able to see only in PF.
> If the packet is not matching the rule, then I am able to see the packet
> in VF, provided packet destination MAC is VF's MAC.
> *Question: *
> 1) Am I mapping the queues wrongly while adding the rules?
> 2) How to Identify which VF using which Queues?
>
>     Request you to provide some help on it.
>
>
>
>
>
> On Tue, Mar 22, 2016 at 1:03 PM, bharath paulraj <bharathpaul@gmail.com>
> wrote:
>
> Thanks a lot Michael.  Finally i am able to see some light. I will try the
> same in our setup and will post you the results.
>
>
>
> Thanks,
>
> Bharath
>
>
>
> On Tue, Mar 22, 2016 at 12:09 PM, Qiu, Michael <michael.qiu@intel.com>
> wrote:
>
> Yes, we could let ovs using 82599 VF to do rx/tx. I don't know what's
> your l2 bridge, but since ovs could work I think your bridge also could
> work. But I only tested with one VF.
>
> Make sure below two patches (bifurcate driver) are included in your kernel:
>
> _https://patchwork.ozlabs.org/patch/476511/_
> _https://patchwork.ozlabs.org/patch/476516/_
>
> Mostly, if your kernel version in 4.2 or newer, it should be included.
>
> After you create VF, before you passthrough the VF to guest:
>
> (vf +1) << 32 + queue-index,
>
>
>  1. where vf is the VF index starting from 0
>  2. the queue-index is 0 if multi-queue support is not turned on, and
>     this value is [0,1] if multiple-queue is turned on
>
>
> echo 1 > /sys/bus/pci/devices/0000\:05\:00.0/sriov_numvfs
> ifconfig $(PF_INTF) up
> ifconfig $(VF0_INFT) up
> ip link set $(PF_INTF) promisc on
> ethtool -K $(PF_INTF) ntuple on
> ethtool -N $(PF_INTF) flow-type udp4 dst-port 4789 action 0x100000000
> (VF0 queue 0)
>
> Here we using flow director to all let packets according to the rules to
> the VF, But I don't know if it could let the packets to other VFs at the
> same time.
>
> Thanks,
> Michael
>
> On 3/17/2016 2:43 PM, bharath paulraj wrote:
> > Hi Lu, Helin, Greg,
> >
> >   Many thanks for your response, which is really quick. Now, If I want
> > to implement L2 bridging with Intel virtualization technologies, using
> > 82599 controller, then Michael is my only hope, as getting the new
> > kernel versions and upstream support will take considerable amount of
> > time.
> >
> >    Michael, Could you please share your experience on L2 bridging
> > using Intel virtualization technologies.
> >
> > Thanks,
> > Bharath
> >
> > On Wed, Mar 16, 2016 at 9:40 PM, Rose, Gregory V
> > <gregory.v.rose@intel.com <mailto:gregory.v.rose@intel.com>> wrote:
> >
> >     Intel has not supported promiscuous mode for virtual functions due
> >     to the security concerns mentioned below.
> >
> >     There will be upstream support in an upcoming Linux kernel for
> >     setting virtual functions as "trusted" and when that is available
> >     then Intel will allow virtual functions to enter unicast
> >     promiscuous mode on those Ethernet controllers that support
> >     promiscuous mode for virtual functions in the HW/FW.  Be aware
> >     that not all Intel Ethernet controllers have support for unicast
> >     promiscuous mode for virtual functions.  The only currently
> >     released product that does is the X710/XL710.
> >
> >     The key take away is that unicast promiscuous mode for X710/XL710
> >     virtual functions requires Linux kernel support, iproute2 package
> >     support and driver support.  Only when all three of these are in
> >     place will the feature work.
> >
> >     Thanks,
> >
> >     - Greg
> >
> >     -----Original Message-----
> >     From: Zhang, Helin
> >     Sent: Wednesday, March 16, 2016 9:04 AM
> >     To: bharath paulraj <bharathpaul@gmail.com
> >     <mailto:bharathpaul@gmail.com>>; Lu, Wenzhuo <wenzhuo.lu@intel.com
> >     <mailto:wenzhuo.lu@intel.com>>; Rowden, Aaron F
> >     <aaron.f.rowden@intel.com <mailto:aaron.f.rowden@intel.com>>;
> >     Rose, Gregory V <gregory.v.rose@intel.com
> >     <mailto:gregory.v.rose@intel.com>>
> >     Cc: dev@dpdk.org <mailto:dev@dpdk.org>; Qiu, Michael
> >     <michael.qiu@intel.com <mailto:michael.qiu@intel.com>>; Jayakumar,
> >     Muthurajan <muthurajan.jayakumar@intel.com
> >     <mailto:muthurajan.jayakumar@intel.com>>
> >     Subject: RE: [dpdk-dev] Reg: promiscuous mode on VF
> >
> >     Hi Bharath
> >
> >     For your question of "why intel does not support unicast
> >     promiscuos mode?", I'd ask Aaron or Greg to give answers.
> >     Thank you very much!
> >
> >     Regards,
> >     Helin
> >
> >     > -----Original Message-----
> >     > From: dev [mailto:dev-bounces@dpdk.org
> >     <mailto:dev-bounces@dpdk.org>] On Behalf Of bharath paulraj
> >     > Sent: Wednesday, March 16, 2016 11:29 PM
> >     > To: Lu, Wenzhuo
> >     > Cc: dev@dpdk.org <mailto:dev@dpdk.org>
> >     > Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF
> >     >
> >     > Hi Lu,
> >     >
> >     > Many thanks for your response. Again I have few more queries.
> >     > If VF unicast promiscuous mode is not supported then can't we
> >     > implement a Layer 2 bridging functionality using intel
> >     virtualization
> >     > technologies? Or Is there any other way, say tweeking some hardware
> >     > registers or drivers, which may help us in implementing Layer 2
> >     bridging.
> >     > Also I would like to know, why intel does not support unicast
> >     promiscuos mode?
> >     > It could have been optional register settings and user should
> >     have had
> >     > a previleage to set or unset it. Besides, security reasons, is
> there
> >     > any other big reason why Intel does not support this?
> >     >
> >     > Thanks,
> >     > Bharath Paulraj
> >     >
> >     > On Wed, Mar 16, 2016 at 6:15 AM, Lu, Wenzhuo
> >     <wenzhuo.lu@intel.com <mailto:wenzhuo.lu@intel.com>>
>
> >     > wrote:
> >     >
> >     > > Hi Bharath,
> >     > >
> >     > > >     2) Is the above supported for 82599 controller? If it is
> >     > > > supported
> >     > > in the NIC,
> >     > > > please provide the steps to enable.
> >     > > Talking about 82599, VF unicast promiscuous mode is not
> supported.
> >     > > Only broadcast and multicast can be supported.
> >     > >
> >     > > >
> >     > > > Thanks,
> >     > > > Bharath Paulraj
> >     > >
> >     >
> >     >
> >     >
> >     > --
> >     > Regards,
> >     > Bharath
> >
> >
> >
> >
> > --
> > Regards,
> > Bharath
>
>
>
>
>
> --
>
> Regards,
>
> Bharath
>
>
>
>
>
> --
>
> Regards,
>
> Bharath
>
>
>
>
>
> --
>
> Regards,
>
> Bharath
>



-- 
Regards,
Bharath