From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Qiu, Michael" Subject: Re: [PATCH v2] ixgbe: Fix disable interrupt twice Date: Tue, 2 Feb 2016 03:15:27 +0000 Message-ID: <533710CFB86FA344BFBF2D6802E6028622F28C8E@SHSMSX101.ccr.corp.intel.com> References: <1454046700-20843-1-git-send-email-michael.qiu@intel.com> <1454047090-21274-1-git-send-email-michael.qiu@intel.com> <6A0DE07E22DDAD4C9103DF62FEBC0909034256DA@shsmsx102.ccr.corp.intel.com> <533710CFB86FA344BFBF2D6802E6028622F28091@SHSMSX101.ccr.corp.intel.com> <6A0DE07E22DDAD4C9103DF62FEBC0909034266D1@shsmsx102.ccr.corp.intel.com> <533710CFB86FA344BFBF2D6802E6028622F28A4D@SHSMSX101.ccr.corp.intel.com> <533710CFB86FA344BFBF2D6802E6028622F28B7B@SHSMSX101.ccr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable To: "Zhang, Helin" , "Lu, Wenzhuo" , "dev@dpdk.org" Return-path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id A4E3295C8 for ; Tue, 2 Feb 2016 04:15:55 +0100 (CET) Content-Language: en-US List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 2/2/2016 11:07 AM, Zhang, Helin wrote:=0A= >=0A= >> -----Original Message-----=0A= >> From: Qiu, Michael=0A= >> Sent: Tuesday, February 2, 2016 10:57 AM=0A= >> To: Zhang, Helin ; Lu, Wenzhuo=0A= >> ; dev@dpdk.org=0A= >> Cc: Zhou, Danny ; Liu, Yong ;= =0A= >> Liang, Cunming =0A= >> Subject: Re: [PATCH v2] ixgbe: Fix disable interrupt twice=0A= >>=0A= >> On 2/2/2016 10:14 AM, Zhang, Helin wrote:=0A= >>>> -----Original Message-----=0A= >>>> From: Qiu, Michael=0A= >>>> Sent: Tuesday, February 2, 2016 10:07 AM=0A= >>>> To: Lu, Wenzhuo; dev@dpdk.org=0A= >>>> Cc: Zhou, Danny; Liu, Yong; Liang, Cunming; Zhang, Helin=0A= >>>> Subject: Re: [PATCH v2] ixgbe: Fix disable interrupt twice=0A= >>>>=0A= >>>> [+cc helin]=0A= >>>>=0A= >>>> On 2/2/2016 9:03 AM, Lu, Wenzhuo wrote:=0A= >>>>> Hi Michael,=0A= >>>>>=0A= >>>>>> -----Original Message-----=0A= >>>>>> From: Qiu, Michael=0A= >>>>>> Sent: Monday, February 1, 2016 4:05 PM=0A= >>>>>> To: Lu, Wenzhuo; dev@dpdk.org=0A= >>>>>> Cc: Zhou, Danny; Liu, Yong; Liang, Cunming=0A= >>>>>> Subject: Re: [PATCH v2] ixgbe: Fix disable interrupt twice=0A= >>>>>>=0A= >>>>>> On 1/29/2016 4:07 PM, Lu, Wenzhuo wrote:=0A= >>>>>>> Hi Michael,=0A= >>>>>>>=0A= >>>>>>>> -----Original Message-----=0A= >>>>>>>> From: Qiu, Michael=0A= >>>>>>>> Sent: Friday, January 29, 2016 1:58 PM=0A= >>>>>>>> To: dev@dpdk.org=0A= >>>>>>>> Cc: Zhou, Danny; Liu, Yong; Liang, Cunming; Lu, Wenzhuo; Qiu,=0A= >>>>>>>> Michael=0A= >>>>>>>> Subject: [PATCH v2] ixgbe: Fix disable interrupt twice=0A= >>>>>>>>=0A= >>>>>>>> Currently, ixgbe vf and pf will disable interrupt twice in stop=0A= >>>>>>>> stage and uninit stage. It will cause an error:=0A= >>>>>>>>=0A= >>>>>>>> testpmd> quit=0A= >>>>>>>>=0A= >>>>>>>> Shutting down port 0...=0A= >>>>>>>> Stopping ports...=0A= >>>>>>>> Done=0A= >>>>>>>> Closing ports...=0A= >>>>>>>> EAL: Error disabling MSI-X interrupts for fd 26=0A= >>>>>>>> Done=0A= >>>>>>>>=0A= >>>>>>>> Becasue the interrupt already been disabled in stop stage.=0A= >>>>>>>> Since it is enabled in init stage, better remove from stop stage.= =0A= >>>>>>> I'm afraid it=92s not a good idea to just remove the intr_disable= =0A= >>>>>>> from=0A= >>>> dev_stop.=0A= >>>>>>> I think dev_stop have the chance to be used independently with=0A= >>>>>>> dev_unint. In=0A= >>>>>> this scenario, we still need intr_disable, right?=0A= >>>>>>> Maybe what we need is some check before we disable the intr:)=0A= >>>>>> Yes, indeed we need some check in disable intr, but it need=0A= >>>>>> additional fields in "struct rte_intr_handle", and it's much saft= =0A= >>>>>> to do so, but as I check i40e/fm10k code, only ixgbe disable it in= =0A= >> dev_stop().=0A= >>>>> I found fm10k doesn=92t enable intr in dev_start. So, I think it's OK= .=0A= >>>>> But i40e=0A= >>>> enables intr in dev_start.=0A= >>>>> To my opinion, it's more like i40e misses the intr_disable in dev_sto= p.=0A= >>>> I don't think i40e miss it, because it not the right please to disable= interrupt.=0A= >>>> because all interrupts are enabled in init stage.=0A= >>>>=0A= >>>> Actually, ixgbe enable the interrupt in init stage, but in dev_start,= =0A= >>>> it disable it first and re-enable, so it just the same with doing noth= ing about=0A= >> interrupt.=0A= >>>> Just think below:=0A= >>>>=0A= >>>> 1. start the port.(interrupt already enabled in init stage, disable=0A= >>>> -->=0A= >>>> re-enable)=0A= >>>> 2. stop the port.(disable interrupt)=0A= >>>> 3. start port again(Try to disable, but failed, already disabled)=0A= >>>>=0A= >>>> Would you think the code has issue?=0A= >>> [Zhang, Helin] in ixgbe PMD, it can be seen that uninit() calls=0A= >>> dev_close(), which calls dev_stop(). So I think the disabling can be do= ne only in=0A= >> dev_stop().=0A= >>> All others can make use of dev_stop to disable the interrupt.=0A= >> As I said, if it is in dev_stop, it will has issue when dev_start --> de= v_stop -->=0A= >> dev_start, this also could applied in i40e and fm10k. If you want to put= it in=0A= >> dev_stop, better to remove enable interrupts in init stage, and only put= it in=0A= >> dev_start.=0A= > Oh, yes, you are talking about the refactoring. That's good, and reasonab= le.=0A= > Please do more validation with LSC, mailbox, rx interrupts, to make sure = there=0A= > is no issue introduced.=0A= =0A= I have no plan to do code refactor, it includes lots of validation, and=0A= will influence many components, time is limited for 2.3. I would like=0A= keep it in uninit and remove it from stop, this only affect ixgbe, and I=0A= have done validation for it.=0A= =0A= Thanks,=0A= Michael=0A= > Thanks,=0A= > Helin=0A= >=0A= >> Thanks,=0A= >> Michael=0A= >>> Regards,=0A= >>> Helin=0A= >>>=0A= >>>> Thanks,=0A= >>>> Michael=0A= >>>>=0A= >>>>> Maybe we can follow fm10k's style.=0A= >>>>>=0A= >>>>>> On other hand, if we remove it in dev_stop, any side effect? In=0A= >>>>>> ixgbe start, it will always disable it first and then re-enable it, = so it's safe.=0A= >>>>> I think you mean we can disable intr anyway even if it has been disab= led.=0A= >>>> Actually, we couldn't, DPDK call VFIO ioctl to kernel to disable=0A= >>>> interrupts, and if we try disable twice, it will return and error.=0A= >>>> That's why I mean we need a flag to show the interrupts stats. If it= =0A= >>>> already disabled, we do not need call in to kernel. just return and=0A= >>>> give a warning message.=0A= >>>>=0A= >>>> Thanks,=0A= >>>> Michael=0A= >>>>=0A= >>>>> Sounds more like why we don't=0A= >>>>> need this patch :)=0A= >>>>>=0A= >>>>>> Thanks,=0A= >>>>>> Michael=0A= >=0A= =0A=