* Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() @ 2014-12-30 18:20 Siva Mannem 2015-01-07 12:53 ` Jiri Pirko 0 siblings, 1 reply; 10+ messages in thread From: Siva Mannem @ 2014-12-30 18:20 UTC (permalink / raw) To: netdev Hi, I am trying to understand the ongoing switch device offload effort and am following the discussions. I have a question regarding IFLA_BRPORT_LEARNING_SYNC flag and how aging happens when this flag is enabled on a port that is attached to a bridge that has vlan filtering enabled. If I understand correctly, when IFLA_BRPORT_LEARNING_SYNC is set on a bridge port, fdb entries that are learnt externally(may be learnt by hardware and driver is notified) are synced to bridges fdb using br_fdb_external_learn_add(). The fdb entries(fdb->added_by_external_learn set to true) that are learnt via this method are also deleted by the aging logic after the aging time even though L2 data forwadring happens in hardware. Is there a way where aging can be disabled for these entries? and let the entries be removed only via br_fdb_external_learn_delete()? or am I missing something? Regards, Siva. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2014-12-30 18:20 Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() Siva Mannem @ 2015-01-07 12:53 ` Jiri Pirko 2015-01-09 1:46 ` Scott Feldman 0 siblings, 1 reply; 10+ messages in thread From: Jiri Pirko @ 2015-01-07 12:53 UTC (permalink / raw) To: Siva Mannem; +Cc: netdev, sfeldma Tue, Dec 30, 2014 at 07:20:21PM CET, siva.mannem.lnx@gmail.com wrote: >Hi, > >I am trying to understand the ongoing switch device offload effort and >am following the discussions. I have a question regarding >IFLA_BRPORT_LEARNING_SYNC flag and how aging happens when this flag is >enabled on a port that is attached to a bridge that has vlan filtering >enabled. > >If I understand correctly, when IFLA_BRPORT_LEARNING_SYNC is set on a >bridge port, fdb entries that are learnt externally(may be learnt by >hardware and driver is notified) are synced to bridges fdb using >br_fdb_external_learn_add(). The fdb >entries(fdb->added_by_external_learn set to true) that are learnt via >this method are also deleted by the aging logic after the aging time >even though L2 data forwadring happens in hardware. Is there a way >where aging can be disabled for these entries? and let the entries be >removed only via br_fdb_external_learn_delete()? or am I missing >something? Currently extenaly learned fdb entries are indeed removed during aging cleanup. I believe that br_fdb_cleanup should check added_by_external_learn and not remove that fdbs. What do you think Scott? ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-07 12:53 ` Jiri Pirko @ 2015-01-09 1:46 ` Scott Feldman 2015-01-09 16:15 ` Arad, Ronen 0 siblings, 1 reply; 10+ messages in thread From: Scott Feldman @ 2015-01-09 1:46 UTC (permalink / raw) To: Jiri Pirko; +Cc: Siva Mannem, Netdev On Wed, Jan 7, 2015 at 4:53 AM, Jiri Pirko <jiri@resnulli.us> wrote: > Tue, Dec 30, 2014 at 07:20:21PM CET, siva.mannem.lnx@gmail.com wrote: >>Hi, >> >>I am trying to understand the ongoing switch device offload effort and >>am following the discussions. I have a question regarding >>IFLA_BRPORT_LEARNING_SYNC flag and how aging happens when this flag is >>enabled on a port that is attached to a bridge that has vlan filtering >>enabled. >> >>If I understand correctly, when IFLA_BRPORT_LEARNING_SYNC is set on a >>bridge port, fdb entries that are learnt externally(may be learnt by >>hardware and driver is notified) are synced to bridges fdb using >>br_fdb_external_learn_add(). The fdb >>entries(fdb->added_by_external_learn set to true) that are learnt via >>this method are also deleted by the aging logic after the aging time >>even though L2 data forwadring happens in hardware. This is correct... >> Is there a way >>where aging can be disabled for these entries? and let the entries be >>removed only via br_fdb_external_learn_delete()? or am I missing >>something? > > Currently extenaly learned fdb entries are indeed removed during aging > cleanup. I believe that br_fdb_cleanup should check added_by_external_learn > and not remove that fdbs. What do you think Scott? Something like that would work, if we added another brport flag to control that. With the current arrangement, using bridge for aging out entries, we want br_fdb_cleanup removing the external_learned fdbs, but if there was another brport flag we could fine tune that. Say new flag is IFLA_BRPORT_AGING_OFFLOAD or something like that. I'm not sure how aging settings for the bridge are pushed down to offload hw, or if there is a different set for hw. But, isn't it nice to let Linux bridge control aging? That way, bridge -s fdb dump shows nice statistics on fdb entries. Hardware isn't involved in the aging processes (other than being told to remove an entry). Simple hardware equals simple driver. Linux remains control point. -scott ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-09 1:46 ` Scott Feldman @ 2015-01-09 16:15 ` Arad, Ronen 2015-01-09 18:47 ` Scott Feldman 0 siblings, 1 reply; 10+ messages in thread From: Arad, Ronen @ 2015-01-09 16:15 UTC (permalink / raw) To: Scott Feldman, Jiri Pirko; +Cc: Siva Mannem, Netdev >-----Original Message----- >From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On >Behalf Of Scott Feldman >Sent: Friday, January 09, 2015 3:47 AM >To: Jiri Pirko >Cc: Siva Mannem; Netdev >Subject: Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of >fdb entries learnt via br_fdb_external_learn_add() > >On Wed, Jan 7, 2015 at 4:53 AM, Jiri Pirko <jiri@resnulli.us> wrote: >> Tue, Dec 30, 2014 at 07:20:21PM CET, siva.mannem.lnx@gmail.com wrote: >>>Hi, >>> >>>I am trying to understand the ongoing switch device offload effort and >>>am following the discussions. I have a question regarding >>>IFLA_BRPORT_LEARNING_SYNC flag and how aging happens when this flag is >>>enabled on a port that is attached to a bridge that has vlan filtering >>>enabled. >>> >>>If I understand correctly, when IFLA_BRPORT_LEARNING_SYNC is set on a >>>bridge port, fdb entries that are learnt externally(may be learnt by >>>hardware and driver is notified) are synced to bridges fdb using >>>br_fdb_external_learn_add(). The fdb >>>entries(fdb->added_by_external_learn set to true) that are learnt via >>>this method are also deleted by the aging logic after the aging time >>>even though L2 data forwadring happens in hardware. > >This is correct... > >>> Is there a way >>>where aging can be disabled for these entries? and let the entries be >>>removed only via br_fdb_external_learn_delete()? or am I missing >>>something? >> >> Currently extenaly learned fdb entries are indeed removed during aging >> cleanup. I believe that br_fdb_cleanup should check added_by_external_learn >> and not remove that fdbs. What do you think Scott? > >Something like that would work, if we added another brport flag to >control that. With the current arrangement, using bridge for aging >out entries, we want br_fdb_cleanup removing the external_learned >fdbs, but if there was another brport flag we could fine tune that. >Say new flag is IFLA_BRPORT_AGING_OFFLOAD or something like that. I'm >not sure how aging settings for the bridge are pushed down to offload >hw, or if there is a different set for hw. > >But, isn't it nice to let Linux bridge control aging? That way, >bridge -s fdb dump shows nice statistics on fdb entries. Hardware >isn't involved in the aging processes (other than being told to remove >an entry). Simple hardware equals simple driver. Linux remains >control point. > It is indeed simpler. However, if the overhead of reading hit bits from the HW and updating freshness of entries using br_fdb_external_learn_add() is too expensive, it would force such platforms to disable learning sync altogether. Therefore, I believe aging offload flag (could be sufficient at bridge level) and external aging interval (possibly longer than the software aging interval) will encourage drivers to use leaning sync. >-scott >-- >To unsubscribe from this list: send the line "unsubscribe netdev" in >the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-09 16:15 ` Arad, Ronen @ 2015-01-09 18:47 ` Scott Feldman 2015-01-10 2:31 ` B Viswanath 2015-01-10 6:51 ` Arad, Ronen 0 siblings, 2 replies; 10+ messages in thread From: Scott Feldman @ 2015-01-09 18:47 UTC (permalink / raw) To: Arad, Ronen; +Cc: Jiri Pirko, Siva Mannem, Netdev On Fri, Jan 9, 2015 at 8:15 AM, Arad, Ronen <ronen.arad@intel.com> wrote: > > >>-----Original Message----- >>From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On >>Behalf Of Scott Feldman >>Sent: Friday, January 09, 2015 3:47 AM >>To: Jiri Pirko >>Cc: Siva Mannem; Netdev >>Subject: Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of >>fdb entries learnt via br_fdb_external_learn_add() >> >>On Wed, Jan 7, 2015 at 4:53 AM, Jiri Pirko <jiri@resnulli.us> wrote: >>> Tue, Dec 30, 2014 at 07:20:21PM CET, siva.mannem.lnx@gmail.com wrote: >>>>Hi, >>>> >>>>I am trying to understand the ongoing switch device offload effort and >>>>am following the discussions. I have a question regarding >>>>IFLA_BRPORT_LEARNING_SYNC flag and how aging happens when this flag is >>>>enabled on a port that is attached to a bridge that has vlan filtering >>>>enabled. >>>> >>>>If I understand correctly, when IFLA_BRPORT_LEARNING_SYNC is set on a >>>>bridge port, fdb entries that are learnt externally(may be learnt by >>>>hardware and driver is notified) are synced to bridges fdb using >>>>br_fdb_external_learn_add(). The fdb >>>>entries(fdb->added_by_external_learn set to true) that are learnt via >>>>this method are also deleted by the aging logic after the aging time >>>>even though L2 data forwadring happens in hardware. >> >>This is correct... >> >>>> Is there a way >>>>where aging can be disabled for these entries? and let the entries be >>>>removed only via br_fdb_external_learn_delete()? or am I missing >>>>something? >>> >>> Currently extenaly learned fdb entries are indeed removed during aging >>> cleanup. I believe that br_fdb_cleanup should check added_by_external_learn >>> and not remove that fdbs. What do you think Scott? >> >>Something like that would work, if we added another brport flag to >>control that. With the current arrangement, using bridge for aging >>out entries, we want br_fdb_cleanup removing the external_learned >>fdbs, but if there was another brport flag we could fine tune that. >>Say new flag is IFLA_BRPORT_AGING_OFFLOAD or something like that. I'm >>not sure how aging settings for the bridge are pushed down to offload >>hw, or if there is a different set for hw. >> >>But, isn't it nice to let Linux bridge control aging? That way, >>bridge -s fdb dump shows nice statistics on fdb entries. Hardware >>isn't involved in the aging processes (other than being told to remove >>an entry). Simple hardware equals simple driver. Linux remains >>control point. >> > It is indeed simpler. However, if the overhead of reading hit bits from the HW > and updating freshness of entries using br_fdb_external_learn_add() is too > expensive, it would force such platforms to disable learning sync altogether. > Therefore, I believe aging offload flag (could be sufficient at bridge level) > and external aging interval (possibly longer than the software aging interval) > will encourage drivers to use leaning sync. >>-scott I'm not sure I follow that last part. Can we list out the use-cases to see what's missing? Case 1: bridge ages out learned_sync'ed entries bridge port learning: off offload port learning: on offload port learning_sync: on Driver calls br_fdb_external_learn_add() periodically to refresh bridge fdb entry to keep it from going stale. Bridge removes aged out fdb entries (and indirectly tells offload device to do the same). Case 2: offload device/bridge age out entries independently bridge port learning: on offload port learning: on offload port learning_sync: off Bridge ages out its stale learned entries, independent of offload device. Offload device ages out its stale learned entries, independent of bridge. Case 3: ? Please help me finish the use-case list so we can see what's missing. -scott ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-09 18:47 ` Scott Feldman @ 2015-01-10 2:31 ` B Viswanath 2015-01-10 6:51 ` Arad, Ronen 1 sibling, 0 replies; 10+ messages in thread From: B Viswanath @ 2015-01-10 2:31 UTC (permalink / raw) To: Scott Feldman; +Cc: Arad, Ronen, Jiri Pirko, Siva Mannem, Netdev Hi Scot, I think we can see the problem in case 1 itself. Driver's can't periodically update kernel about refreshing the fdb entry in reality. Some of the switches I worked on a while ago have 16K entries in a MAC table, Surely it has grown folds now. For the driver to be able to refresh kernel periodically would be a nightmare, performance wise, to poll this many entries from hardware, see which of them is updated. The driver is updating the kernel about learnt entry. I think it makes sence to have driver update the kernel about ageing too. Thanks Viswanath On 10 January 2015 at 00:17, Scott Feldman <sfeldma@gmail.com> wrote: > On Fri, Jan 9, 2015 at 8:15 AM, Arad, Ronen <ronen.arad@intel.com> wrote: >> >> >>>-----Original Message----- >>>From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On >>>Behalf Of Scott Feldman >>>Sent: Friday, January 09, 2015 3:47 AM >>>To: Jiri Pirko >>>Cc: Siva Mannem; Netdev >>>Subject: Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of >>>fdb entries learnt via br_fdb_external_learn_add() >>> >>>On Wed, Jan 7, 2015 at 4:53 AM, Jiri Pirko <jiri@resnulli.us> wrote: >>>> Tue, Dec 30, 2014 at 07:20:21PM CET, siva.mannem.lnx@gmail.com wrote: >>>>>Hi, >>>>> >>>>>I am trying to understand the ongoing switch device offload effort and >>>>>am following the discussions. I have a question regarding >>>>>IFLA_BRPORT_LEARNING_SYNC flag and how aging happens when this flag is >>>>>enabled on a port that is attached to a bridge that has vlan filtering >>>>>enabled. >>>>> >>>>>If I understand correctly, when IFLA_BRPORT_LEARNING_SYNC is set on a >>>>>bridge port, fdb entries that are learnt externally(may be learnt by >>>>>hardware and driver is notified) are synced to bridges fdb using >>>>>br_fdb_external_learn_add(). The fdb >>>>>entries(fdb->added_by_external_learn set to true) that are learnt via >>>>>this method are also deleted by the aging logic after the aging time >>>>>even though L2 data forwadring happens in hardware. >>> >>>This is correct... >>> >>>>> Is there a way >>>>>where aging can be disabled for these entries? and let the entries be >>>>>removed only via br_fdb_external_learn_delete()? or am I missing >>>>>something? >>>> >>>> Currently extenaly learned fdb entries are indeed removed during aging >>>> cleanup. I believe that br_fdb_cleanup should check added_by_external_learn >>>> and not remove that fdbs. What do you think Scott? >>> >>>Something like that would work, if we added another brport flag to >>>control that. With the current arrangement, using bridge for aging >>>out entries, we want br_fdb_cleanup removing the external_learned >>>fdbs, but if there was another brport flag we could fine tune that. >>>Say new flag is IFLA_BRPORT_AGING_OFFLOAD or something like that. I'm >>>not sure how aging settings for the bridge are pushed down to offload >>>hw, or if there is a different set for hw. >>> >>>But, isn't it nice to let Linux bridge control aging? That way, >>>bridge -s fdb dump shows nice statistics on fdb entries. Hardware >>>isn't involved in the aging processes (other than being told to remove >>>an entry). Simple hardware equals simple driver. Linux remains >>>control point. >>> >> It is indeed simpler. However, if the overhead of reading hit bits from the HW >> and updating freshness of entries using br_fdb_external_learn_add() is too >> expensive, it would force such platforms to disable learning sync altogether. >> Therefore, I believe aging offload flag (could be sufficient at bridge level) >> and external aging interval (possibly longer than the software aging interval) >> will encourage drivers to use leaning sync. >>>-scott > > I'm not sure I follow that last part. > > Can we list out the use-cases to see what's missing? > > Case 1: bridge ages out learned_sync'ed entries > > bridge port learning: off > offload port learning: on > offload port learning_sync: on > > Driver calls br_fdb_external_learn_add() periodically to refresh > bridge fdb entry > to keep it from going stale. > Bridge removes aged out fdb entries (and indirectly tells offload > device to do the same). > > Case 2: offload device/bridge age out entries independently > > bridge port learning: on > offload port learning: on > offload port learning_sync: off > > Bridge ages out its stale learned entries, independent of offload device. > Offload device ages out its stale learned entries, independent of bridge. > > Case 3: ? > > Please help me finish the use-case list so we can see what's missing. > > -scott > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-09 18:47 ` Scott Feldman 2015-01-10 2:31 ` B Viswanath @ 2015-01-10 6:51 ` Arad, Ronen 2015-01-10 20:28 ` Scott Feldman 1 sibling, 1 reply; 10+ messages in thread From: Arad, Ronen @ 2015-01-10 6:51 UTC (permalink / raw) To: Scott Feldman, Netdev; +Cc: Jiri Pirko, Siva Mannem [...] >> It is indeed simpler. However, if the overhead of reading hit bits from the >HW >> and updating freshness of entries using br_fdb_external_learn_add() is too >> expensive, it would force such platforms to disable learning sync >altogether. >> Therefore, I believe aging offload flag (could be sufficient at bridge >level) >> and external aging interval (possibly longer than the software aging >interval) >> will encourage drivers to use leaning sync. >>>-scott > >I'm not sure I follow that last part. > >Can we list out the use-cases to see what's missing? > >Case 1: bridge ages out learned_sync'ed entries > >bridge port learning: off >offload port learning: on >offload port learning_sync: on > >Driver calls br_fdb_external_learn_add() periodically to refresh >bridge fdb entry >to keep it from going stale. >Bridge removes aged out fdb entries (and indirectly tells offload >device to do the same). > >Case 2: offload device/bridge age out entries independently > >bridge port learning: on >offload port learning: on >offload port learning_sync: off > >Bridge ages out its stale learned entries, independent of offload device. >Offload device ages out its stale learned entries, independent of bridge. > >Case 3: ? > >Please help me finish the use-case list so we can see what's missing. Case 3: offload device ages out external entries and notifies bridge bridge port learning: on or off (Bridge only learns from packets seen (Rx/Tx)) offload port learning: on offload port learning_sync: on bridge aging of external learn: off offload device aging: on Switch port/device driver ages entries (could be by HW aging or soft aging in driver/firmware), notifies bridge about aged entries using br_fdb_externallearn_del(). This allows efficient HW aging and batched notification at a pace independent of bridge aging interval. User still enjoys a single VLAN-aware FDB within the bridge module and having all entries in one place. Externally learned entries are identified as such by iproute2 "bridge fdb show" command. Device does not have to implement ndo_bridge_fdb_dump() for each offload port as the bridge module provides it for the common FDB. Case 4: bridge ages owned and external entries at different intervals bridge port learning: on (Effectively only for Rx/Tx traffic seen by software bridge) offload port learning: on (transient traffic and RxTx, overlap with bridge learned entries possible) offload port learning_sync: on bridge aging of external learn: on offload device aging: off bridge aging interval for owned entries: T1 bridge aging interval for external entries: T2 (Typically T2 > T1) This allows for fine-tuning the overhead of periodic updates of entries freshness from offload port device. The bottom line of cases 3-4 is that it is desirable to use the common bridge FDB as long as bridge aging of externally learned entries could be controlled by the offload device: Either be at a longer interval or disabled. > >-scott >-- >To unsubscribe from this list: send the line "unsubscribe netdev" in >the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-10 6:51 ` Arad, Ronen @ 2015-01-10 20:28 ` Scott Feldman 2015-01-12 11:00 ` Arad, Ronen 0 siblings, 1 reply; 10+ messages in thread From: Scott Feldman @ 2015-01-10 20:28 UTC (permalink / raw) To: Arad, Ronen; +Cc: Netdev, Jiri Pirko, Siva Mannem, marichika4 Perfect, I think we have a good working set of use-cases. Thanks for adding those. Your case 3 seems do-able without too much work since we already know which ones where externally added, just need another per-bridge-port flag to turn off bridging aging of externally learned entries. This will address the performance issue you (and B Viswanath) raised. What about the entry stats, from the user's 'bridge -s fdb show" perspective for the bridge fdb entries? Will these numbers match expectations? I think case 1 and case 4 provide a coherent stats view. Case 3 seems to be lacking in this regard. On Fri, Jan 9, 2015 at 10:51 PM, Arad, Ronen <ronen.arad@intel.com> wrote: > [...] >>> It is indeed simpler. However, if the overhead of reading hit bits from the >>HW >>> and updating freshness of entries using br_fdb_external_learn_add() is too >>> expensive, it would force such platforms to disable learning sync >>altogether. >>> Therefore, I believe aging offload flag (could be sufficient at bridge >>level) >>> and external aging interval (possibly longer than the software aging >>interval) >>> will encourage drivers to use leaning sync. >>>>-scott >> >>I'm not sure I follow that last part. >> >>Can we list out the use-cases to see what's missing? >> >>Case 1: bridge ages out learned_sync'ed entries >> >>bridge port learning: off >>offload port learning: on >>offload port learning_sync: on >> >>Driver calls br_fdb_external_learn_add() periodically to refresh >>bridge fdb entry >>to keep it from going stale. >>Bridge removes aged out fdb entries (and indirectly tells offload >>device to do the same). >> >>Case 2: offload device/bridge age out entries independently >> >>bridge port learning: on >>offload port learning: on >>offload port learning_sync: off >> >>Bridge ages out its stale learned entries, independent of offload device. >>Offload device ages out its stale learned entries, independent of bridge. >> >>Case 3: ? >> >>Please help me finish the use-case list so we can see what's missing. > > > Case 3: offload device ages out external entries and notifies bridge > > bridge port learning: on or off (Bridge only learns from packets seen (Rx/Tx)) > offload port learning: on > offload port learning_sync: on > bridge aging of external learn: off > offload device aging: on > > Switch port/device driver ages entries (could be by HW aging or soft aging in > driver/firmware), > notifies bridge about aged entries using br_fdb_externallearn_del(). > This allows efficient HW aging and batched notification at a pace independent > of bridge aging interval. > User still enjoys a single VLAN-aware FDB within the bridge module and having > all entries in one place. Externally learned entries are identified as such by > iproute2 "bridge fdb show" command. Device does not have to implement > ndo_bridge_fdb_dump() for each offload port as the bridge module provides it > for the common FDB. > > Case 4: bridge ages owned and external entries at different intervals > > bridge port learning: on (Effectively only for Rx/Tx traffic seen by > software bridge) > offload port learning: on (transient traffic and RxTx, overlap with bridge > learned entries possible) > offload port learning_sync: on > bridge aging of external learn: on > offload device aging: off > bridge aging interval for owned entries: T1 > bridge aging interval for external entries: T2 (Typically T2 > T1) > > This allows for fine-tuning the overhead of periodic updates of entries > freshness from offload port device. > > The bottom line of cases 3-4 is that it is desirable to use the common bridge > FDB as long as bridge aging of externally learned entries could be controlled > by the offload device: Either be at a longer interval or disabled. > >> >>-scott >>-- >>To unsubscribe from this list: send the line "unsubscribe netdev" in >>the body of a message to majordomo@vger.kernel.org >>More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-10 20:28 ` Scott Feldman @ 2015-01-12 11:00 ` Arad, Ronen 2015-01-12 12:24 ` B Viswanath 0 siblings, 1 reply; 10+ messages in thread From: Arad, Ronen @ 2015-01-12 11:00 UTC (permalink / raw) To: Scott Feldman, Netdev; +Cc: Jiri Pirko, Siva Mannem, marichika4 >-----Original Message----- >From: Scott Feldman [mailto:sfeldma@gmail.com] >Sent: Saturday, January 10, 2015 10:29 PM >To: Arad, Ronen >Cc: Netdev; Jiri Pirko; Siva Mannem; marichika4@gmail.com >Subject: Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of >fdb entries learnt via br_fdb_external_learn_add() > >Perfect, I think we have a good working set of use-cases. Thanks for >adding those. Your case 3 seems do-able without too much work since >we already know which ones where externally added, just need another >per-bridge-port flag to turn off bridging aging of externally learned >entries. This will address the performance issue you (and B >Viswanath) raised. > What about the entry stats, from the user's >'bridge -s fdb show" perspective for the bridge fdb entries? Will >these numbers match expectations? I think case 1 and case 4 provide a >coherent stats view. Case 3 seems to be lacking in this regard. > I think that statistics accuracy is orthogonal to the mechanism used for aging in all the cases where learning sync is enabled (i.e. cases 1, 3, 4). Accuracy only depends on how frequent the switch port driver notifies the bridge FDB. The best accuracy achievable is with updates once per-second. At the other extreme the switch port driver does not refresh entries. It only notifies the bridge after entries are removed from the HW. In this extreme case the statistics will really show the time since an entry was first learned and not the time since it was last re-learned in HW. Switch port driver could pick some acceptable rate to update the bridge module. Within a typical 5 minutes aging interval, updates every 10 seconds or every 30 seconds could be a reasonable tradeoff between statistics accuracy and system overhead. >On Fri, Jan 9, 2015 at 10:51 PM, Arad, Ronen <ronen.arad@intel.com> wrote: >> [...] >>>> It is indeed simpler. However, if the overhead of reading hit bits from >the >>>HW >>>> and updating freshness of entries using br_fdb_external_learn_add() is too >>>> expensive, it would force such platforms to disable learning sync >>>altogether. >>>> Therefore, I believe aging offload flag (could be sufficient at bridge >>>level) >>>> and external aging interval (possibly longer than the software aging >>>interval) >>>> will encourage drivers to use leaning sync. >>>>>-scott >>> >>>I'm not sure I follow that last part. >>> >>>Can we list out the use-cases to see what's missing? >>> >>>Case 1: bridge ages out learned_sync'ed entries >>> >>>bridge port learning: off >>>offload port learning: on >>>offload port learning_sync: on >>> >>>Driver calls br_fdb_external_learn_add() periodically to refresh >>>bridge fdb entry >>>to keep it from going stale. >>>Bridge removes aged out fdb entries (and indirectly tells offload >>>device to do the same). >>> >>>Case 2: offload device/bridge age out entries independently >>> >>>bridge port learning: on >>>offload port learning: on >>>offload port learning_sync: off >>> >>>Bridge ages out its stale learned entries, independent of offload device. >>>Offload device ages out its stale learned entries, independent of bridge. >>> >>>Case 3: ? >>> >>>Please help me finish the use-case list so we can see what's missing. >> >> >> Case 3: offload device ages out external entries and notifies bridge >> >> bridge port learning: on or off (Bridge only learns from packets seen >(Rx/Tx)) >> offload port learning: on >> offload port learning_sync: on >> bridge aging of external learn: off >> offload device aging: on >> >> Switch port/device driver ages entries (could be by HW aging or soft aging >in >> driver/firmware), >> notifies bridge about aged entries using br_fdb_externallearn_del(). >> This allows efficient HW aging and batched notification at a pace >independent >> of bridge aging interval. >> User still enjoys a single VLAN-aware FDB within the bridge module and >having >> all entries in one place. Externally learned entries are identified as such >by >> iproute2 "bridge fdb show" command. Device does not have to implement >> ndo_bridge_fdb_dump() for each offload port as the bridge module provides it >> for the common FDB. >> >> Case 4: bridge ages owned and external entries at different intervals >> >> bridge port learning: on (Effectively only for Rx/Tx traffic seen by >> software bridge) >> offload port learning: on (transient traffic and RxTx, overlap with bridge >> learned entries possible) >> offload port learning_sync: on >> bridge aging of external learn: on >> offload device aging: off >> bridge aging interval for owned entries: T1 >> bridge aging interval for external entries: T2 (Typically T2 > T1) >> >> This allows for fine-tuning the overhead of periodic updates of entries >> freshness from offload port device. >> >> The bottom line of cases 3-4 is that it is desirable to use the common >bridge >> FDB as long as bridge aging of externally learned entries could be >controlled >> by the offload device: Either be at a longer interval or disabled. >> >>> >>>-scott >>>-- >>>To unsubscribe from this list: send the line "unsubscribe netdev" in >>>the body of a message to majordomo@vger.kernel.org >>>More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() 2015-01-12 11:00 ` Arad, Ronen @ 2015-01-12 12:24 ` B Viswanath 0 siblings, 0 replies; 10+ messages in thread From: B Viswanath @ 2015-01-12 12:24 UTC (permalink / raw) To: Arad, Ronen; +Cc: Scott Feldman, Netdev, Jiri Pirko, Siva Mannem On 12 January 2015 at 16:30, Arad, Ronen <ronen.arad@intel.com> wrote: > > >>-----Original Message----- >>From: Scott Feldman [mailto:sfeldma@gmail.com] >>Sent: Saturday, January 10, 2015 10:29 PM >>To: Arad, Ronen >>Cc: Netdev; Jiri Pirko; Siva Mannem; marichika4@gmail.com >>Subject: Re: Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of >>fdb entries learnt via br_fdb_external_learn_add() >> >>Perfect, I think we have a good working set of use-cases. Thanks for >>adding those. Your case 3 seems do-able without too much work since >>we already know which ones where externally added, just need another >>per-bridge-port flag to turn off bridging aging of externally learned >>entries. This will address the performance issue you (and B >>Viswanath) raised. >> What about the entry stats, from the user's >>'bridge -s fdb show" perspective for the bridge fdb entries? Will >>these numbers match expectations? I think case 1 and case 4 provide a >>coherent stats view. Case 3 seems to be lacking in this regard. >> > I think that statistics accuracy is orthogonal to the mechanism used for aging > in all the cases where learning sync is enabled (i.e. cases 1, 3, 4). > Accuracy only depends on how frequent the switch port driver notifies the > bridge FDB. The best accuracy achievable is with updates once per-second. At > the other extreme the switch port driver does not refresh entries. It only > notifies the bridge after entries are removed from the HW. In this extreme case > the statistics will really show the time since an entry was first learned and > not the time since it was last re-learned in HW. > Switch port driver could pick some > acceptable rate to update the bridge module. Within a typical 5 minutes aging > interval, updates every 10 seconds or every 30 seconds could be a reasonable > tradeoff between statistics accuracy and system overhead. > I would also like to add that 'whether the end user will really be interested in the FDB, (and the hit notifications from silicon)' depends on the actual deployment. In a L3 dominant deployment, people may not care much about FDB at all. So any time spent inside the driver trying to update the FDB for statistical reasons will be a waste of CPU time. On the other hand for an Access switch (or a Controller) deployment, FDB may really be useful. Unfortunately, the driver will not be able to guess the deployment. I am thinking aloud here, but may be we can let this update-timer to be a configurable item with default being current behaviour (determined by driver). It serves both purposes. Thanks Vissu >>On Fri, Jan 9, 2015 at 10:51 PM, Arad, Ronen <ronen.arad@intel.com> wrote: >>> [...] >>>>> It is indeed simpler. However, if the overhead of reading hit bits from >>the >>>>HW >>>>> and updating freshness of entries using br_fdb_external_learn_add() is too >>>>> expensive, it would force such platforms to disable learning sync >>>>altogether. >>>>> Therefore, I believe aging offload flag (could be sufficient at bridge >>>>level) >>>>> and external aging interval (possibly longer than the software aging >>>>interval) >>>>> will encourage drivers to use leaning sync. >>>>>>-scott >>>> >>>>I'm not sure I follow that last part. >>>> >>>>Can we list out the use-cases to see what's missing? >>>> >>>>Case 1: bridge ages out learned_sync'ed entries >>>> >>>>bridge port learning: off >>>>offload port learning: on >>>>offload port learning_sync: on >>>> >>>>Driver calls br_fdb_external_learn_add() periodically to refresh >>>>bridge fdb entry >>>>to keep it from going stale. >>>>Bridge removes aged out fdb entries (and indirectly tells offload >>>>device to do the same). >>>> >>>>Case 2: offload device/bridge age out entries independently >>>> >>>>bridge port learning: on >>>>offload port learning: on >>>>offload port learning_sync: off >>>> >>>>Bridge ages out its stale learned entries, independent of offload device. >>>>Offload device ages out its stale learned entries, independent of bridge. >>>> >>>>Case 3: ? >>>> >>>>Please help me finish the use-case list so we can see what's missing. >>> >>> >>> Case 3: offload device ages out external entries and notifies bridge >>> >>> bridge port learning: on or off (Bridge only learns from packets seen >>(Rx/Tx)) >>> offload port learning: on >>> offload port learning_sync: on >>> bridge aging of external learn: off >>> offload device aging: on >>> >>> Switch port/device driver ages entries (could be by HW aging or soft aging >>in >>> driver/firmware), >>> notifies bridge about aged entries using br_fdb_externallearn_del(). >>> This allows efficient HW aging and batched notification at a pace >>independent >>> of bridge aging interval. >>> User still enjoys a single VLAN-aware FDB within the bridge module and >>having >>> all entries in one place. Externally learned entries are identified as such >>by >>> iproute2 "bridge fdb show" command. Device does not have to implement >>> ndo_bridge_fdb_dump() for each offload port as the bridge module provides it >>> for the common FDB. >>> >>> Case 4: bridge ages owned and external entries at different intervals >>> >>> bridge port learning: on (Effectively only for Rx/Tx traffic seen by >>> software bridge) >>> offload port learning: on (transient traffic and RxTx, overlap with bridge >>> learned entries possible) >>> offload port learning_sync: on >>> bridge aging of external learn: on >>> offload device aging: off >>> bridge aging interval for owned entries: T1 >>> bridge aging interval for external entries: T2 (Typically T2 > T1) >>> >>> This allows for fine-tuning the overhead of periodic updates of entries >>> freshness from offload port device. >>> >>> The bottom line of cases 3-4 is that it is desirable to use the common >>bridge >>> FDB as long as bridge aging of externally learned entries could be >>controlled >>> by the offload device: Either be at a longer interval or disabled. >>> >>>> >>>>-scott >>>>-- >>>>To unsubscribe from this list: send the line "unsubscribe netdev" in >>>>the body of a message to majordomo@vger.kernel.org >>>>More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-01-12 12:24 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-12-30 18:20 Clarification regarding IFLA_BRPORT_LEARNING_SYNC and aging of fdb entries learnt via br_fdb_external_learn_add() Siva Mannem 2015-01-07 12:53 ` Jiri Pirko 2015-01-09 1:46 ` Scott Feldman 2015-01-09 16:15 ` Arad, Ronen 2015-01-09 18:47 ` Scott Feldman 2015-01-10 2:31 ` B Viswanath 2015-01-10 6:51 ` Arad, Ronen 2015-01-10 20:28 ` Scott Feldman 2015-01-12 11:00 ` Arad, Ronen 2015-01-12 12:24 ` B Viswanath
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.