All of lore.kernel.org
 help / color / mirror / Atom feed
* dsa: handling more than 1 cpu port
@ 2016-12-14 10:01 John Crispin
  2016-12-14 10:31 ` Andrew Lunn
  0 siblings, 1 reply; 6+ messages in thread
From: John Crispin @ 2016-12-14 10:01 UTC (permalink / raw)
  To: Andrew Lunn, Florian Fainelli; +Cc: netdev

Hi Andrew,

switches supported by qca8k have 2 gmacs that we can wire an external
mii interface to. Usually this is used to wire 2 of the SoCs MACs to the
switch. Thw switch itself is aware of one of the MACs being the cpu port
and expects this to be port/mac0. Using the other will break the
hardware offloading features. The original series from Matthieu added a
feature to configure the switch in a pass-through mode [1]. This
resulted in us having to define the pass-through inside the dts which
means that we loose userland configurability. Assume the setup as
follows ...

port0 - cpu port wired to SoCs MAC0
port1-4 - the lan ports
port5 - the wan port
port6 - wired to the SoCs MAC1

What i have done now is bring up one bridge for port1-4 and a second one
for port5-6. Once setup I can pass traffic on the SoCs MAC1 and it will
flow via port6 and egress on port5. So far so good, however due to the
way the port based vlans are setup and how the bridge_join/leave() logic
works, port5/6 will also fwd traffic to the cpu port. the driver has now
to tell that we are trunking traffic on eth1 via port6. also the MII
mode is not known to the driver. Adding some hackish register writes
will make this work nicely. My proposed way of fixing this cleanly in an
upstream friendly way would be as follows

1) add an extra dsa,ethernet property to the 2nd MII port

dsa@0 {
        compatible = "qca,ar8xxx";

        dsa,ethernet = <&gmac1>;

	[...]

        switch@0 {
		[...]

                port@5 {
                        reg = <5>;
                        label = "wan";
                        phy-handle = <&phy_port5>;
                };

                port@6 {
                        reg = <6>;
                        label = "gmac2";

                        dsa,ethernet = <&gmac2>;
                        fixed-link {
                                speed = <1000>;
                                full-duplex;
                        };
                };
        };
};

2) fix up the port_bridge_join/leave() logic such that if a port is
present in the bridge that has a reference to a ethernet interface it
will remove all ports in the bridge from the port based vlan of the
actual cpu port.

3) in case of this switch we also need to fiddle with the bcast/mcast
flooding registers

would this be ok and would it be better to probe the extra dsa_ethernet
inside the subsystem or the driver ? i was considering to do add a
dsa_is_trunk_port() or similar to achieve this.

	John




[1] https://patchwork.ozlabs.org/patch/477525/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: dsa: handling more than 1 cpu port
  2016-12-14 10:01 dsa: handling more than 1 cpu port John Crispin
@ 2016-12-14 10:31 ` Andrew Lunn
  2016-12-14 10:35   ` John Crispin
  0 siblings, 1 reply; 6+ messages in thread
From: Andrew Lunn @ 2016-12-14 10:31 UTC (permalink / raw)
  To: John Crispin; +Cc: Florian Fainelli, netdev

On Wed, Dec 14, 2016 at 11:01:54AM +0100, John Crispin wrote:
> Hi Andrew,
> 
> switches supported by qca8k have 2 gmacs that we can wire an external
> mii interface to. Usually this is used to wire 2 of the SoCs MACs to the
> switch. Thw switch itself is aware of one of the MACs being the cpu port
> and expects this to be port/mac0. Using the other will break the
> hardware offloading features.

Just to be sure here. There is no way to use the second port connected
to the CPU as a CPU port?

The Marvell chips do allow this. So i developed a proof of concept
which had a mapping between cpu ports and slave ports. slave port X
should you cpu port y for its traffic. This never got past proof of
concept. 

If this can be made to work for qca8k, i would prefer having this
general concept, than specific hacks for pass through.

	Andrew

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: dsa: handling more than 1 cpu port
  2016-12-14 10:31 ` Andrew Lunn
@ 2016-12-14 10:35   ` John Crispin
  2016-12-14 11:00     ` Andrew Lunn
  0 siblings, 1 reply; 6+ messages in thread
From: John Crispin @ 2016-12-14 10:35 UTC (permalink / raw)
  To: Andrew Lunn; +Cc: Florian Fainelli, netdev



On 14/12/2016 11:31, Andrew Lunn wrote:
> On Wed, Dec 14, 2016 at 11:01:54AM +0100, John Crispin wrote:
>> Hi Andrew,
>>
>> switches supported by qca8k have 2 gmacs that we can wire an external
>> mii interface to. Usually this is used to wire 2 of the SoCs MACs to the
>> switch. Thw switch itself is aware of one of the MACs being the cpu port
>> and expects this to be port/mac0. Using the other will break the
>> hardware offloading features.
> 
> Just to be sure here. There is no way to use the second port connected
> to the CPU as a CPU port?

both macs are considered cpu ports and both allow for the tag to be
injected. for HW NAT/routing/... offloading to work, the lan ports neet
to trunk via port0 and not port6 however.

> 
> The Marvell chips do allow this. So i developed a proof of concept
> which had a mapping between cpu ports and slave ports. slave port X
> should you cpu port y for its traffic. This never got past proof of
> concept. 
> 
> If this can be made to work for qca8k, i would prefer having this
> general concept, than specific hacks for pass through.

oh cool, can you send those patches my way please ? how do you configure
this from userland ? does the cpu port get its on swdev which i just add
to my lan bridge and then add the 2nd cpu port to the wan bridge ?

	John

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: dsa: handling more than 1 cpu port
  2016-12-14 10:35   ` John Crispin
@ 2016-12-14 11:00     ` Andrew Lunn
  2016-12-14 11:11       ` John Crispin
  2016-12-14 15:45       ` John Crispin
  0 siblings, 2 replies; 6+ messages in thread
From: Andrew Lunn @ 2016-12-14 11:00 UTC (permalink / raw)
  To: John Crispin; +Cc: Florian Fainelli, netdev

On Wed, Dec 14, 2016 at 11:35:30AM +0100, John Crispin wrote:
> 
> 
> On 14/12/2016 11:31, Andrew Lunn wrote:
> > On Wed, Dec 14, 2016 at 11:01:54AM +0100, John Crispin wrote:
> >> Hi Andrew,
> >>
> >> switches supported by qca8k have 2 gmacs that we can wire an external
> >> mii interface to. Usually this is used to wire 2 of the SoCs MACs to the
> >> switch. Thw switch itself is aware of one of the MACs being the cpu port
> >> and expects this to be port/mac0. Using the other will break the
> >> hardware offloading features.
> > 
> > Just to be sure here. There is no way to use the second port connected
> > to the CPU as a CPU port?
> 
> both macs are considered cpu ports and both allow for the tag to be
> injected. for HW NAT/routing/... offloading to work, the lan ports neet
> to trunk via port0 and not port6 however.

Maybe you can do a restricted version of the generic solution. LAN
ports are mapped to cpu port0. WAN port to cpu port 6?

> > The Marvell chips do allow this. So i developed a proof of concept
> > which had a mapping between cpu ports and slave ports. slave port X
> > should you cpu port y for its traffic. This never got past proof of
> > concept. 
> > 
> > If this can be made to work for qca8k, i would prefer having this
> > general concept, than specific hacks for pass through.
> 
> oh cool, can you send those patches my way please ? how do you configure
> this from userland ? does the cpu port get its on swdev which i just add
> to my lan bridge and then add the 2nd cpu port to the wan bridge ?

https://github.com/lunn/linux/tree/v4.1-rc4-net-next-multiple-cpu

You don't configure anything from userland. Which was actually a
criticism. It is in device tree. But my solution is generic. Having
one WAN port and four bridges LAN ports is a pure marketing
requirement. The hardware will happily do two WAN ports and 3 LAN
ports, for example. And the switch is happy to take traffic for the
WAN port and a LAN port over the same CPU port, and keep the traffic
separate. So we can have some form of load balancing. We are not
limited to 1->1, 1->4, we can do 1->2, 1->3 to increase the overall
performance. And to the user it is all transparent.

This PoC is for the old DSA binding. The new binding makes it easier
to express this. Which is one of the reasons for the new binding.

	Andrew

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: dsa: handling more than 1 cpu port
  2016-12-14 11:00     ` Andrew Lunn
@ 2016-12-14 11:11       ` John Crispin
  2016-12-14 15:45       ` John Crispin
  1 sibling, 0 replies; 6+ messages in thread
From: John Crispin @ 2016-12-14 11:11 UTC (permalink / raw)
  To: Andrew Lunn; +Cc: Florian Fainelli, netdev



On 14/12/2016 12:00, Andrew Lunn wrote:
> On Wed, Dec 14, 2016 at 11:35:30AM +0100, John Crispin wrote:
>>
>>
>> On 14/12/2016 11:31, Andrew Lunn wrote:
>>> On Wed, Dec 14, 2016 at 11:01:54AM +0100, John Crispin wrote:
>>>> Hi Andrew,
>>>>
>>>> switches supported by qca8k have 2 gmacs that we can wire an external
>>>> mii interface to. Usually this is used to wire 2 of the SoCs MACs to the
>>>> switch. Thw switch itself is aware of one of the MACs being the cpu port
>>>> and expects this to be port/mac0. Using the other will break the
>>>> hardware offloading features.
>>>
>>> Just to be sure here. There is no way to use the second port connected
>>> to the CPU as a CPU port?
>>
>> both macs are considered cpu ports and both allow for the tag to be
>> injected. for HW NAT/routing/... offloading to work, the lan ports neet
>> to trunk via port0 and not port6 however.
> 
> Maybe you can do a restricted version of the generic solution. LAN
> ports are mapped to cpu port0. WAN port to cpu port 6?

hardcoding it is exactly what i want to avoid, same as using magic name
matching. the dts should describe the HW and not the usage dictated by
what is printed on the casing of the switch. as you mention below this
is marketing chatter. imho ports should have any name and we should be
able to bridge them as we feel happy and attach the bridges to any cpu
port that we want.

> 
>>> The Marvell chips do allow this. So i developed a proof of concept
>>> which had a mapping between cpu ports and slave ports. slave port X
>>> should you cpu port y for its traffic. This never got past proof of
>>> concept. 
>>>
>>> If this can be made to work for qca8k, i would prefer having this
>>> general concept, than specific hacks for pass through.
>>
>> oh cool, can you send those patches my way please ? how do you configure
>> this from userland ? does the cpu port get its on swdev which i just add
>> to my lan bridge and then add the 2nd cpu port to the wan bridge ?
> 
> https://github.com/lunn/linux/tree/v4.1-rc4-net-next-multiple-cpu
> 
> You don't configure anything from userland. Which was actually a
> criticism. It is in device tree. But my solution is generic. Having
> one WAN port and four bridges LAN ports is a pure marketing
> requirement. The hardware will happily do two WAN ports and 3 LAN
> ports, for example. And the switch is happy to take traffic for the
> WAN port and a LAN port over the same CPU port, and keep the traffic
> separate. So we can have some form of load balancing. We are not
> limited to 1->1, 1->4, we can do 1->2, 1->3 to increase the overall
> performance. And to the user it is all transparent.
> 
> This PoC is for the old DSA binding. The new binding makes it easier
> to express this. Which is one of the reasons for the new binding.
> 
> 	Andrew
> 

i'll have a look at the patches. thanks !

	John

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: dsa: handling more than 1 cpu port
  2016-12-14 11:00     ` Andrew Lunn
  2016-12-14 11:11       ` John Crispin
@ 2016-12-14 15:45       ` John Crispin
  1 sibling, 0 replies; 6+ messages in thread
From: John Crispin @ 2016-12-14 15:45 UTC (permalink / raw)
  To: Andrew Lunn; +Cc: Florian Fainelli, netdev



On 14/12/2016 12:00, Andrew Lunn wrote:
> On Wed, Dec 14, 2016 at 11:35:30AM +0100, John Crispin wrote:
>>
>>
>> On 14/12/2016 11:31, Andrew Lunn wrote:
>>> On Wed, Dec 14, 2016 at 11:01:54AM +0100, John Crispin wrote:
>>>> Hi Andrew,
>>>>
>>>> switches supported by qca8k have 2 gmacs that we can wire an external
>>>> mii interface to. Usually this is used to wire 2 of the SoCs MACs to the
>>>> switch. Thw switch itself is aware of one of the MACs being the cpu port
>>>> and expects this to be port/mac0. Using the other will break the
>>>> hardware offloading features.
>>>
>>> Just to be sure here. There is no way to use the second port connected
>>> to the CPU as a CPU port?
>>
>> both macs are considered cpu ports and both allow for the tag to be
>> injected. for HW NAT/routing/... offloading to work, the lan ports neet
>> to trunk via port0 and not port6 however.
> 
> Maybe you can do a restricted version of the generic solution. LAN
> ports are mapped to cpu port0. WAN port to cpu port 6?
> 
>>> The Marvell chips do allow this. So i developed a proof of concept
>>> which had a mapping between cpu ports and slave ports. slave port X
>>> should you cpu port y for its traffic. This never got past proof of
>>> concept. 
>>>
>>> If this can be made to work for qca8k, i would prefer having this
>>> general concept, than specific hacks for pass through.
>>
>> oh cool, can you send those patches my way please ? how do you configure
>> this from userland ? does the cpu port get its on swdev which i just add
>> to my lan bridge and then add the 2nd cpu port to the wan bridge ?
> 
> https://github.com/lunn/linux/tree/v4.1-rc4-net-next-multiple-cpu
> 
> You don't configure anything from userland. Which was actually a
> criticism. It is in device tree. But my solution is generic. Having
> one WAN port and four bridges LAN ports is a pure marketing
> requirement. The hardware will happily do two WAN ports and 3 LAN
> ports, for example. And the switch is happy to take traffic for the
> WAN port and a LAN port over the same CPU port, and keep the traffic
> separate. So we can have some form of load balancing. We are not
> limited to 1->1, 1->4, we can do 1->2, 1->3 to increase the overall
> performance. And to the user it is all transparent.
> 
> This PoC is for the old DSA binding. The new binding makes it easier
> to express this. Which is one of the reasons for the new binding.
> 
> 	Andrew
> 

Hi Andrew,

spent some time looking at this and thinking about possible solutions.
my initial idea was to detect which cpu port to based on the cpu port
being included inside the bridge. however that wont allow us to control
ports using the tag outside of a bridge. i think that your approach is
the only sane one. we could add a sysfs interface later, allowing us to
change the default cpu port <-> mappings, but the device tree needs to
include some sane defaults. i'll use your patches as a base for a series.

	John

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2016-12-14 15:45 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-14 10:01 dsa: handling more than 1 cpu port John Crispin
2016-12-14 10:31 ` Andrew Lunn
2016-12-14 10:35   ` John Crispin
2016-12-14 11:00     ` Andrew Lunn
2016-12-14 11:11       ` John Crispin
2016-12-14 15:45       ` John Crispin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.