Associating nodes with phandles for pci devices

Associating nodes with phandles for pci devices

[Chris Packham]

Hi,

I'm working on a new board that has a marvell,dsa switch. Similar to the kirkwood-mv88f6281gtw-ge board. However the major difference is that instead of being connected to an integrated Ethernet port it is connected via a PCI-e Ethernet port (basically a NIC hardwired onto the board).

Generally you don't need to define PCI devices in the .dts because the bus is scanned at run time and the method for identifying devices is well defined. But to satisfy the marvell,dsa binding[1] I need to tell it the Ethernet port it is connected to. The answer might be in the PCI bindings but the urls in the documentation[2] don't appear to be valid anymore.

Is it possible to enumerate PCI devices in the .dts? Or is there a way of satisfying the dsa requirements without knowing the Ethernet device?

Thanks,
Chris
--
[1] - http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/net/dsa/dsa.txt
[2] - http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/pci/pci.txt--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Arnd Bergmann]

On Friday 23 January 2015 19:02:33 Chris Packham wrote:
> Hi,
> 
> I'm working on a new board that has a marvell,dsa switch. Similar to the kirkwood-mv88f6281gtw-ge board. However the major difference is that instead of being connected to an integrated Ethernet port it is connected via a PCI-e Ethernet port (basically a NIC hardwired onto the board).
> 
> Generally you don't need to define PCI devices in the .dts because the bus is scanned at run time and the method for identifying devices is well defined. But to satisfy the marvell,dsa binding[1] I need to tell it the Ethernet port it is connected to. The answer might be in the PCI bindings but the urls in the documentation[2] don't appear to be valid anymore.
> 
> Is it possible to enumerate PCI devices in the .dts? Or is there a way of satisfying the dsa requirements without knowing the Ethernet device?
> 

In general, it is possible, and we do that on PowerPC, but it may be that
there are parts missing on ARM and you have to try it out.

In particular, you need to know the address of the PCI device and then
create a device node that will be associated with the pci_dev->dev.of_node
pointer.

	Arnd

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Chris Packham]

Hi Arnd,

Thanks for the quick response (and sorry for the horribly formatted 
message, travelling with only a webmail interface at the moment).

On Fri, 23 Jan 2015, Arnd Bergmann wrote:

> On Friday 23 January 2015 19:02:33 Chris Packham wrote:
>> Hi,
>>
>> I'm working on a new board that has a marvell,dsa switch. Similar to 
>> the kirkwood-mv88f6281gtw-ge board. However the major difference is 
>> that instead of being connected to an integrated Ethernet port it is 
>> connected via a PCI-e Ethernet port (basically a NIC hardwired onto the 
>> board).
>>
>> Generally you don't need to define PCI devices in the .dts because the 
>> bus is scanned at run time and the method for identifying devices is 
>> well defined. But to satisfy the marvell,dsa binding[1] I need to tell 
>> it the Ethernet port it is connected to. The answer might be in the PCI 
>> bindings but the urls in the documentation[2] don't appear to be valid 
>> anymore.
>>
>> Is it possible to enumerate PCI devices in the .dts? Or is there a way 
>> of satisfying the dsa requirements without knowing the Ethernet device?
>>
>
> In general, it is possible, and we do that on PowerPC, but it may be that
> there are parts missing on ARM and you have to try it out.
>
> In particular, you need to know the address of the PCI device and then
> create a device node that will be associated with the pci_dev->dev.of_node
> pointer.

Encouraging to know. I think I can figure out the relevant addresses for 
my device based on the run-time scan.

Can you think of a good example. I found a few ppc boards that have an 
PCI-ISA bridge that get's enumerated in the .dts. I think I can make sense
of them but because they are bridges there's a lot of extra stuff for the 
down stream devices.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Florian Fainelli]

On 23/01/15 11:38, Chris Packham wrote:
> 
> Hi Arnd,
> 
> Thanks for the quick response (and sorry for the horribly formatted
> message, travelling with only a webmail interface at the moment).
> 
> On Fri, 23 Jan 2015, Arnd Bergmann wrote:
> 
>> On Friday 23 January 2015 19:02:33 Chris Packham wrote:
>>> Hi,
>>>
>>> I'm working on a new board that has a marvell,dsa switch. Similar to
>>> the kirkwood-mv88f6281gtw-ge board. However the major difference is
>>> that instead of being connected to an integrated Ethernet port it is
>>> connected via a PCI-e Ethernet port (basically a NIC hardwired onto
>>> the board).
>>>
>>> Generally you don't need to define PCI devices in the .dts because
>>> the bus is scanned at run time and the method for identifying devices
>>> is well defined. But to satisfy the marvell,dsa binding[1] I need to
>>> tell it the Ethernet port it is connected to. The answer might be in
>>> the PCI bindings but the urls in the documentation[2] don't appear to
>>> be valid anymore.
>>>
>>> Is it possible to enumerate PCI devices in the .dts? Or is there a
>>> way of satisfying the dsa requirements without knowing the Ethernet
>>> device?
>>>
>>
>> In general, it is possible, and we do that on PowerPC, but it may be that
>> there are parts missing on ARM and you have to try it out.
>>
>> In particular, you need to know the address of the PCI device and then
>> create a device node that will be associated with the
>> pci_dev->dev.of_node
>> pointer.
> 
> Encouraging to know. I think I can figure out the relevant addresses for
> my device based on the run-time scan.

BTW, one thing that is high in my TODO list regarding DSA is to stop
using this platform device/driver architecture and use the actual device
driver model. Your switch driver entry point would be a pci_drive
probe's function where you end-up registering a switch the DSA. This
should solve portions of your problem although this is a long shot as we
need to convert existing drivers as well.

> 
> Can you think of a good example. I found a few ppc boards that have an
> PCI-ISA bridge that get's enumerated in the .dts. I think I can make sense
> of them but because they are bridges there's a lot of extra stuff for
> the down stream devices.

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Arnd Bergmann]

On Saturday 24 January 2015 08:38:12 Chris Packham wrote:
> 
> Can you think of a good example. I found a few ppc boards that have an 
> PCI-ISA bridge that get's enumerated in the .dts. I think I can make sense
> of them but because they are bridges there's a lot of extra stuff for the 
> down stream devices.

I found a copy of the standard document at:

http://www.o3one.org/hwdocs/openfirmware/pci_supplement_2_1.pdf

I believe the only property we require in order to connect a Linux PCI
device to a device node is 'reg', which is in a very special format
for PCI devices, as there are three cells that encode all sorts of
information. In this case, you only need the 'device' and 'function'
bits to match. The bus numbers can be reassigned on probing the
buses, and the structure is implied by the device/function numbers of
the parents all the way up to the host bridge (if you have multiple
bridges at all).

Something like

	pci@1230000 {
		device_type = "pci";
		ranges = ...
		#address-cells = <3>;
		#size-cells = <2>;

		dsa@01,0 { /* device 1, function 0 */
			reg = <0x00000800 0 0  0 0>;
		};
		dsa@01,1 { /* device 1, function 1 */
			reg = <0x00000900 0 0  0 0>;
		};
	};

The above follows the "DD,F" naming for unit strings in PCI devices
as in section 2.2.1.2 of that document. If you only have one function,
this can be shortened to just the device number, which is in bits
11-15 of the first word of the reg property.

Note that the binding is meant to encode all the probing information,
so it documents how to represent everything from the config space
in properties, but we don't actually use that for probing in Linux
and instead rely on a very reduced interpretation of the binding.

	Arnd
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Chris Packham]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3080 bytes --]

Hi All,

Back onto this from a brief hiatus

On 01/24/2015 11:40 AM, Florian Fainelli wrote:
> On 23/01/15 11:38, Chris Packham wrote:
>>
>> Hi Arnd,
>>
>> Thanks for the quick response (and sorry for the horribly formatted
>> message, travelling with only a webmail interface at the moment).
>>
>> On Fri, 23 Jan 2015, Arnd Bergmann wrote:
>>
>>> On Friday 23 January 2015 19:02:33 Chris Packham wrote:
>>>> Hi,
>>>>
>>>> I'm working on a new board that has a marvell,dsa switch. Similar to
>>>> the kirkwood-mv88f6281gtw-ge board. However the major difference is
>>>> that instead of being connected to an integrated Ethernet port it is
>>>> connected via a PCI-e Ethernet port (basically a NIC hardwired onto
>>>> the board).
>>>>
>>>> Generally you don't need to define PCI devices in the .dts because
>>>> the bus is scanned at run time and the method for identifying devices
>>>> is well defined. But to satisfy the marvell,dsa binding[1] I need to
>>>> tell it the Ethernet port it is connected to. The answer might be in
>>>> the PCI bindings but the urls in the documentation[2] don't appear to
>>>> be valid anymore.
>>>>
>>>> Is it possible to enumerate PCI devices in the .dts? Or is there a
>>>> way of satisfying the dsa requirements without knowing the Ethernet
>>>> device?
>>>>
>>>
>>> In general, it is possible, and we do that on PowerPC, but it may be that
>>> there are parts missing on ARM and you have to try it out.
>>>
>>> In particular, you need to know the address of the PCI device and then
>>> create a device node that will be associated with the
>>> pci_dev->dev.of_node
>>> pointer.
>>
>> Encouraging to know. I think I can figure out the relevant addresses for
>> my device based on the run-time scan.

With Arnd's help I've figured out the appropriate reg property for my 
PCI based Ethernet nodes.

	pcie-controller {
		pcie@1,0 {
			ethernet0: tg3@01,0 {
				reg = <0x00000000 0 0 0 0>;
			};
			ethernet1: tg3@01,1 {
				reg = <0x00000100 0 0 0 0>;
			};
		};
	};

>
> BTW, one thing that is high in my TODO list regarding DSA is to stop
> using this platform device/driver architecture and use the actual device
> driver model. Your switch driver entry point would be a pci_drive
> probe's function where you end-up registering a switch the DSA. This
> should solve portions of your problem although this is a long shot as we
> need to convert existing drivers as well.
>

My current problem is that of_find_device_by_node() (called by 
dsa_of_probe) can't find the device. After a bit of debugging I can see 
that the populated platform bus only goes as far as 
/soc/pcie-controller, anything beyond that seems to be missing. I can't 
see anything obvious in of_platform_bus_create() that would actually 
stop it from continuing down the pcie-controller branch provided the dts 
actually enumerates the children.

Any suggestions?N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·zøœzÚÞz)í…æèw*\x1fjg¬±¨\x1e¶‰šŽŠÝ¢j.ïÛ°\½½MŽúgjÌæa×\x02››–' ™©Þ¢¸\f¢·¦j:+v‰¨ŠwèjØm¶Ÿÿ¾\a«‘êçzZ+ƒùšŽŠÝ¢j"ú!¶i

Re: Associating nodes with phandles for pci devices

[Arnd Bergmann]

On Sunday 08 March 2015 22:01:17 Chris Packham wrote:
> 
> >
> > BTW, one thing that is high in my TODO list regarding DSA is to stop
> > using this platform device/driver architecture and use the actual device
> > driver model. Your switch driver entry point would be a pci_drive
> > probe's function where you end-up registering a switch the DSA. This
> > should solve portions of your problem although this is a long shot as we
> > need to convert existing drivers as well.
> >
> 
> My current problem is that of_find_device_by_node() (called by 
> dsa_of_probe) can't find the device. After a bit of debugging I can see 
> that the populated platform bus only goes as far as 
> /soc/pcie-controller, anything beyond that seems to be missing. I can't 
> see anything obvious in of_platform_bus_create() that would actually 
> stop it from continuing down the pcie-controller branch provided the dts 
> actually enumerates the children.
> 
> Any suggestions?

The PCI devices are not populated by of_platform_bus_create. Instead,
pci_scan_device() calls pci_set_of_node() to set the of_node
pointer for each PCI device.

This would fail for instance if the root bus does not have a correct
of_node pointer.

	Arnd
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Chris Packham]

On 03/09/2015 11:42 AM, Arnd Bergmann wrote:
> On Sunday 08 March 2015 22:01:17 Chris Packham wrote:
>>
>>>
>>> BTW, one thing that is high in my TODO list regarding DSA is to stop
>>> using this platform device/driver architecture and use the actual device
>>> driver model. Your switch driver entry point would be a pci_drive
>>> probe's function where you end-up registering a switch the DSA. This
>>> should solve portions of your problem although this is a long shot as we
>>> need to convert existing drivers as well.
>>>
>>
>> My current problem is that of_find_device_by_node() (called by
>> dsa_of_probe) can't find the device. After a bit of debugging I can see
>> that the populated platform bus only goes as far as
>> /soc/pcie-controller, anything beyond that seems to be missing. I can't
>> see anything obvious in of_platform_bus_create() that would actually
>> stop it from continuing down the pcie-controller branch provided the dts
>> actually enumerates the children.
>>
>> Any suggestions?
>
> The PCI devices are not populated by of_platform_bus_create. Instead,
> pci_scan_device() calls pci_set_of_node() to set the of_node
> pointer for each PCI device.
>
> This would fail for instance if the root bus does not have a correct
> of_node pointer.
>

Hmm. I instrumented pci_set_of_node and I do see my 2 network devices.

devfn = 00000000
of_node = bfff3f90 /soc/pcie-controller/pcie@1,0/tg3@01,0

devfn = 00000001
of_node = bfff4118 /soc/pcie-controller/pcie@1,0/tg3@01,1

So at least that's happening. I guess the problem is that ultimately 
they are devices on the pci bus as opposed to the platform bus and 
of_find_device_by_node() only looks at the platform bus

dev = bus_find_device(&platform_bus_type, NULL, np, of_dev_node_match);

What I might have to do for now is get the DSA driver code to go looking 
on the pci bus if of_find_device_by_node() fails to find the named node. 
Florian mentioned making the DSA code use the device driver model which 
would help me but I'm kinda stuck with the kernel version I'm using for 
now at least.


--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Florian Fainelli]

2015-03-08 16:03 GMT-07:00 Chris Packham <Chris.Packham-6g8wRflRTwXFdCa3tKVlE6U/zSkkHjvu@public.gmane.org>:
>
>
> On 03/09/2015 11:42 AM, Arnd Bergmann wrote:
>> On Sunday 08 March 2015 22:01:17 Chris Packham wrote:
>>>
>>>>
>>>> BTW, one thing that is high in my TODO list regarding DSA is to stop
>>>> using this platform device/driver architecture and use the actual device
>>>> driver model. Your switch driver entry point would be a pci_drive
>>>> probe's function where you end-up registering a switch the DSA. This
>>>> should solve portions of your problem although this is a long shot as we
>>>> need to convert existing drivers as well.
>>>>
>>>
>>> My current problem is that of_find_device_by_node() (called by
>>> dsa_of_probe) can't find the device. After a bit of debugging I can see
>>> that the populated platform bus only goes as far as
>>> /soc/pcie-controller, anything beyond that seems to be missing. I can't
>>> see anything obvious in of_platform_bus_create() that would actually
>>> stop it from continuing down the pcie-controller branch provided the dts
>>> actually enumerates the children.
>>>
>>> Any suggestions?
>>
>> The PCI devices are not populated by of_platform_bus_create. Instead,
>> pci_scan_device() calls pci_set_of_node() to set the of_node
>> pointer for each PCI device.
>>
>> This would fail for instance if the root bus does not have a correct
>> of_node pointer.
>>
>
> Hmm. I instrumented pci_set_of_node and I do see my 2 network devices.
>
> devfn = 00000000
> of_node = bfff3f90 /soc/pcie-controller/pcie@1,0/tg3@01,0
>
> devfn = 00000001
> of_node = bfff4118 /soc/pcie-controller/pcie@1,0/tg3@01,1
>
> So at least that's happening. I guess the problem is that ultimately
> they are devices on the pci bus as opposed to the platform bus and
> of_find_device_by_node() only looks at the platform bus
>
> dev = bus_find_device(&platform_bus_type, NULL, np, of_dev_node_match);
>
> What I might have to do for now is get the DSA driver code to go looking
> on the pci bus if of_find_device_by_node() fails to find the named node.

Could you try these two patches:
https://github.com/ffainelli/linux/commit/f57b2891e6118f5769c9fe17ed07e32def4c7c74
https://github.com/ffainelli/linux/commit/1ccd7b38dddd667a6ef6afe1cc3e9bb8c5599b58

> Florian mentioned making the DSA code use the device driver model which
> would help me but I'm kinda stuck with the kernel version I'm using for
> now at least.

Right, this is still taking a bit more time than I expected, but the
two patches above remove the restriction for the network device to be
backed by a platform_device, so hopefully that can get you going. I
will CC you on the next round of patches so you can keep track of that
progress.
-- 
Florian
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Associating nodes with phandles for pci devices

[Chris Packham]

Hi Florian,

On 03/09/2015 04:40 PM, Florian Fainelli wrote:
> 2015-03-08 16:03 GMT-07:00 Chris Packham <Chris.Packham@alliedtelesis.co.nz>:
>>
>>
>> On 03/09/2015 11:42 AM, Arnd Bergmann wrote:
>>> On Sunday 08 March 2015 22:01:17 Chris Packham wrote:
>>>>
>>>>>
>>>>> BTW, one thing that is high in my TODO list regarding DSA is to stop
>>>>> using this platform device/driver architecture and use the actual device
>>>>> driver model. Your switch driver entry point would be a pci_drive
>>>>> probe's function where you end-up registering a switch the DSA. This
>>>>> should solve portions of your problem although this is a long shot as we
>>>>> need to convert existing drivers as well.
>>>>>
>>>>
>>>> My current problem is that of_find_device_by_node() (called by
>>>> dsa_of_probe) can't find the device. After a bit of debugging I can see
>>>> that the populated platform bus only goes as far as
>>>> /soc/pcie-controller, anything beyond that seems to be missing. I can't
>>>> see anything obvious in of_platform_bus_create() that would actually
>>>> stop it from continuing down the pcie-controller branch provided the dts
>>>> actually enumerates the children.
>>>>
>>>> Any suggestions?
>>>
>>> The PCI devices are not populated by of_platform_bus_create. Instead,
>>> pci_scan_device() calls pci_set_of_node() to set the of_node
>>> pointer for each PCI device.
>>>
>>> This would fail for instance if the root bus does not have a correct
>>> of_node pointer.
>>>
>>
>> Hmm. I instrumented pci_set_of_node and I do see my 2 network devices.
>>
>> devfn = 00000000
>> of_node = bfff3f90 /soc/pcie-controller/pcie@1,0/tg3@01,0
>>
>> devfn = 00000001
>> of_node = bfff4118 /soc/pcie-controller/pcie@1,0/tg3@01,1
>>
>> So at least that's happening. I guess the problem is that ultimately
>> they are devices on the pci bus as opposed to the platform bus and
>> of_find_device_by_node() only looks at the platform bus
>>
>> dev = bus_find_device(&platform_bus_type, NULL, np, of_dev_node_match);
>>
>> What I might have to do for now is get the DSA driver code to go looking
>> on the pci bus if of_find_device_by_node() fails to find the named node.
>
> Could you try these two patches:
> https://github.com/ffainelli/linux/commit/f57b2891e6118f5769c9fe17ed07e32def4c7c74
> https://github.com/ffainelli/linux/commit/1ccd7b38dddd667a6ef6afe1cc3e9bb8c5599b58

Yes they work for me. I had to play with the 2nd one a little due to 
being back on an older kernel.

In the meantime I did go ahead and implement my version of the code 
falling back to PCI but your version is much more generic.

Are either of these in the pipeline for inclusion in Linus' tree? 
(probably not the one that is 2 hrs old).

>
>> Florian mentioned making the DSA code use the device driver model which
>> would help me but I'm kinda stuck with the kernel version I'm using for
>> now at least.
>
> Right, this is still taking a bit more time than I expected, but the
> two patches above remove the restriction for the network device to be
> backed by a platform_device, so hopefully that can get you going. I
> will CC you on the next round of patches so you can keep track of that
> progress.
>

OK thanks. One of these days I'll get the board I'm working on 
upstreamed. Hopefully I can at least keep something minimally working 
against the tip of Linus' tree for testing.