Doubt on first access for PCIe device

Doubt on first access for PCIe device

[abhijit]

Hi All,

I was referring Linux code for PCIe enumeration and I have one doubt 
w.r.t to very first operation that must be done on device.

Currently while scanning for device, we directly read vendor ID from 
device. But PCIE base specification 
"CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2 
specifies that,

"Functions must capture the Bus and Device Numbers supplied with all 
Type 0 Configuration Write Requests completed by the Function and supply 
these numbers in the Bus and Device Number fields of the Requester ID 
for all Requests initiated by the Device/Function."

Here I am assuming, the completer ID will be device number and function 
number that will eventually programmed in to  device. In that case, my 
question is, without first write, how read request(VENDOR ID read) is 
serviced/routed?

Sorry I am feeling bit confused, help will be appreciated.
-- 
Best wishes,
Abhijit Naik

Re: Doubt on first access for PCIe device

[Jon Masters]

On 04/11/2017 10:15 AM, abhijit wrote:

> Here I am assuming, the completer ID will be device number and function 
> number that will eventually programmed in to  device. In that case, my 
> question is, without first write, how read request(VENDOR ID read) is 
> serviced/routed?

You'll want to read about PCIe enumeration at boot time and how the BIOS
walks the topology to assign these (which an OS may later re-number). In
particular, read about ECAM for configuration. This uses memory mapped
config space read/write accessors that target a memory address space
under the control of the root complex.

Jon.

Re: Doubt on first access for PCIe device

[Lorenzo Pieralisi]

On Tue, Apr 11, 2017 at 07:45:40PM +0530, abhijit wrote:
> 
> Hi All,
> 
> I was referring Linux code for PCIe enumeration and I have one doubt
> w.r.t to very first operation that must be done on device.
> 
> Currently while scanning for device, we directly read vendor ID from
> device. But PCIE base specification
> "CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2
> specifies that,
> 
> "Functions must capture the Bus and Device Numbers supplied with all
> Type 0 Configuration Write Requests completed by the Function and
> supply these numbers in the Bus and Device Number fields of the
> Requester ID for all Requests initiated by the Device/Function."

Please note:

"for all Requests initiated by the Device/Function".
                  ^^^^^^^^^

> Here I am assuming, the completer ID will be device number and
> function number that will eventually programmed in to  device. In
> that case, my question is, without first write, how read
> request(VENDOR ID read) is serviced/routed?

That's not initiated by the endpoint device, it is routed by
requester ID, which is the requester ID of the configuration
request - ie master (ie root complex) requester ID.

Read 2.2.9 - Completion rules

HTH,
Lorenzo

Re: Doubt on first access for PCIe device

[abhijit]

Hi Lorenzo,

Thank you very much for reply.

As you mentioned I noted that, "If a Function must generate a Completion 
prior to the initial device Configuration Write Request, 0’s must be 
entered into the Bus Number and Device Number fields"

By above statement this means that completion ID will be 0x0000. For the 
very first device on any bus,  device number will be 0 and function 
number will be 0.

Isn't completer (for configuration read) packet will be addressing 
itself instead of root complex with bus, device and function number 0?

Regards,
     Abhijit


On Tuesday 11 April 2017 07:45 PM, abhijit wrote:
>
> Hi All,
>
> I was referring Linux code for PCIe enumeration and I have one doubt
> w.r.t to very first operation that must be done on device.
>
> Currently while scanning for device, we directly read vendor ID from
> device. But PCIE base specification
> "CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2
> specifies that,
>
> "Functions must capture the Bus and Device Numbers supplied with all
> Type 0 Configuration Write Requests completed by the Function and supply
> these numbers in the Bus and Device Number fields of the Requester ID
> for all Requests initiated by the Device/Function."
>
> Here I am assuming, the completer ID will be device number and function
> number that will eventually programmed in to  device. In that case, my
> question is, without first write, how read request(VENDOR ID read) is
> serviced/routed?
>
> Sorry I am feeling bit confused, help will be appreciated.

Re: Doubt on first access for PCIe device

[Lorenzo Pieralisi]

On Fri, Apr 21, 2017 at 03:33:45PM +0530, abhijit naik wrote:
>    Hi Lorenzo,
>    Thank you very much for reply.
>    As you mentioned I noted that, "If a Function must generate a Completion
>    prior to the initial device Configuration Write Request, 0's must be
>    entered into the Bus Number and Device Number fields"
>    By above statement this means that completion ID will be 0x0000. For the
>    first device on any bus,  device number will be 0 and function number will
>    be 0.
>    Isn't completer (for configuration read) packet will be addressing itself
>    instead of root complex with bus, device and function number 0?

I don't understand what you are asking. The completion (containing
the read data) is routed back to the initiator via ID (of the requester)
routing.

2.2.4.2
2.2.9

Lorenzo

>    On Wed, Apr 19, 2017 at 3:56 PM, Lorenzo Pieralisi
>    <lorenzo.pieralisi@arm.com> wrote:
> 
>      On Tue, Apr 11, 2017 at 07:45:40PM +0530, abhijit wrote:
>      >
>      > Hi All,
>      >
>      > I was referring Linux code for PCIe enumeration and I have one doubt
>      > w.r.t to very first operation that must be done on device.
>      >
>      > Currently while scanning for device, we directly read vendor ID from
>      > device. But PCIE base specification
>      > "CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2
>      > specifies that,
>      >
>      > "Functions must capture the Bus and Device Numbers supplied with all
>      > Type 0 Configuration Write Requests completed by the Function and
>      > supply these numbers in the Bus and Device Number fields of the
>      > Requester ID for all Requests initiated by the Device/Function."
> 
>      Please note:
>      "for all Requests initiated by the Device/Function".
>                        ^^^^^^^^^
>      > Here I am assuming, the completer ID will be device number and
>      > function number that will eventually programmed in to  device. In
>      > that case, my question is, without first write, how read
>      > request(VENDOR ID read) is serviced/routed?
> 
>      That's not initiated by the endpoint device, it is routed by
>      requester ID, which is the requester ID of the configuration
>      request - ie master (ie root complex) requester ID.
> 
>      Read 2.2.9 - Completion rules
> 
>      HTH,
>      Lorenzo
> 
>    --
>    Best wishes,
>    Abhijit Naik

Re: Doubt on first access for PCIe device

[abhijit]

Hi Lorenzo,

Yes that's true..

My question is: what if EP function default completer ID means BDF has 
values = 0(meaning B = 0, D = 0, f = 0) and Root complex requester ID 
also has same BDF, that is 0.

Is that valid case as per PCIe standard, as in this case, requester ID 
and completion ID have same BDF?

Thank you very much for your replies

Regards,
     Abhijit


On Friday 21 April 2017 09:29 PM, Lorenzo Pieralisi wrote:
> On Fri, Apr 21, 2017 at 03:33:45PM +0530, abhijit naik wrote:
>>    Hi Lorenzo,
>>    Thank you very much for reply.
>>    As you mentioned I noted that, "If a Function must generate a Completion
>>    prior to the initial device Configuration Write Request, 0's must be
>>    entered into the Bus Number and Device Number fields"
>>    By above statement this means that completion ID will be 0x0000. For the
>>    first device on any bus,  device number will be 0 and function number will
>>    be 0.
>>    Isn't completer (for configuration read) packet will be addressing itself
>>    instead of root complex with bus, device and function number 0?
>
> I don't understand what you are asking. The completion (containing
> the read data) is routed back to the initiator via ID (of the requester)
> routing.
>
> 2.2.4.2
> 2.2.9
>
> Lorenzo
>
>>    On Wed, Apr 19, 2017 at 3:56 PM, Lorenzo Pieralisi
>>    <lorenzo.pieralisi@arm.com> wrote:
>>
>>      On Tue, Apr 11, 2017 at 07:45:40PM +0530, abhijit wrote:
>>      >
>>      > Hi All,
>>      >
>>      > I was referring Linux code for PCIe enumeration and I have one doubt
>>      > w.r.t to very first operation that must be done on device.
>>      >
>>      > Currently while scanning for device, we directly read vendor ID from
>>      > device. But PCIE base specification
>>      > "CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2
>>      > specifies that,
>>      >
>>      > "Functions must capture the Bus and Device Numbers supplied with all
>>      > Type 0 Configuration Write Requests completed by the Function and
>>      > supply these numbers in the Bus and Device Number fields of the
>>      > Requester ID for all Requests initiated by the Device/Function."
>>
>>      Please note:
>>      "for all Requests initiated by the Device/Function".
>>                        ^^^^^^^^^
>>      > Here I am assuming, the completer ID will be device number and
>>      > function number that will eventually programmed in to  device. In
>>      > that case, my question is, without first write, how read
>>      > request(VENDOR ID read) is serviced/routed?
>>
>>      That's not initiated by the endpoint device, it is routed by
>>      requester ID, which is the requester ID of the configuration
>>      request - ie master (ie root complex) requester ID.
>>
>>      Read 2.2.9 - Completion rules
>>
>>      HTH,
>>      Lorenzo
>>
>>    --
>>    Best wishes,
>>    Abhijit Naik

Re: Doubt on first access for PCIe device

[Lorenzo Pieralisi]

On Mon, Apr 24, 2017 at 11:52:27AM +0530, abhijit wrote:
> Hi Lorenzo,
> 
> Yes that's true..
> 
> My question is: what if EP function default completer ID means BDF
> has values = 0(meaning B = 0, D = 0, f = 0) and Root complex
> requester ID also has same BDF, that is 0.
> 
> Is that valid case as per PCIe standard, as in this case, requester
> ID and completion ID have same BDF?

Read "2.2.9: Completion rules" and you will have your question answered
there ;-)

Lorenzo

> 
> Thank you very much for your replies
> 
> Regards,
>     Abhijit
> 
> 
> On Friday 21 April 2017 09:29 PM, Lorenzo Pieralisi wrote:
> >On Fri, Apr 21, 2017 at 03:33:45PM +0530, abhijit naik wrote:
> >>   Hi Lorenzo,
> >>   Thank you very much for reply.
> >>   As you mentioned I noted that, "If a Function must generate a Completion
> >>   prior to the initial device Configuration Write Request, 0's must be
> >>   entered into the Bus Number and Device Number fields"
> >>   By above statement this means that completion ID will be 0x0000. For the
> >>   first device on any bus,  device number will be 0 and function number will
> >>   be 0.
> >>   Isn't completer (for configuration read) packet will be addressing itself
> >>   instead of root complex with bus, device and function number 0?
> >
> >I don't understand what you are asking. The completion (containing
> >the read data) is routed back to the initiator via ID (of the requester)
> >routing.
> >
> >2.2.4.2
> >2.2.9
> >
> >Lorenzo
> >
> >>   On Wed, Apr 19, 2017 at 3:56 PM, Lorenzo Pieralisi
> >>   <lorenzo.pieralisi@arm.com> wrote:
> >>
> >>     On Tue, Apr 11, 2017 at 07:45:40PM +0530, abhijit wrote:
> >>     >
> >>     > Hi All,
> >>     >
> >>     > I was referring Linux code for PCIe enumeration and I have one doubt
> >>     > w.r.t to very first operation that must be done on device.
> >>     >
> >>     > Currently while scanning for device, we directly read vendor ID from
> >>     > device. But PCIE base specification
> >>     > "CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2
> >>     > specifies that,
> >>     >
> >>     > "Functions must capture the Bus and Device Numbers supplied with all
> >>     > Type 0 Configuration Write Requests completed by the Function and
> >>     > supply these numbers in the Bus and Device Number fields of the
> >>     > Requester ID for all Requests initiated by the Device/Function."
> >>
> >>     Please note:
> >>     "for all Requests initiated by the Device/Function".
> >>                       ^^^^^^^^^
> >>     > Here I am assuming, the completer ID will be device number and
> >>     > function number that will eventually programmed in to  device. In
> >>     > that case, my question is, without first write, how read
> >>     > request(VENDOR ID read) is serviced/routed?
> >>
> >>     That's not initiated by the endpoint device, it is routed by
> >>     requester ID, which is the requester ID of the configuration
> >>     request - ie master (ie root complex) requester ID.
> >>
> >>     Read 2.2.9 - Completion rules
> >>
> >>     HTH,
> >>     Lorenzo
> >>
> >>   --
> >>   Best wishes,
> >>   Abhijit Naik

Re: Doubt on first access for PCIe device

[abhijit]

Hi Lorenzo,

I think, you are referring to statement,

"The Completion ID field is not meaningful prior to the software 
initialization and configuration of the completing device (using at 
least one Configuration Write Request), and for this case the
Requester must ignore the value returned in the Completer ID field."

Actually one of experimental Endpoint function IP that we are using is 
not responding for configuration request if Requester ID and Completion 
ID are same even during configuration phase. So wanted to cross-check 
whether we are missing out something.

It seems that behavior of endpoint IP is not correct.

Thank you very much for all your time and patience :)

Regards,
     Abhijit

On Monday 24 April 2017 03:31 PM, Lorenzo Pieralisi wrote:
> On Mon, Apr 24, 2017 at 11:52:27AM +0530, abhijit wrote:
>> Hi Lorenzo,
>>
>> Yes that's true..
>>
>> My question is: what if EP function default completer ID means BDF
>> has values = 0(meaning B = 0, D = 0, f = 0) and Root complex
>> requester ID also has same BDF, that is 0.
>>
>> Is that valid case as per PCIe standard, as in this case, requester
>> ID and completion ID have same BDF?
>
> Read "2.2.9: Completion rules" and you will have your question answered
> there ;-)
>
> Lorenzo
>
>>
>> Thank you very much for your replies
>>
>> Regards,
>>     Abhijit
>>
>>
>> On Friday 21 April 2017 09:29 PM, Lorenzo Pieralisi wrote:
>>> On Fri, Apr 21, 2017 at 03:33:45PM +0530, abhijit naik wrote:
>>>>   Hi Lorenzo,
>>>>   Thank you very much for reply.
>>>>   As you mentioned I noted that, "If a Function must generate a Completion
>>>>   prior to the initial device Configuration Write Request, 0's must be
>>>>   entered into the Bus Number and Device Number fields"
>>>>   By above statement this means that completion ID will be 0x0000. For the
>>>>   first device on any bus,  device number will be 0 and function number will
>>>>   be 0.
>>>>   Isn't completer (for configuration read) packet will be addressing itself
>>>>   instead of root complex with bus, device and function number 0?
>>>
>>> I don't understand what you are asking. The completion (containing
>>> the read data) is routed back to the initiator via ID (of the requester)
>>> routing.
>>>
>>> 2.2.4.2
>>> 2.2.9
>>>
>>> Lorenzo
>>>
>>>>   On Wed, Apr 19, 2017 at 3:56 PM, Lorenzo Pieralisi
>>>>   <lorenzo.pieralisi@arm.com> wrote:
>>>>
>>>>     On Tue, Apr 11, 2017 at 07:45:40PM +0530, abhijit wrote:
>>>>     >
>>>>     > Hi All,
>>>>     >
>>>>     > I was referring Linux code for PCIe enumeration and I have one doubt
>>>>     > w.r.t to very first operation that must be done on device.
>>>>     >
>>>>     > Currently while scanning for device, we directly read vendor ID from
>>>>     > device. But PCIE base specification
>>>>     > "CB-PCI_Express_Base_4.0r0.7_November-11-2016.pdf" section 2.2.6.2
>>>>     > specifies that,
>>>>     >
>>>>     > "Functions must capture the Bus and Device Numbers supplied with all
>>>>     > Type 0 Configuration Write Requests completed by the Function and
>>>>     > supply these numbers in the Bus and Device Number fields of the
>>>>     > Requester ID for all Requests initiated by the Device/Function."
>>>>
>>>>     Please note:
>>>>     "for all Requests initiated by the Device/Function".
>>>>                       ^^^^^^^^^
>>>>     > Here I am assuming, the completer ID will be device number and
>>>>     > function number that will eventually programmed in to  device. In
>>>>     > that case, my question is, without first write, how read
>>>>     > request(VENDOR ID read) is serviced/routed?
>>>>
>>>>     That's not initiated by the endpoint device, it is routed by
>>>>     requester ID, which is the requester ID of the configuration
>>>>     request - ie master (ie root complex) requester ID.
>>>>
>>>>     Read 2.2.9 - Completion rules
>>>>
>>>>     HTH,
>>>>     Lorenzo
>>>>
>>>>   --
>>>>   Best wishes,
>>>>   Abhijit Naik