[Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Heyi Guo]

Hi folks,

In physical world, a PCIe switch including one upstream port and several downstream ports is a single physical device, however we treat each port as a device in qemu world. In qemu docs/pcie.txt, we have below statements:

Line 230: Be aware that PCI Express Downstream Ports can't be hot-plugged into
Line 231: an existing PCI Express Upstream Port.

To my understanding, it implies PCIe downstream ports *can* be hot-plugged into something which is not an existing upstream port. If it is true, how can we do that? AFAIK monitor command device_add can only add one device at a time.

Please help to show the truth.

Thanks,

Heyi

Re: [Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Michael S. Tsirkin]

On Wed, Apr 03, 2019 at 03:32:09PM +0800, Heyi Guo wrote:
> Hi folks,
> 
> In physical world, a PCIe switch including one upstream port and several downstream ports is a single physical device, however we treat each port as a device in qemu world. In qemu docs/pcie.txt, we have below statements:
> 
> Line 230: Be aware that PCI Express Downstream Ports can't be hot-plugged into
> Line 231: an existing PCI Express Upstream Port.
> 
> To my understanding, it implies PCIe downstream ports *can* be hot-plugged into something which is not an existing upstream port. If it is true, how can we do that? AFAIK monitor command device_add can only add one device at a time.
> 
> Please help to show the truth.
> 
> Thanks,
> 
> Heyi
> 

afaik they can only be plugged into upstearm ports with or without hotplug.

-- 
mst

Re: [Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Auger Eric]

Hi Heyi,

On 4/3/19 8:50 PM, Michael S. Tsirkin wrote:
> On Wed, Apr 03, 2019 at 03:32:09PM +0800, Heyi Guo wrote:
>> Hi folks,
>>
>> In physical world, a PCIe switch including one upstream port and several downstream ports is a single physical device, however we treat each port as a device in qemu world. In qemu docs/pcie.txt, we have below statements:
>>
>> Line 230: Be aware that PCI Express Downstream Ports can't be hot-plugged into
>> Line 231: an existing PCI Express Upstream Port.
>>
>> To my understanding, it implies PCIe downstream ports *can* be hot-plugged into something which is not an existing upstream port. If it is true, how can we do that? AFAIK monitor command device_add can only add one device at a time.
>>
>> Please help to show the truth.
>>
>> Thanks,
>>
>> Heyi
>>
> 
> afaik they can only be plugged into upstearm ports with or without hotplug.

Hotplug on upstream port does not look supported, as mentionned in the doc:
(QEMU) device_add driver=xio3130-downstream id=down0 bus=upstream_port1
{"error": {"class": "GenericError", "desc": "Bus 'upstream_port1' does
not support hotplugging"}}

Looks the std way to use the downstream port is the one documented in 2.2.3:

2.2.3 Plugging a PCI Express device into a Switch:
-device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z]
-device x3130-upstream,id=upstream_port1,bus=root_port1[,addr=x]
-device
xio3130-downstream,id=downstream_port1,bus=upstream_port1,chassis=x1,slot=y1[,addr=z1]]
-device <dev>,bus=downstream_port1

For my curiosity why do you want to hotplug a downstream port in another
place than an upstream port?

Thanks

Eric


> 

Re: [Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Heyi Guo]

Hi Eric,

My real interesting is about the hotplug of PCIe switch, which means we don't need to provide lots of PCIe root ports or PCIe down stream ports at the beginning, but we can extend the capacity by hot adding PCIe switches which can provide more hot-pluggable slots for endpoint devices.

The document docs/pcie.txt says "PCI Express Downstream Ports can't be hot-plugged into an existing PCI Express Upstream Port" which confuses me. Does it actually mean Downstream Ports can't be hot-plugged? For they can't be hot-plugged into an existing Upstream Port as the doc says, either they can't be hot-plugged into an non-existing Upstream Port or another place...

Thanks,

Heyi


On 2019/4/4 15:39, Auger Eric wrote:
> Hi Heyi,
>
> On 4/3/19 8:50 PM, Michael S. Tsirkin wrote:
>> On Wed, Apr 03, 2019 at 03:32:09PM +0800, Heyi Guo wrote:
>>> Hi folks,
>>>
>>> In physical world, a PCIe switch including one upstream port and several downstream ports is a single physical device, however we treat each port as a device in qemu world. In qemu docs/pcie.txt, we have below statements:
>>>
>>> Line 230: Be aware that PCI Express Downstream Ports can't be hot-plugged into
>>> Line 231: an existing PCI Express Upstream Port.
>>>
>>> To my understanding, it implies PCIe downstream ports *can* be hot-plugged into something which is not an existing upstream port. If it is true, how can we do that? AFAIK monitor command device_add can only add one device at a time.
>>>
>>> Please help to show the truth.
>>>
>>> Thanks,
>>>
>>> Heyi
>>>
>> afaik they can only be plugged into upstearm ports with or without hotplug.
> Hotplug on upstream port does not look supported, as mentionned in the doc:
> (QEMU) device_add driver=xio3130-downstream id=down0 bus=upstream_port1
> {"error": {"class": "GenericError", "desc": "Bus 'upstream_port1' does
> not support hotplugging"}}
>
> Looks the std way to use the downstream port is the one documented in 2.2.3:
>
> 2.2.3 Plugging a PCI Express device into a Switch:
> -device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z]
> -device x3130-upstream,id=upstream_port1,bus=root_port1[,addr=x]
> -device
> xio3130-downstream,id=downstream_port1,bus=upstream_port1,chassis=x1,slot=y1[,addr=z1]]
> -device <dev>,bus=downstream_port1
>
> For my curiosity why do you want to hotplug a downstream port in another
> place than an upstream port?
>
> Thanks
>
> Eric
>
>
> .
>

Re: [Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Heyi Guo]

Hi Eric,

Could you help to confirm?

Thanks,

Heyi


On 2019/4/7 9:59, Heyi Guo wrote:
> Hi Eric,
>
> My real interesting is about the hotplug of PCIe switch, which means we don't need to provide lots of PCIe root ports or PCIe down stream ports at the beginning, but we can extend the capacity by hot adding PCIe switches which can provide more hot-pluggable slots for endpoint devices.
>
> The document docs/pcie.txt says "PCI Express Downstream Ports can't be hot-plugged into an existing PCI Express Upstream Port" which confuses me. Does it actually mean Downstream Ports can't be hot-plugged? For they can't be hot-plugged into an existing Upstream Port as the doc says, either they can't be hot-plugged into an non-existing Upstream Port or another place...
>
> Thanks,
>
> Heyi
>
>
> On 2019/4/4 15:39, Auger Eric wrote:
>> Hi Heyi,
>>
>> On 4/3/19 8:50 PM, Michael S. Tsirkin wrote:
>>> On Wed, Apr 03, 2019 at 03:32:09PM +0800, Heyi Guo wrote:
>>>> Hi folks,
>>>>
>>>> In physical world, a PCIe switch including one upstream port and several downstream ports is a single physical device, however we treat each port as a device in qemu world. In qemu docs/pcie.txt, we have below statements:
>>>>
>>>> Line 230: Be aware that PCI Express Downstream Ports can't be hot-plugged into
>>>> Line 231: an existing PCI Express Upstream Port.
>>>>
>>>> To my understanding, it implies PCIe downstream ports *can* be hot-plugged into something which is not an existing upstream port. If it is true, how can we do that? AFAIK monitor command device_add can only add one device at a time.
>>>>
>>>> Please help to show the truth.
>>>>
>>>> Thanks,
>>>>
>>>> Heyi
>>>>
>>> afaik they can only be plugged into upstearm ports with or without hotplug.
>> Hotplug on upstream port does not look supported, as mentionned in the doc:
>> (QEMU) device_add driver=xio3130-downstream id=down0 bus=upstream_port1
>> {"error": {"class": "GenericError", "desc": "Bus 'upstream_port1' does
>> not support hotplugging"}}
>>
>> Looks the std way to use the downstream port is the one documented in 2.2.3:
>>
>> 2.2.3 Plugging a PCI Express device into a Switch:
>> -device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z]
>> -device x3130-upstream,id=upstream_port1,bus=root_port1[,addr=x]
>> -device
>> xio3130-downstream,id=downstream_port1,bus=upstream_port1,chassis=x1,slot=y1[,addr=z1]]
>> -device <dev>,bus=downstream_port1
>>
>> For my curiosity why do you want to hotplug a downstream port in another
>> place than an upstream port?
>>
>> Thanks
>>
>> Eric
>>
>>
>> .
>>
>

Re: [Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Auger Eric]

Hi Heyi,

On 4/11/19 1:30 PM, Heyi Guo wrote:
> Hi Eric,
> 
> Could you help to confirm?

Practically I have not tried anything else than hot-plugging the
downstream port downstream to the upstream port. This looks not
supported as documented.

Michael answered that up to him, the downstream port could only be
plugged downto the upstream port.

On my side I have not aware of any usage of the downstream port outside
of the switch's scope.

Thanks

Eric
> 
> Thanks,
> 
> Heyi
> 
> 
> On 2019/4/7 9:59, Heyi Guo wrote:
>> Hi Eric,
>>
>> My real interesting is about the hotplug of PCIe switch, which means
>> we don't need to provide lots of PCIe root ports or PCIe down stream
>> ports at the beginning, but we can extend the capacity by hot adding
>> PCIe switches which can provide more hot-pluggable slots for endpoint
>> devices.
>>
>> The document docs/pcie.txt says "PCI Express Downstream Ports can't be
>> hot-plugged into an existing PCI Express Upstream Port" which confuses
>> me. Does it actually mean Downstream Ports can't be hot-plugged? For
>> they can't be hot-plugged into an existing Upstream Port as the doc
>> says, either they can't be hot-plugged into an non-existing Upstream
>> Port or another place...
>>
>> Thanks,
>>
>> Heyi
>>
>>
>> On 2019/4/4 15:39, Auger Eric wrote:
>>> Hi Heyi,
>>>
>>> On 4/3/19 8:50 PM, Michael S. Tsirkin wrote:
>>>> On Wed, Apr 03, 2019 at 03:32:09PM +0800, Heyi Guo wrote:
>>>>> Hi folks,
>>>>>
>>>>> In physical world, a PCIe switch including one upstream port and
>>>>> several downstream ports is a single physical device, however we
>>>>> treat each port as a device in qemu world. In qemu docs/pcie.txt,
>>>>> we have below statements:
>>>>>
>>>>> Line 230: Be aware that PCI Express Downstream Ports can't be
>>>>> hot-plugged into
>>>>> Line 231: an existing PCI Express Upstream Port.
>>>>>
>>>>> To my understanding, it implies PCIe downstream ports *can* be
>>>>> hot-plugged into something which is not an existing upstream port.
>>>>> If it is true, how can we do that? AFAIK monitor command device_add
>>>>> can only add one device at a time.
>>>>>
>>>>> Please help to show the truth.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Heyi
>>>>>
>>>> afaik they can only be plugged into upstearm ports with or without
>>>> hotplug.
>>> Hotplug on upstream port does not look supported, as mentionned in
>>> the doc:
>>> (QEMU) device_add driver=xio3130-downstream id=down0 bus=upstream_port1
>>> {"error": {"class": "GenericError", "desc": "Bus 'upstream_port1' does
>>> not support hotplugging"}}
>>>
>>> Looks the std way to use the downstream port is the one documented in
>>> 2.2.3:
>>>
>>> 2.2.3 Plugging a PCI Express device into a Switch:
>>> -device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z]
>>> -device x3130-upstream,id=upstream_port1,bus=root_port1[,addr=x]
>>> -device
>>> xio3130-downstream,id=downstream_port1,bus=upstream_port1,chassis=x1,slot=y1[,addr=z1]]
>>>
>>> -device <dev>,bus=downstream_port1
>>>
>>> For my curiosity why do you want to hotplug a downstream port in another
>>> place than an upstream port?
>>>
>>> Thanks
>>>
>>> Eric
>>>
>>>
>>> .
>>>
>>
> 
> 

Re: [Qemu-devel] Question: can we hot plug a PCIe switch on machine "virt"

[Heyi Guo]

Hi Eric and Michael,

Appreciate your sharing of this information. But sorry I'm still a little confused due to my poor English...


On 2019/4/11 20:19, Auger Eric wrote:
> Hi Heyi,
>
> On 4/11/19 1:30 PM, Heyi Guo wrote:
>> Hi Eric,
>>
>> Could you help to confirm?
> Practically I have not tried anything else than hot-plugging the
> downstream port downstream to the upstream port. This looks not
> supported as documented.
Do you mean you *have* tried hot-plugging the downstream port to the upstream port? Have you succeeded? Could you tell how to do that?

>
> Michael answered that up to him, the downstream port could only be
> plugged downto the upstream port.
Is only code-plugging supported here, or hot-plugging is supported as well?

Thanks,
Heyi

>
> On my side I have not aware of any usage of the downstream port outside
> of the switch's scope.
>
> Thanks
>
> Eric
>> Thanks,
>>
>> Heyi
>>
>>
>> On 2019/4/7 9:59, Heyi Guo wrote:
>>> Hi Eric,
>>>
>>> My real interesting is about the hotplug of PCIe switch, which means
>>> we don't need to provide lots of PCIe root ports or PCIe down stream
>>> ports at the beginning, but we can extend the capacity by hot adding
>>> PCIe switches which can provide more hot-pluggable slots for endpoint
>>> devices.
>>>
>>> The document docs/pcie.txt says "PCI Express Downstream Ports can't be
>>> hot-plugged into an existing PCI Express Upstream Port" which confuses
>>> me. Does it actually mean Downstream Ports can't be hot-plugged? For
>>> they can't be hot-plugged into an existing Upstream Port as the doc
>>> says, either they can't be hot-plugged into an non-existing Upstream
>>> Port or another place...
>>>
>>> Thanks,
>>>
>>> Heyi
>>>
>>>
>>> On 2019/4/4 15:39, Auger Eric wrote:
>>>> Hi Heyi,
>>>>
>>>> On 4/3/19 8:50 PM, Michael S. Tsirkin wrote:
>>>>> On Wed, Apr 03, 2019 at 03:32:09PM +0800, Heyi Guo wrote:
>>>>>> Hi folks,
>>>>>>
>>>>>> In physical world, a PCIe switch including one upstream port and
>>>>>> several downstream ports is a single physical device, however we
>>>>>> treat each port as a device in qemu world. In qemu docs/pcie.txt,
>>>>>> we have below statements:
>>>>>>
>>>>>> Line 230: Be aware that PCI Express Downstream Ports can't be
>>>>>> hot-plugged into
>>>>>> Line 231: an existing PCI Express Upstream Port.
>>>>>>
>>>>>> To my understanding, it implies PCIe downstream ports *can* be
>>>>>> hot-plugged into something which is not an existing upstream port.
>>>>>> If it is true, how can we do that? AFAIK monitor command device_add
>>>>>> can only add one device at a time.
>>>>>>
>>>>>> Please help to show the truth.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Heyi
>>>>>>
>>>>> afaik they can only be plugged into upstearm ports with or without
>>>>> hotplug.
>>>> Hotplug on upstream port does not look supported, as mentionned in
>>>> the doc:
>>>> (QEMU) device_add driver=xio3130-downstream id=down0 bus=upstream_port1
>>>> {"error": {"class": "GenericError", "desc": "Bus 'upstream_port1' does
>>>> not support hotplugging"}}
>>>>
>>>> Looks the std way to use the downstream port is the one documented in
>>>> 2.2.3:
>>>>
>>>> 2.2.3 Plugging a PCI Express device into a Switch:
>>>> -device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z]
>>>> -device x3130-upstream,id=upstream_port1,bus=root_port1[,addr=x]
>>>> -device
>>>> xio3130-downstream,id=downstream_port1,bus=upstream_port1,chassis=x1,slot=y1[,addr=z1]]
>>>>
>>>> -device <dev>,bus=downstream_port1
>>>>
>>>> For my curiosity why do you want to hotplug a downstream port in another
>>>> place than an upstream port?
>>>>
>>>> Thanks
>>>>
>>>> Eric
>>>>
>>>>
>>>> .
>>>>
>>
> .
>