* [virtio-dev] [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
@ 2023-06-13 17:30 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
This short series introduces legacy registers access commands for the owner
group member PCI PF to access the legacy registers of the member VFs.
If in future any SIOV devices to support legacy registers, they
can be easily supported using same commands by using the group
member identifiers of the future SIOV devices.
More details as overview, motivation, use case are further described
below.
Patch summary:
--------------
patch-1 split rows of admin opcode tables by a line
patch-2 adds administrative virtuqueue commands
patch-3 adds its conformance section and links
It uses the newly introduced administrative virtqueue facility with 4 new
commands which uses the existing virtio_admin_cmd.
Usecase:
--------
1. A hypervisor/system needs to provide transitional
virtio devices to the guest VM at scale of thousands,
typically, one to eight devices per VM.
2. A hypervisor/system needs to provide such devices using a
vendor agnostic driver in the hypervisor system.
3. A hypervisor system prefers to have single stack regardless of
virtio device type (net/blk) and be future compatible with a
single vfio stack using SR-IOV or other scalable device
virtualization technology to map PCI devices to the guest VM.
(as transitional or otherwise)
Motivation/Background:
----------------------
The existing virtio transitional PCI device is missing support for
PCI SR-IOV based devices. Currently it does not work beyond
PCI PF, or as software emulated device in reality. Currently it
has below cited system level limitations:
[a] PCIe spec citation:
VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
[b] cpu arch citiation:
Intel 64 and IA-32 Architectures Software Developer’s Manual:
The processor’s I/O address space is separate and distinct from
the physical-memory address space. The I/O address space consists
of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
[c] PCIe spec citation:
If a bridge implements an I/O address range,...I/O address range will be
aligned to a 4 KB boundary.
Overview:
---------
Above usecase requirements is solved by PCI PF group owner accessing
its group member PCI VFs legacy registers using an admin virtqueue of
the group owner PCI PF.
Two new admin virtqueue commands are added which read/write PCI VF
registers.
Software usage example:
-----------------------
One way to use and map to the guest VM is by using vfio driver
framework in Linux kernel.
+----------------------+
|pci_dev_id = 0x100X |
+---------------|pci_rev_id = 0x0 |-----+
|vfio device |BAR0 = I/O region | |
| |Other attributes | |
| +----------------------+ |
| |
+ +--------------+ +-----------------+ |
| |I/O BAR to AQ | | Other vfio | |
| |rd/wr mapper | | functionalities | |
| +--------------+ +-----------------+ |
| |
+------+-------------------------+-----------+
| |
| Driver notification
| |
| |
+----+------------+ +----+------------+
| +-----+ | | PCI VF device A |
| | AQ |-------------+---->+-------------+ |
| +-----+ | | | | legacy regs | |
| PCI PF device | | | +-------------+ |
+-----------------+ | +-----------------+
|
| +----+------------+
| | PCI VF device N |
+---->+-------------+ |
| | legacy regs | |
| +-------------+ |
+-----------------+
2. Virtio pci driver to bind to the listed device id and
use it in the host.
3. Use it in a light weight hypervisor to run bare-metal OS.
Please review.
Alternatives considered:
========================
1. Exposing BAR0 as MMIO BAR that follows legacy registers template
Pros:
a. Kind of works with legacy drivers as some of them have used API
which is agnostic to MMIO vs IOBAR.
b. Does not require hypervisor intervantion
Cons:
a. Device reset is extremely hard to implement in device at scale as
driver does not wait for device reset completion
b. Device register width related problems persist that hypervisor if
wishes, it cannot be fixed.
2. Accessing VF registers by tunneling it through new legacy PCI capability
Pros:
a. Self contained, but cannot work with future PCI SIOV devices
Cons:
a. Equally slow as AQ access
b. Still requires new capability for notification access
c. Requires hardware to build low level registers access which is not worth
for long term future
3. Accessing VF notification region using new PF BAR
Cons:
a. Requires hardware to build new PCI steering logic per PF to forward
notification from the PF to VF, requires double the amount of logic
compared to today
b. Requires very large additional PF BAR whose size must be max_Vfs * BAR size.
4. Trapping CVQ, configuration region, LEGACY_HDR
Cons:
a. This does not fullfil the very basic requirement to not trap the
1.x objects (configuration registers, vqs)
b. Requires feature negotiations mediation in hypervisor software
c. Requires constant device type specific knowledge in hypervisor driver
(Does not scale for 30+ device types)
conclusion for picking AQ approach:
==================================
1. Overall AQ based access is simpler to implement with combination of
best from software and device so that legacy registers do not get baked
in the device hardware
2. AQ allows hypervisor software to intercept legacy registers and make
corrections if needed
3. Provides trade-off between performance, device complexity vs spec,
while still maintaining passthrough mode for the VFs with minimal
hypervisor intercepts only for legacy registers access
4. AQ mechanism is designed for accessing other member devices registers
as noted in AQ submission, it utilizes the existing infrastructure over
other alternatives.
5. Uses existing driver notification region similar to legacy notification
saves hardware resources
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v5->v6:
- fixed previous missed abbreviation of LCC and LD
- added text for the PCI capability for the group member device
v4->v5:
- split pci transport and generic command section to new patch
- removed multiple references to the VF
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
- split from pci transport specific patch
- split conformance to transport and generic sections
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
- rename fields from register to region
- avoided abbreviation for legacy, device and config
v3->v4:
- moved noted to the conformance section details in next patch
- removed queue notify address query AQ command on Michael's suggestion,
though it is fine. Instead replaced with extending virtio_pci_notify_cap
to indicate that legacy queue notifications can be done on the
notification location
- fixed spelling errors
- replaced administrative virtqueue to administration virtqueue
- moved legacy interface normative references to legacy conformance
section
v2->v3:
- added new patch to split raws of admin vq opcode table
- adddressed Jason and Michael's comment to split single register
access command to common config and device specific commands.
- dropped the suggetion to introduce enable/disable command as
admin command cap bit already covers it.
- added other alternative design considered and discussed in detail in v0, v1 and v2
v1->v2:
- addressed comments from Michael
- added theory of operation
- grammar corrections
- removed group fields description from individual commands as
it is already present in generic section
- added endianness normative for legacy device registers region
- renamed the file to drop vf and add legacy prefix
- added overview in commit log
- renamed subsection to reflect command
v0->v1:
- addressed comments, suggesetions and ideas from Michael Tsirkin and Jason Wang
- far more simpler design than MMR access
- removed complexities of MMR device ids
- removed complexities of MMR registers and extended capabilities
- dropped adding new extended capabilities because if if they are
added, a pci device still needs to have existing capabilities
in the legacy configuration space and hypervisor driver do not
need to access them
Parav Pandit (4):
admin: Split opcode table rows with a line
admin: Fix section numbering
admin: Add group member legacy register access commands
transport-pci: Introduce group legacy group member config region
access
admin-cmds-legacy-access.tex | 138 ++++++++++++++++++++++++++++++++++
admin.tex | 18 ++++-
conformance.tex | 5 ++
transport-pci-legacy-regs.tex | 48 ++++++++++++
transport-pci.tex | 20 +++++
5 files changed, 226 insertions(+), 3 deletions(-)
create mode 100644 admin-cmds-legacy-access.tex
create mode 100644 transport-pci-legacy-regs.tex
--
2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
@ 2023-06-13 17:30 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
This short series introduces legacy registers access commands for the owner
group member PCI PF to access the legacy registers of the member VFs.
If in future any SIOV devices to support legacy registers, they
can be easily supported using same commands by using the group
member identifiers of the future SIOV devices.
More details as overview, motivation, use case are further described
below.
Patch summary:
--------------
patch-1 split rows of admin opcode tables by a line
patch-2 adds administrative virtuqueue commands
patch-3 adds its conformance section and links
It uses the newly introduced administrative virtqueue facility with 4 new
commands which uses the existing virtio_admin_cmd.
Usecase:
--------
1. A hypervisor/system needs to provide transitional
virtio devices to the guest VM at scale of thousands,
typically, one to eight devices per VM.
2. A hypervisor/system needs to provide such devices using a
vendor agnostic driver in the hypervisor system.
3. A hypervisor system prefers to have single stack regardless of
virtio device type (net/blk) and be future compatible with a
single vfio stack using SR-IOV or other scalable device
virtualization technology to map PCI devices to the guest VM.
(as transitional or otherwise)
Motivation/Background:
----------------------
The existing virtio transitional PCI device is missing support for
PCI SR-IOV based devices. Currently it does not work beyond
PCI PF, or as software emulated device in reality. Currently it
has below cited system level limitations:
[a] PCIe spec citation:
VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
[b] cpu arch citiation:
Intel 64 and IA-32 Architectures Software Developer’s Manual:
The processor’s I/O address space is separate and distinct from
the physical-memory address space. The I/O address space consists
of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
[c] PCIe spec citation:
If a bridge implements an I/O address range,...I/O address range will be
aligned to a 4 KB boundary.
Overview:
---------
Above usecase requirements is solved by PCI PF group owner accessing
its group member PCI VFs legacy registers using an admin virtqueue of
the group owner PCI PF.
Two new admin virtqueue commands are added which read/write PCI VF
registers.
Software usage example:
-----------------------
One way to use and map to the guest VM is by using vfio driver
framework in Linux kernel.
+----------------------+
|pci_dev_id = 0x100X |
+---------------|pci_rev_id = 0x0 |-----+
|vfio device |BAR0 = I/O region | |
| |Other attributes | |
| +----------------------+ |
| |
+ +--------------+ +-----------------+ |
| |I/O BAR to AQ | | Other vfio | |
| |rd/wr mapper | | functionalities | |
| +--------------+ +-----------------+ |
| |
+------+-------------------------+-----------+
| |
| Driver notification
| |
| |
+----+------------+ +----+------------+
| +-----+ | | PCI VF device A |
| | AQ |-------------+---->+-------------+ |
| +-----+ | | | | legacy regs | |
| PCI PF device | | | +-------------+ |
+-----------------+ | +-----------------+
|
| +----+------------+
| | PCI VF device N |
+---->+-------------+ |
| | legacy regs | |
| +-------------+ |
+-----------------+
2. Virtio pci driver to bind to the listed device id and
use it in the host.
3. Use it in a light weight hypervisor to run bare-metal OS.
Please review.
Alternatives considered:
========================
1. Exposing BAR0 as MMIO BAR that follows legacy registers template
Pros:
a. Kind of works with legacy drivers as some of them have used API
which is agnostic to MMIO vs IOBAR.
b. Does not require hypervisor intervantion
Cons:
a. Device reset is extremely hard to implement in device at scale as
driver does not wait for device reset completion
b. Device register width related problems persist that hypervisor if
wishes, it cannot be fixed.
2. Accessing VF registers by tunneling it through new legacy PCI capability
Pros:
a. Self contained, but cannot work with future PCI SIOV devices
Cons:
a. Equally slow as AQ access
b. Still requires new capability for notification access
c. Requires hardware to build low level registers access which is not worth
for long term future
3. Accessing VF notification region using new PF BAR
Cons:
a. Requires hardware to build new PCI steering logic per PF to forward
notification from the PF to VF, requires double the amount of logic
compared to today
b. Requires very large additional PF BAR whose size must be max_Vfs * BAR size.
4. Trapping CVQ, configuration region, LEGACY_HDR
Cons:
a. This does not fullfil the very basic requirement to not trap the
1.x objects (configuration registers, vqs)
b. Requires feature negotiations mediation in hypervisor software
c. Requires constant device type specific knowledge in hypervisor driver
(Does not scale for 30+ device types)
conclusion for picking AQ approach:
==================================
1. Overall AQ based access is simpler to implement with combination of
best from software and device so that legacy registers do not get baked
in the device hardware
2. AQ allows hypervisor software to intercept legacy registers and make
corrections if needed
3. Provides trade-off between performance, device complexity vs spec,
while still maintaining passthrough mode for the VFs with minimal
hypervisor intercepts only for legacy registers access
4. AQ mechanism is designed for accessing other member devices registers
as noted in AQ submission, it utilizes the existing infrastructure over
other alternatives.
5. Uses existing driver notification region similar to legacy notification
saves hardware resources
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v5->v6:
- fixed previous missed abbreviation of LCC and LD
- added text for the PCI capability for the group member device
v4->v5:
- split pci transport and generic command section to new patch
- removed multiple references to the VF
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
- split from pci transport specific patch
- split conformance to transport and generic sections
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
- rename fields from register to region
- avoided abbreviation for legacy, device and config
v3->v4:
- moved noted to the conformance section details in next patch
- removed queue notify address query AQ command on Michael's suggestion,
though it is fine. Instead replaced with extending virtio_pci_notify_cap
to indicate that legacy queue notifications can be done on the
notification location
- fixed spelling errors
- replaced administrative virtqueue to administration virtqueue
- moved legacy interface normative references to legacy conformance
section
v2->v3:
- added new patch to split raws of admin vq opcode table
- adddressed Jason and Michael's comment to split single register
access command to common config and device specific commands.
- dropped the suggetion to introduce enable/disable command as
admin command cap bit already covers it.
- added other alternative design considered and discussed in detail in v0, v1 and v2
v1->v2:
- addressed comments from Michael
- added theory of operation
- grammar corrections
- removed group fields description from individual commands as
it is already present in generic section
- added endianness normative for legacy device registers region
- renamed the file to drop vf and add legacy prefix
- added overview in commit log
- renamed subsection to reflect command
v0->v1:
- addressed comments, suggesetions and ideas from Michael Tsirkin and Jason Wang
- far more simpler design than MMR access
- removed complexities of MMR device ids
- removed complexities of MMR registers and extended capabilities
- dropped adding new extended capabilities because if if they are
added, a pci device still needs to have existing capabilities
in the legacy configuration space and hypervisor driver do not
need to access them
Parav Pandit (4):
admin: Split opcode table rows with a line
admin: Fix section numbering
admin: Add group member legacy register access commands
transport-pci: Introduce group legacy group member config region
access
admin-cmds-legacy-access.tex | 138 ++++++++++++++++++++++++++++++++++
admin.tex | 18 ++++-
conformance.tex | 5 ++
transport-pci-legacy-regs.tex | 48 ++++++++++++
transport-pci.tex | 20 +++++
5 files changed, 226 insertions(+), 3 deletions(-)
create mode 100644 admin-cmds-legacy-access.tex
create mode 100644 transport-pci-legacy-regs.tex
--
2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] [PATCH v6 1/4] admin: Split opcode table rows with a line
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-13 17:30 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
Currently all opcode appears to be in a single row.
Separate them with a line similar to other tables.
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v2->v3:
- new patch
---
admin.tex | 2 ++
1 file changed, 2 insertions(+)
diff --git a/admin.tex b/admin.tex
index 2efd4d7..e51f9e6 100644
--- a/admin.tex
+++ b/admin.tex
@@ -114,7 +114,9 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
opcode & Name & Command Description \\
\hline \hline
0x0000 & VIRTIO_ADMIN_CMD_LIST_QUERY & Provides to driver list of commands supported for this group type \\
+\hline
0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\
+\hline
0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
\hline
0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\
--
2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-comment] [PATCH v6 1/4] admin: Split opcode table rows with a line
@ 2023-06-13 17:30 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
Currently all opcode appears to be in a single row.
Separate them with a line similar to other tables.
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v2->v3:
- new patch
---
admin.tex | 2 ++
1 file changed, 2 insertions(+)
diff --git a/admin.tex b/admin.tex
index 2efd4d7..e51f9e6 100644
--- a/admin.tex
+++ b/admin.tex
@@ -114,7 +114,9 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
opcode & Name & Command Description \\
\hline \hline
0x0000 & VIRTIO_ADMIN_CMD_LIST_QUERY & Provides to driver list of commands supported for this group type \\
+\hline
0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\
+\hline
0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
\hline
0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\
--
2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-dev] [PATCH v6 2/4] admin: Fix section numbering
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-13 17:30 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
Requirements are put one additional level down. Fix it.
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v4->v5:
- new patch
---
admin.tex | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/admin.tex b/admin.tex
index e51f9e6..fd3b97d 100644
--- a/admin.tex
+++ b/admin.tex
@@ -286,7 +286,7 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
supporting multiple group types, the list of supported commands
might differ between different group types.
-\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+\devicenormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
The device MUST validate \field{opcode}, \field{group_type} and
\field{group_member_id}, and if any of these has an invalid or
@@ -378,7 +378,7 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
\field{VF Enable} refer to registers within the SR-IOV Extended
Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
-\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+\drivernormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
The driver MAY discover whether device supports a specific group type
by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
--
2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-comment] [PATCH v6 2/4] admin: Fix section numbering
@ 2023-06-13 17:30 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
Requirements are put one additional level down. Fix it.
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v4->v5:
- new patch
---
admin.tex | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/admin.tex b/admin.tex
index e51f9e6..fd3b97d 100644
--- a/admin.tex
+++ b/admin.tex
@@ -286,7 +286,7 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
supporting multiple group types, the list of supported commands
might differ between different group types.
-\devicenormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+\devicenormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
The device MUST validate \field{opcode}, \field{group_type} and
\field{group_member_id}, and if any of these has an invalid or
@@ -378,7 +378,7 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
\field{VF Enable} refer to registers within the SR-IOV Extended
Capability as specified by \hyperref[intro:PCIe]{[PCIe]}.
-\drivernormative{\paragraph}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
+\drivernormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
The driver MAY discover whether device supports a specific group type
by issuing VIRTIO_ADMIN_CMD_LIST_QUERY with the matching
--
2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-dev] [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-13 17:30 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
Introduce group member legacy common configuration and legacy device
configuration access read/write commands.
Group member legacy registers access commands enable group owner driver
software to access legacy registers on behalf of the guest virtual
machine.
Usecase:
========
1. A hypervisor/system needs to provide transitional
virtio devices to the guest VM at scale of thousands,
typically, one to eight devices per VM.
2. A hypervisor/system needs to provide such devices using a
vendor agnostic driver in the hypervisor system.
3. A hypervisor system prefers to have single stack regardless of
virtio device type (net/blk) and be future compatible with a
single vfio stack using SR-IOV or other scalable device
virtualization technology to map PCI devices to the guest VM.
(as transitional or otherwise)
Motivation/Background:
=====================
The existing virtio transitional PCI device is missing support for
PCI SR-IOV based devices. Currently it does not work beyond
PCI PF, or as software emulated device in reality. Currently it
has below cited system level limitations:
[a] PCIe spec citation:
VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
[b] cpu arch citiation:
Intel 64 and IA-32 Architectures Software Developer’s Manual:
The processor’s I/O address space is separate and distinct from
the physical-memory address space. The I/O address space consists
of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
[c] PCIe spec citation:
If a bridge implements an I/O address range,...I/O address range will be
aligned to a 4 KB boundary.
Overview:
=========
Above usecase requirements is solved by PCI PF group owner accessing
its group member PCI VFs legacy registers using an admin virtqueue of
the group owner PCI PF.
Two types of administration commands are added which read/write PCI VF
registers.
Software usage example:
=======================
1. One way to use and map to the guest VM is by using vfio driver
framework in Linux kernel.
+----------------------+
|pci_dev_id = 0x100X |
+---------------|pci_rev_id = 0x0 |-----+
|vfio device |BAR0 = I/O region | |
| |Other attributes | |
| +----------------------+ |
| |
+ +--------------+ +-----------------+ |
| |I/O BAR to AQ | | Other vfio | |
| |rd/wr mapper& | | functionalities | |
| | forwarder | | | |
| +--------------+ +-----------------+ |
| |
+------+-------------------------+-----------+
| |
Config region |
access Driver notifications
| |
+----+------------+ +----+------------+
| +-----+ | | PCI VF device A |
| | AQ |-------------+---->+-------------+ |
| +-----+ | | | | legacy regs | |
| PCI PF device | | | +-------------+ |
+-----------------+ | +-----------------+
|
| +----+------------+
| | PCI VF device N |
+---->+-------------+ |
| | legacy regs | |
| +-------------+ |
+-----------------+
2. Continue to use the virtio pci driver to bind to the
listed device id and use it as in the host.
3. Use it in a light weight hypervisor to run bare-metal OS.
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v5->v6:
- fixed previous missed abbreviation of LCC and LD
v4->v5:
- split from pci transport specific patch
- split conformance to transport and generic sections
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
- rename fields from register to region
- avoided abbreviation for legacy, device and config
---
admin-cmds-legacy-access.tex | 138 +++++++++++++++++++++++++++++++++++
admin.tex | 12 ++-
conformance.tex | 2 +
3 files changed, 151 insertions(+), 1 deletion(-)
create mode 100644 admin-cmds-legacy-access.tex
diff --git a/admin-cmds-legacy-access.tex b/admin-cmds-legacy-access.tex
new file mode 100644
index 0000000..89a4f49
--- /dev/null
+++ b/admin-cmds-legacy-access.tex
@@ -0,0 +1,138 @@
+\subsubsection{Legacy Interfaces: legacy configuration access commands}\label{sec:Basic Facilities of a Virtio Device / Device groups / Group
+administration commands / Legacy Interfaces: legacy configuration access commands}
+
+The group owner device can optionally enable the driver to access its member devices legacy common
+configuration and device configuration region using following listed
+administrative commands:
+
+\begin{enumerate}
+\item Legacy Common Configuration Region Write Command
+\item Legacy common Configuration Region Read Command
+\item Legacy Device Configuration Region Write Command
+\item Legacy Device Configuration Region Read Command
+\end{enumerate}
+
+\paragraph{Legacy Common Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group
+administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Write Command}
+
+This command writes in a legacy common configuration region of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+This command uses following structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_common_cfg_wr_data {
+ u8 offset; /* Starting byte offset of the common configuration region to write */
+ u8 region_data[];
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE.
+The driver sets valid \field{offset} and associated \field{region_data} bytes to
+write to the common configuration region.
+
+This command does not have any command specific result.
+
+\paragraph{Legacy Common Configuration Region Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Read Command}
+
+This command reads from a legacy common configuration region of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+This command uses following listed structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_common_cfg_rd_data {
+ u8 offset; /* Starting byte offset of the common configuration region to read */
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ.
+
+The driver sets valid \field{offset} of the region from where to read
+\field{region_data}.
+
+When command completes successfully, \field{command_specific_result}
+uses following listed structure:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_common_cfg_rd_result {
+ u8 region_data[];
+};
+\end{lstlisting}
+
+\paragraph{Legacy Device Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Write Command}
+
+This command writes in a legacy device configuration registers of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+This command uses following structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_ld_reg_wr_data {
+ u8 offset; /* Starting byte offset of the device configuration region to write */
+ u8 region_data[];
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_WRITE.
+The driver sets valid \field{offset} and associated \field{region_data} bytes to
+the device configuration region.
+
+This command does not have any command specific result.
+
+\paragraph{Legacy Device Registers Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Read Command}
+
+This command reads from a legacy device configuration region of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+
+This command uses following listed structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_dev_cfg_rd_data {
+ u8 offset; /* Starting byte offset of the device configuration region to read */
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_READ.
+The driver sets valid \field{offset} of the device configurationi region from
+where to read \field{region_data}.
+
+When command completes successfully, \field{command_specific_result}
+uses following listed structure:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_ld_reg_rd_result {
+ u8 region_data[];
+};
+\end{lstlisting}
+
+\devicenormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
+
+If the group owner device supports legacy registers access for its group members,
+the device MUST set all corresponding bits for commands
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ in
+the command result of VIRTIO_ADMIN_CMD_LIST_QUERY in
+\field{device_admin_cmd_opcodes}.
+
+The device MUST encode and decode legacy device specific registers using
+little-endian format.
+
+The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands for the invalid offset
+which is outside the legacy common configuration register's address range.
+
+The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE,
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ commands for the invalid offset
+which is outside the legacy device specific register's address range.
+
+\drivernormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
+
+The driver MUST encode and decode legacy device specific registers using
+little-endian format.
+
+The driver SHOULD send VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands with a valid offset which
+is in the legacy common configuration region address range.
+
+The driver SHOULD send commands VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ with a valid offset which is in the legacy
+device specific configuration region address range.
diff --git a/admin.tex b/admin.tex
index fd3b97d..49a0b6f 100644
--- a/admin.tex
+++ b/admin.tex
@@ -117,7 +117,15 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
\hline
0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\
\hline
-0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
+0x0002 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE & Write legacy common configuration region of a member device \\
+\hline
+0x0003 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ & Read legacy common configuration region of a member device \\
+\hline
+0x0004 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE & Write legacy device configuration region of a member device \\
+\hline
+0x0005 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ & Read legacy device configuration region of a member device \\
+\hline
+0x0006 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
\hline
0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\
\hline
@@ -286,6 +294,8 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
supporting multiple group types, the list of supported commands
might differ between different group types.
+\input{admin-cmds-legacy-access.tex}
+
\devicenormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
The device MUST validate \field{opcode}, \field{group_type} and
diff --git a/conformance.tex b/conformance.tex
index 01ccd69..6bac4b0 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -260,6 +260,8 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
\item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Layout}
\item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Endianness}
\item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Message Framing / Legacy Interface: Message Framing}
+\item Section \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
+\item Section \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
\item Section \ref{sec:General Initialization And Device Operation / Device Initialization / Legacy Interface: Device Initialization}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery / Legacy Interfaces: A Note on PCI Device Discovery}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
--
2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-comment] [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-13 17:30 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
Introduce group member legacy common configuration and legacy device
configuration access read/write commands.
Group member legacy registers access commands enable group owner driver
software to access legacy registers on behalf of the guest virtual
machine.
Usecase:
========
1. A hypervisor/system needs to provide transitional
virtio devices to the guest VM at scale of thousands,
typically, one to eight devices per VM.
2. A hypervisor/system needs to provide such devices using a
vendor agnostic driver in the hypervisor system.
3. A hypervisor system prefers to have single stack regardless of
virtio device type (net/blk) and be future compatible with a
single vfio stack using SR-IOV or other scalable device
virtualization technology to map PCI devices to the guest VM.
(as transitional or otherwise)
Motivation/Background:
=====================
The existing virtio transitional PCI device is missing support for
PCI SR-IOV based devices. Currently it does not work beyond
PCI PF, or as software emulated device in reality. Currently it
has below cited system level limitations:
[a] PCIe spec citation:
VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
[b] cpu arch citiation:
Intel 64 and IA-32 Architectures Software Developer’s Manual:
The processor’s I/O address space is separate and distinct from
the physical-memory address space. The I/O address space consists
of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
[c] PCIe spec citation:
If a bridge implements an I/O address range,...I/O address range will be
aligned to a 4 KB boundary.
Overview:
=========
Above usecase requirements is solved by PCI PF group owner accessing
its group member PCI VFs legacy registers using an admin virtqueue of
the group owner PCI PF.
Two types of administration commands are added which read/write PCI VF
registers.
Software usage example:
=======================
1. One way to use and map to the guest VM is by using vfio driver
framework in Linux kernel.
+----------------------+
|pci_dev_id = 0x100X |
+---------------|pci_rev_id = 0x0 |-----+
|vfio device |BAR0 = I/O region | |
| |Other attributes | |
| +----------------------+ |
| |
+ +--------------+ +-----------------+ |
| |I/O BAR to AQ | | Other vfio | |
| |rd/wr mapper& | | functionalities | |
| | forwarder | | | |
| +--------------+ +-----------------+ |
| |
+------+-------------------------+-----------+
| |
Config region |
access Driver notifications
| |
+----+------------+ +----+------------+
| +-----+ | | PCI VF device A |
| | AQ |-------------+---->+-------------+ |
| +-----+ | | | | legacy regs | |
| PCI PF device | | | +-------------+ |
+-----------------+ | +-----------------+
|
| +----+------------+
| | PCI VF device N |
+---->+-------------+ |
| | legacy regs | |
| +-------------+ |
+-----------------+
2. Continue to use the virtio pci driver to bind to the
listed device id and use it as in the host.
3. Use it in a light weight hypervisor to run bare-metal OS.
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v5->v6:
- fixed previous missed abbreviation of LCC and LD
v4->v5:
- split from pci transport specific patch
- split conformance to transport and generic sections
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
- rename fields from register to region
- avoided abbreviation for legacy, device and config
---
admin-cmds-legacy-access.tex | 138 +++++++++++++++++++++++++++++++++++
admin.tex | 12 ++-
conformance.tex | 2 +
3 files changed, 151 insertions(+), 1 deletion(-)
create mode 100644 admin-cmds-legacy-access.tex
diff --git a/admin-cmds-legacy-access.tex b/admin-cmds-legacy-access.tex
new file mode 100644
index 0000000..89a4f49
--- /dev/null
+++ b/admin-cmds-legacy-access.tex
@@ -0,0 +1,138 @@
+\subsubsection{Legacy Interfaces: legacy configuration access commands}\label{sec:Basic Facilities of a Virtio Device / Device groups / Group
+administration commands / Legacy Interfaces: legacy configuration access commands}
+
+The group owner device can optionally enable the driver to access its member devices legacy common
+configuration and device configuration region using following listed
+administrative commands:
+
+\begin{enumerate}
+\item Legacy Common Configuration Region Write Command
+\item Legacy common Configuration Region Read Command
+\item Legacy Device Configuration Region Write Command
+\item Legacy Device Configuration Region Read Command
+\end{enumerate}
+
+\paragraph{Legacy Common Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group
+administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Write Command}
+
+This command writes in a legacy common configuration region of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+This command uses following structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_common_cfg_wr_data {
+ u8 offset; /* Starting byte offset of the common configuration region to write */
+ u8 region_data[];
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE.
+The driver sets valid \field{offset} and associated \field{region_data} bytes to
+write to the common configuration region.
+
+This command does not have any command specific result.
+
+\paragraph{Legacy Common Configuration Region Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Read Command}
+
+This command reads from a legacy common configuration region of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+This command uses following listed structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_common_cfg_rd_data {
+ u8 offset; /* Starting byte offset of the common configuration region to read */
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ.
+
+The driver sets valid \field{offset} of the region from where to read
+\field{region_data}.
+
+When command completes successfully, \field{command_specific_result}
+uses following listed structure:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_common_cfg_rd_result {
+ u8 region_data[];
+};
+\end{lstlisting}
+
+\paragraph{Legacy Device Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Write Command}
+
+This command writes in a legacy device configuration registers of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+This command uses following structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_ld_reg_wr_data {
+ u8 offset; /* Starting byte offset of the device configuration region to write */
+ u8 region_data[];
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_WRITE.
+The driver sets valid \field{offset} and associated \field{region_data} bytes to
+the device configuration region.
+
+This command does not have any command specific result.
+
+\paragraph{Legacy Device Registers Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Read Command}
+
+This command reads from a legacy device configuration region of a group member device.
+This command follows \field{struct virtio_admin_cmd}.
+
+This command uses following listed structure for \field{command_specific_data}:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_legacy_dev_cfg_rd_data {
+ u8 offset; /* Starting byte offset of the device configuration region to read */
+};
+\end{lstlisting}
+
+The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_READ.
+The driver sets valid \field{offset} of the device configurationi region from
+where to read \field{region_data}.
+
+When command completes successfully, \field{command_specific_result}
+uses following listed structure:
+
+\begin{lstlisting}
+struct virtio_admin_cmd_ld_reg_rd_result {
+ u8 region_data[];
+};
+\end{lstlisting}
+
+\devicenormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
+
+If the group owner device supports legacy registers access for its group members,
+the device MUST set all corresponding bits for commands
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ in
+the command result of VIRTIO_ADMIN_CMD_LIST_QUERY in
+\field{device_admin_cmd_opcodes}.
+
+The device MUST encode and decode legacy device specific registers using
+little-endian format.
+
+The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands for the invalid offset
+which is outside the legacy common configuration register's address range.
+
+The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE,
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ commands for the invalid offset
+which is outside the legacy device specific register's address range.
+
+\drivernormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
+
+The driver MUST encode and decode legacy device specific registers using
+little-endian format.
+
+The driver SHOULD send VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands with a valid offset which
+is in the legacy common configuration region address range.
+
+The driver SHOULD send commands VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ with a valid offset which is in the legacy
+device specific configuration region address range.
diff --git a/admin.tex b/admin.tex
index fd3b97d..49a0b6f 100644
--- a/admin.tex
+++ b/admin.tex
@@ -117,7 +117,15 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
\hline
0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\
\hline
-0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
+0x0002 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE & Write legacy common configuration region of a member device \\
+\hline
+0x0003 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ & Read legacy common configuration region of a member device \\
+\hline
+0x0004 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE & Write legacy device configuration region of a member device \\
+\hline
+0x0005 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ & Read legacy device configuration region of a member device \\
+\hline
+0x0006 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
\hline
0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\
\hline
@@ -286,6 +294,8 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
supporting multiple group types, the list of supported commands
might differ between different group types.
+\input{admin-cmds-legacy-access.tex}
+
\devicenormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
The device MUST validate \field{opcode}, \field{group_type} and
diff --git a/conformance.tex b/conformance.tex
index 01ccd69..6bac4b0 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -260,6 +260,8 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
\item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Layout}
\item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Endianness}
\item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Message Framing / Legacy Interface: Message Framing}
+\item Section \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
+\item Section \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
\item Section \ref{sec:General Initialization And Device Operation / Device Initialization / Legacy Interface: Device Initialization}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery / Legacy Interfaces: A Note on PCI Device Discovery}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
--
2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-dev] [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-13 17:30 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
This patch links how in a PCI transport a group owner can access group
member (PCI VFs) legacy registers using a legacy registers access
commands using administration virtqueue infrastructure.
Additionally it extend the PCI notification capability through which a
PCI VF device indicates to the driver which PCI BAR region to be used
for driver notifications.
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v5->v6:
- aligned pci capability to 4B as required by PCI spec
- added text for the PCI capability for the group member device
v4->v5:
- split pci transport and generic command section to new patch
- removed multiple references to the VF
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
v3->v4:
- moved noted to the conformance section details in next patch
- removed queue notify address query AQ command on Michael's suggestion,
though it is fine. Instead replaced with extending virtio_pci_notify_cap
to indicate that legacy queue notifications can be done on the
notification location.
- fixed spelling errors.
- replaced administrative virtqueue to administration virtqueue
- added queue notification capability register to indicate legacy q
notification supported
v2->v3:
- adddressed Jason and Michael's comment to split single register
access command to common config and device specific commands.
- dropped the suggetion to introduce enable/disable command as
admin command cap bit already covers it.
v1->v2:
- addressed comments from Michael
- added theory of operation
- grammar corrections
- removed group fields description from individual commands as
it is already present in generic section
- added endianness normative for legacy device registers region
- renamed the file to drop vf and add legacy prefix
- added overview in commit log
- renamed subsection to reflect command
---
conformance.tex | 3 +++
transport-pci-legacy-regs.tex | 48 +++++++++++++++++++++++++++++++++++
transport-pci.tex | 20 +++++++++++++++
3 files changed, 71 insertions(+)
create mode 100644 transport-pci-legacy-regs.tex
diff --git a/conformance.tex b/conformance.tex
index 6bac4b0..93f409c 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -267,6 +267,9 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtio Device Configuration Layout Detection / Legacy Interface: A Note on Device Layout Detection}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtqueue Configuration / Legacy Interface: A Note on Virtqueue Configuration}
+\item Section \ref{devicenormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+\item Section \ref{drivernormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+\item Section \ref{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
\item Section \ref{sec:Virtio Transport Options / Virtio Over MMIO / Legacy interface}
\item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Setting the Virtio Revision / Legacy Interfaces: A Note on Setting the Virtio Revision}
\item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue / Legacy Interface: A Note on Configuring a Virtqueue}
diff --git a/transport-pci-legacy-regs.tex b/transport-pci-legacy-regs.tex
new file mode 100644
index 0000000..b671b3e
--- /dev/null
+++ b/transport-pci-legacy-regs.tex
@@ -0,0 +1,48 @@
+\subsection{Legacy Interfaces: Group member device Configuration Region Access}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces: Group Member Device Configuration Region Access}
+
+As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]} PCI VFs
+do not support the I/O space BAR. Due to this limitation, a PCI VF hardware
+device that supports legacy interfaces cannot be attached as passthrough
+device to the guest virtual machine.
+
+To have a transitional or legacy virtio device in the guest virtual machine,
+a PCI PF device as a group owner may support accessing its group member
+device's legacy configuration region using the group administration commands
+listed at \ref{sec:Basic Facilities of a Virtio Device / Device groups / Group
+administration commands / Legacy Interfaces: legacy configuration access commands}.
+These legacy configuration access commands are transported via the
+administration virtqueue. This mechanism enables minimal involvement of the
+hypervisor software only for the purpose of the legacy configuration region access.
+
+When a guest virtual machine requests legacy configuration and device
+specific regions of group member device, a hypervisor software accesses
+it using an administration virtqueue on behalf of the guest virtual machine.
+
+Even though virtqueue driver notifications can be communicated through
+administration virtqueue, if the group member device support such
+notifications using a memory-mapped operation, such driver notifications
+are sent using a group member device defined notification region.
+
+The group owner device should not expose PCI BAR0 for the group member
+devices when it prefers to support legacy interface for legacy configuration
+access using its group owner.
+This facilitates hypervisor software to operate with least amount of
+complexities to emulate the legacy interface I/O space BAR and passthrough
+other PCI BARs and PCI device capabilities to the guest virtual machine
+without any translation.
+
+The group member device should not expose PCI BAR0 in various PCI capabilities.
+
+\devicenormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+
+When a PCI SR-IOV group owner device supports
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE, VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group owner device SHOULD
+NOT expose PCI BAR0 in the SR-IOV Extended capability. This is to facilitate the
+software to emulate I/O region BAR0 for supporting the legacy interface.
+
+\drivernormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+
+The driver SHOULD NOT emulate I/O region BAR0 if a device group member exposes
+a PCI BAR0.
diff --git a/transport-pci.tex b/transport-pci.tex
index a5c6719..3647485 100644
--- a/transport-pci.tex
+++ b/transport-pci.tex
@@ -541,6 +541,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
struct virtio_pci_notify_cap {
struct virtio_pci_cap cap;
le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
+ u8 legacy_q_notify_supported;
+ u8 reserved[3];
};
\end{lstlisting}
@@ -560,6 +562,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
the same Queue Notify address for all queues.
\end{note}
+\field{legacy_q_notify_supported} when set to 1, indicates that the device
+supports legacy queue notifications at this notification location. Legacy Queue
+Notification address is derived within a BAR for a virtqueue:
+
+\begin{lstlisting}
+ cap.offset
+\end{lstlisting}
+
\devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
The device MUST present at least one notification capability.
@@ -596,6 +606,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
cap.length >= queue_notify_off * notify_off_multiplier + 4
\end{lstlisting}
+\paragraph{Legacy Interfaces: Notification capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
+
+The device SHOULD set \field{legacy_q_notify_supported} when the device
+notification location supports legacy driver notifications.
+
+The driver in the hypervisor MAY use notification location for accessing
+legacy queue notify address when the device supports it.
+
\subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
The VIRTIO_PCI_CAP_ISR_CFG capability
@@ -1212,3 +1230,5 @@ \subsubsection{Driver Handling Interrupts}\label{sec:Virtio Transport Options /
re-examine the configuration space to see what changed.
\end{itemize}
\end{itemize}
+
+\input{transport-pci-legacy-regs.tex}
--
2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-comment] [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-13 17:30 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-13 17:30 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, yishaih, maorg, shahafs, Parav Pandit
This patch links how in a PCI transport a group owner can access group
member (PCI VFs) legacy registers using a legacy registers access
commands using administration virtqueue infrastructure.
Additionally it extend the PCI notification capability through which a
PCI VF device indicates to the driver which PCI BAR region to be used
for driver notifications.
Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
Signed-off-by: Parav Pandit <parav@nvidia.com>
---
changelog:
v5->v6:
- aligned pci capability to 4B as required by PCI spec
- added text for the PCI capability for the group member device
v4->v5:
- split pci transport and generic command section to new patch
- removed multiple references to the VF
- written the description of the command as generic with member
and group device terminology
- reflected many section names to remove VF
v3->v4:
- moved noted to the conformance section details in next patch
- removed queue notify address query AQ command on Michael's suggestion,
though it is fine. Instead replaced with extending virtio_pci_notify_cap
to indicate that legacy queue notifications can be done on the
notification location.
- fixed spelling errors.
- replaced administrative virtqueue to administration virtqueue
- added queue notification capability register to indicate legacy q
notification supported
v2->v3:
- adddressed Jason and Michael's comment to split single register
access command to common config and device specific commands.
- dropped the suggetion to introduce enable/disable command as
admin command cap bit already covers it.
v1->v2:
- addressed comments from Michael
- added theory of operation
- grammar corrections
- removed group fields description from individual commands as
it is already present in generic section
- added endianness normative for legacy device registers region
- renamed the file to drop vf and add legacy prefix
- added overview in commit log
- renamed subsection to reflect command
---
conformance.tex | 3 +++
transport-pci-legacy-regs.tex | 48 +++++++++++++++++++++++++++++++++++
transport-pci.tex | 20 +++++++++++++++
3 files changed, 71 insertions(+)
create mode 100644 transport-pci-legacy-regs.tex
diff --git a/conformance.tex b/conformance.tex
index 6bac4b0..93f409c 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -267,6 +267,9 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtio Device Configuration Layout Detection / Legacy Interface: A Note on Device Layout Detection}
\item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtqueue Configuration / Legacy Interface: A Note on Virtqueue Configuration}
+\item Section \ref{devicenormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+\item Section \ref{drivernormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+\item Section \ref{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
\item Section \ref{sec:Virtio Transport Options / Virtio Over MMIO / Legacy interface}
\item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Setting the Virtio Revision / Legacy Interfaces: A Note on Setting the Virtio Revision}
\item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue / Legacy Interface: A Note on Configuring a Virtqueue}
diff --git a/transport-pci-legacy-regs.tex b/transport-pci-legacy-regs.tex
new file mode 100644
index 0000000..b671b3e
--- /dev/null
+++ b/transport-pci-legacy-regs.tex
@@ -0,0 +1,48 @@
+\subsection{Legacy Interfaces: Group member device Configuration Region Access}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces: Group Member Device Configuration Region Access}
+
+As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]} PCI VFs
+do not support the I/O space BAR. Due to this limitation, a PCI VF hardware
+device that supports legacy interfaces cannot be attached as passthrough
+device to the guest virtual machine.
+
+To have a transitional or legacy virtio device in the guest virtual machine,
+a PCI PF device as a group owner may support accessing its group member
+device's legacy configuration region using the group administration commands
+listed at \ref{sec:Basic Facilities of a Virtio Device / Device groups / Group
+administration commands / Legacy Interfaces: legacy configuration access commands}.
+These legacy configuration access commands are transported via the
+administration virtqueue. This mechanism enables minimal involvement of the
+hypervisor software only for the purpose of the legacy configuration region access.
+
+When a guest virtual machine requests legacy configuration and device
+specific regions of group member device, a hypervisor software accesses
+it using an administration virtqueue on behalf of the guest virtual machine.
+
+Even though virtqueue driver notifications can be communicated through
+administration virtqueue, if the group member device support such
+notifications using a memory-mapped operation, such driver notifications
+are sent using a group member device defined notification region.
+
+The group owner device should not expose PCI BAR0 for the group member
+devices when it prefers to support legacy interface for legacy configuration
+access using its group owner.
+This facilitates hypervisor software to operate with least amount of
+complexities to emulate the legacy interface I/O space BAR and passthrough
+other PCI BARs and PCI device capabilities to the guest virtual machine
+without any translation.
+
+The group member device should not expose PCI BAR0 in various PCI capabilities.
+
+\devicenormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+
+When a PCI SR-IOV group owner device supports
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
+VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE, VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
+VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group owner device SHOULD
+NOT expose PCI BAR0 in the SR-IOV Extended capability. This is to facilitate the
+software to emulate I/O region BAR0 for supporting the legacy interface.
+
+\drivernormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
+
+The driver SHOULD NOT emulate I/O region BAR0 if a device group member exposes
+a PCI BAR0.
diff --git a/transport-pci.tex b/transport-pci.tex
index a5c6719..3647485 100644
--- a/transport-pci.tex
+++ b/transport-pci.tex
@@ -541,6 +541,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
struct virtio_pci_notify_cap {
struct virtio_pci_cap cap;
le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
+ u8 legacy_q_notify_supported;
+ u8 reserved[3];
};
\end{lstlisting}
@@ -560,6 +562,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
the same Queue Notify address for all queues.
\end{note}
+\field{legacy_q_notify_supported} when set to 1, indicates that the device
+supports legacy queue notifications at this notification location. Legacy Queue
+Notification address is derived within a BAR for a virtqueue:
+
+\begin{lstlisting}
+ cap.offset
+\end{lstlisting}
+
\devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
The device MUST present at least one notification capability.
@@ -596,6 +606,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
cap.length >= queue_notify_off * notify_off_multiplier + 4
\end{lstlisting}
+\paragraph{Legacy Interfaces: Notification capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
+
+The device SHOULD set \field{legacy_q_notify_supported} when the device
+notification location supports legacy driver notifications.
+
+The driver in the hypervisor MAY use notification location for accessing
+legacy queue notify address when the device supports it.
+
\subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
The VIRTIO_PCI_CAP_ISR_CFG capability
@@ -1212,3 +1230,5 @@ \subsubsection{Driver Handling Interrupts}\label{sec:Virtio Transport Options /
re-examine the configuration space to see what changed.
\end{itemize}
\end{itemize}
+
+\input{transport-pci-legacy-regs.tex}
--
2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply related [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 12:38 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 12:38 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
Hi Michael,
> From: Parav Pandit <parav@nvidia.com>
> Sent: Tuesday, June 13, 2023 1:30 PM
>
> This short series introduces legacy registers access commands for the owner
> group member PCI PF to access the legacy registers of the member VFs.
>
[..]
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
>
> ---
> changelog:
> v5->v6:
> - fixed previous missed abbreviation of LCC and LD
> - added text for the PCI capability for the group member device
> v4->v5:
> - split pci transport and generic command section to new patch
> - removed multiple references to the VF
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - split from pci transport specific patch
> - split conformance to transport and generic sections
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - rename fields from register to region
> - avoided abbreviation for legacy, device and config
Do you have any further comments?
I would like to raise for the vote now as this is in oven for several months now and users need this in 1.3 spec.
If there are any editorial fixes, I can fix them now with v7 or
if they are small, I can fix them during the voting wait period like how we completed AQ fixes.
Please let me know.
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
@ 2023-06-19 12:38 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 12:38 UTC (permalink / raw)
To: virtio-comment, mst, cohuck, david.edmondson
Cc: virtio-dev, sburla, jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
Hi Michael,
> From: Parav Pandit <parav@nvidia.com>
> Sent: Tuesday, June 13, 2023 1:30 PM
>
> This short series introduces legacy registers access commands for the owner
> group member PCI PF to access the legacy registers of the member VFs.
>
[..]
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
>
> ---
> changelog:
> v5->v6:
> - fixed previous missed abbreviation of LCC and LD
> - added text for the PCI capability for the group member device
> v4->v5:
> - split pci transport and generic command section to new patch
> - removed multiple references to the VF
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - split from pci transport specific patch
> - split conformance to transport and generic sections
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - rename fields from register to region
> - avoided abbreviation for legacy, device and config
Do you have any further comments?
I would like to raise for the vote now as this is in oven for several months now and users need this in 1.3 spec.
If there are any editorial fixes, I can fix them now with v7 or
if they are small, I can fix them during the voting wait period like how we completed AQ fixes.
Please let me know.
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
2023-06-19 12:38 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 15:18 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 15:18 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 12:38:29PM +0000, Parav Pandit wrote:
> Hi Michael,
>
> > From: Parav Pandit <parav@nvidia.com>
> > Sent: Tuesday, June 13, 2023 1:30 PM
> >
> > This short series introduces legacy registers access commands for the owner
> > group member PCI PF to access the legacy registers of the member VFs.
> >
>
> [..]
>
> > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> > Signed-off-by: Parav Pandit <parav@nvidia.com>
> >
> > ---
> > changelog:
> > v5->v6:
> > - fixed previous missed abbreviation of LCC and LD
> > - added text for the PCI capability for the group member device
> > v4->v5:
> > - split pci transport and generic command section to new patch
> > - removed multiple references to the VF
> > - written the description of the command as generic with member
> > and group device terminology
> > - reflected many section names to remove VF
> > - split from pci transport specific patch
> > - split conformance to transport and generic sections
> > - written the description of the command as generic with member
> > and group device terminology
> > - reflected many section names to remove VF
> > - rename fields from register to region
> > - avoided abbreviation for legacy, device and config
>
> Do you have any further comments?
> I would like to raise for the vote now as this is in oven for several months now and users need this in 1.3 spec.
> If there are any editorial fixes, I can fix them now with v7 or
> if they are small, I can fix them during the voting wait period like how we completed AQ fixes.
> Please let me know.
Just back from travel. I remember having comments, will send in a day or
two. I don't think we are talking about editorial things though.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
@ 2023-06-19 15:18 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 15:18 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 12:38:29PM +0000, Parav Pandit wrote:
> Hi Michael,
>
> > From: Parav Pandit <parav@nvidia.com>
> > Sent: Tuesday, June 13, 2023 1:30 PM
> >
> > This short series introduces legacy registers access commands for the owner
> > group member PCI PF to access the legacy registers of the member VFs.
> >
>
> [..]
>
> > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> > Signed-off-by: Parav Pandit <parav@nvidia.com>
> >
> > ---
> > changelog:
> > v5->v6:
> > - fixed previous missed abbreviation of LCC and LD
> > - added text for the PCI capability for the group member device
> > v4->v5:
> > - split pci transport and generic command section to new patch
> > - removed multiple references to the VF
> > - written the description of the command as generic with member
> > and group device terminology
> > - reflected many section names to remove VF
> > - split from pci transport specific patch
> > - split conformance to transport and generic sections
> > - written the description of the command as generic with member
> > and group device terminology
> > - reflected many section names to remove VF
> > - rename fields from register to region
> > - avoided abbreviation for legacy, device and config
>
> Do you have any further comments?
> I would like to raise for the vote now as this is in oven for several months now and users need this in 1.3 spec.
> If there are any editorial fixes, I can fix them now with v7 or
> if they are small, I can fix them during the voting wait period like how we completed AQ fixes.
> Please let me know.
Just back from travel. I remember having comments, will send in a day or
two. I don't think we are talking about editorial things though.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
2023-06-19 15:18 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 15:58 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 15:58 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 11:19 AM
>
> Just back from travel. I remember having comments, will send in a day or two. I
> don't think we are talking about editorial things though.
Let me know what are those new one those were not asked in v4.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
@ 2023-06-19 15:58 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 15:58 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 11:19 AM
>
> Just back from travel. I remember having comments, will send in a day or two. I
> don't think we are talking about editorial things though.
Let me know what are those new one those were not asked in v4.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 16:16 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:16 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> This patch links how in a PCI transport a group owner can access group
> member (PCI VFs) legacy registers using a legacy registers access
> commands using administration virtqueue infrastructure.
>
> Additionally it extend the PCI notification capability through which a
> PCI VF device indicates to the driver which PCI BAR region to be used
> for driver notifications.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
I'd love to just put this stuff in a separate tex document
and just link there from the generic thing.
Or if it's too hard, please create new sections for
this stuff. But in this case each of these sections
has to start with repeating the description when
things apply. E.g. "when using legacy interface"
when using commands "A B C to access" etc etc.
I wrote a bunch of comments below.
> ---
> changelog:
> v5->v6:
> - aligned pci capability to 4B as required by PCI spec
> - added text for the PCI capability for the group member device
> v4->v5:
> - split pci transport and generic command section to new patch
> - removed multiple references to the VF
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> v3->v4:
> - moved noted to the conformance section details in next patch
> - removed queue notify address query AQ command on Michael's suggestion,
> though it is fine. Instead replaced with extending virtio_pci_notify_cap
> to indicate that legacy queue notifications can be done on the
> notification location.
> - fixed spelling errors.
> - replaced administrative virtqueue to administration virtqueue
> - added queue notification capability register to indicate legacy q
> notification supported
> v2->v3:
> - adddressed Jason and Michael's comment to split single register
> access command to common config and device specific commands.
> - dropped the suggetion to introduce enable/disable command as
> admin command cap bit already covers it.
> v1->v2:
> - addressed comments from Michael
> - added theory of operation
> - grammar corrections
> - removed group fields description from individual commands as
> it is already present in generic section
> - added endianness normative for legacy device registers region
> - renamed the file to drop vf and add legacy prefix
>
> - added overview in commit log
> - renamed subsection to reflect command
> ---
> conformance.tex | 3 +++
> transport-pci-legacy-regs.tex | 48 +++++++++++++++++++++++++++++++++++
> transport-pci.tex | 20 +++++++++++++++
> 3 files changed, 71 insertions(+)
> create mode 100644 transport-pci-legacy-regs.tex
>
> diff --git a/conformance.tex b/conformance.tex
> index 6bac4b0..93f409c 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -267,6 +267,9 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtio Device Configuration Layout Detection / Legacy Interface: A Note on Device Layout Detection}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtqueue Configuration / Legacy Interface: A Note on Virtqueue Configuration}
> +\item Section \ref{devicenormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +\item Section \ref{drivernormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +\item Section \ref{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over MMIO / Legacy interface}
> \item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Setting the Virtio Revision / Legacy Interfaces: A Note on Setting the Virtio Revision}
> \item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue / Legacy Interface: A Note on Configuring a Virtqueue}
> diff --git a/transport-pci-legacy-regs.tex b/transport-pci-legacy-regs.tex
> new file mode 100644
> index 0000000..b671b3e
> --- /dev/null
> +++ b/transport-pci-legacy-regs.tex
> @@ -0,0 +1,48 @@
> +\subsection{Legacy Interfaces: Group member device Configuration Region Access}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces: Group Member Device Configuration Region Access}
> +
> +As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]} PCI VFs
> +do not support the I/O space BAR. Due to this limitation, a PCI VF hardware
> +device that supports legacy interfaces cannot be attached as passthrough
> +device to the guest virtual machine.
That's not the only reason. Let's keep things generic and start by
saying something like "Some configurations with a need to
support legacy drivers can not implement a transitional
device. For example .... ".
> +
> +To have a transitional or legacy virtio device in the guest virtual machine,
> +a PCI PF device as a group owner may
may -> can outside conformance statements.
> support accessing its group member
Pls keep it generic. A group owner ....
> +device's legacy configuration region using the group administration commands
> +listed at \ref{sec:Basic Facilities of a Virtio Device / Device groups / Group
> +administration commands / Legacy Interfaces: legacy configuration access commands}.
> +These legacy configuration access commands are transported via the
> +administration virtqueue.
are these called legacy configuration access commands then?
why do you introduce this term? it does not look like it's used
anywhere else.
I would prefer it if you just use the term "legacy interface"
it is well defined and widely used in the spec.
again leave virtqueue out of it pls we'll have other ways
down the road. what matters is that they are transferred
to a group member by sending legacy configuration access commands
to the group owner.
> This mechanism enables minimal involvement of the
> +hypervisor software only for the purpose of the legacy configuration region access.
Leave marketing out of it pls, kill last sentence.
> +
> +When a guest virtual machine requests legacy configuration and device
> +specific regions
wait so now it's legacy configuration and device specific
regions and these two are distinct?
> of group member device, a hypervisor software accesses
> +it using an administration virtqueue
please talk of administration commands. virtqueue does not matter
at all.
> on behalf of the guest virtual machine.
To put it in our terms, something like this:
when a legacy driver accesses a member device of
a group using the
legacy interface, a modern driver can intercept
the access and forward it to the owner device.
> +
> +Even though virtqueue driver notifications can be communicated through
> +administration virtqueue, if the group member device support such
supports?
> +notifications using a memory-mapped operation, such driver notifications
> +are sent using a group member device defined notification region.
what does "are sent" imply here? is there a matching conformance
statement? who sends them?
> +
> +The group owner device
this is the 1st you mention any group or its owner.
> should not expose PCI BAR0
Not PCI BAR0. That is in header. You mean VF BAR0.
> for the group member
> +devices when it prefers to support legacy interface for legacy configuration
> +access using its group owner.
don't use should outside conformance
I think this specific case actually should be more specific.
Something like:
- For PCI SRIOV groups, when group owner device implements commands
A,B,C the group owner's VF BAR 0 is hardwired to 0
in its PCI SRIOV capability.
> +This facilitates hypervisor software to operate with least amount of
> +complexities
complexity is its own plural
> to emulate the legacy interface I/O space BAR and passthrough
> +other PCI BARs and PCI device capabilities to the guest virtual machine
> +without any translation.
> +
> +The group member device should not expose PCI BAR0 in various PCI capabilities.
> +
> +\devicenormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +
> +When a PCI SR-IOV group owner device supports
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE, VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group owner device SHOULD
> +NOT expose PCI BAR0 in the SR-IOV Extended capability. This is to facilitate the
> +software to emulate I/O region BAR0 for supporting the legacy interface.
not PCI BAR0 - VF BAR0. Check the PCI spec
you can not "not expose it". if you want to
register can be "unimplemented".
Base Address registers are hardwired to zero
But it is better to be specific on what should happen. hardwire VF BAR0 to 0, right?
> +
> +\drivernormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +
> +The driver SHOULD NOT emulate I/O region BAR0 if a device group member exposes
> +a PCI BAR0.
what does "emulate" mean here? which driver it is?
> diff --git a/transport-pci.tex b/transport-pci.tex
> index a5c6719..3647485 100644
> --- a/transport-pci.tex
> +++ b/transport-pci.tex
> @@ -541,6 +541,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> struct virtio_pci_notify_cap {
> struct virtio_pci_cap cap;
> le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
> + u8 legacy_q_notify_supported;
I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> + u8 reserved[3];
align with spaces pls.
> };
> \end{lstlisting}
>
> @@ -560,6 +562,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> the same Queue Notify address for all queues.
> \end{note}
>
> +\field{legacy_q_notify_supported} when set to 1,
Do you mean something like:
When \field{legacy_q_notify_supported} this indicates that ...
> indicates that the device
> +supports legacy queue notifications at this notification location.
And what location? this refers to what? the location described by
this capability maybe?
More specifically, assume a transitional device (with an IO BAR
as normal). Does presence of this flag in the capability
imply we can use a legacy queue but use this
notification with this capability?
I worry that if we allow that, it opens floodgates of people
who are too lazy to upgrade to 1.0 but just hack
this notification in there.
> Legacy Queue
> +Notification address is derived within a BAR for a virtqueue:
> +
> +\begin{lstlisting}
> + cap.offset
> +\end{lstlisting}
> +
> \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
> The device MUST present at least one notification capability.
>
I don't get what this is saying. I *think* you really mean is
that queue_notify_off is 0.
> @@ -596,6 +606,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> cap.length >= queue_notify_off * notify_off_multiplier + 4
> \end{lstlisting}
>
> +\paragraph{Legacy Interfaces: Notification capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
> +
> +The device SHOULD set \field{legacy_q_notify_supported}
why SHOULD and not MUST here - if driver checks for it then it won't work without no?
> when the device
> +notification location supports legacy driver notifications.
legacy driver notifications is ambigous.
what you want here and elsewhere is something like
"when used through a legacy interface, device supports driver
> +
> +The driver in the hypervisor MAY use notification location for accessing
> +legacy queue notify address when the device supports it.
> +
> \subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
>
> The VIRTIO_PCI_CAP_ISR_CFG capability
> @@ -1212,3 +1230,5 @@ \subsubsection{Driver Handling Interrupts}\label{sec:Virtio Transport Options /
> re-examine the configuration space to see what changed.
> \end{itemize}
> \end{itemize}
> +
> +\input{transport-pci-legacy-regs.tex}
> --
> 2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 16:16 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:16 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> This patch links how in a PCI transport a group owner can access group
> member (PCI VFs) legacy registers using a legacy registers access
> commands using administration virtqueue infrastructure.
>
> Additionally it extend the PCI notification capability through which a
> PCI VF device indicates to the driver which PCI BAR region to be used
> for driver notifications.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
I'd love to just put this stuff in a separate tex document
and just link there from the generic thing.
Or if it's too hard, please create new sections for
this stuff. But in this case each of these sections
has to start with repeating the description when
things apply. E.g. "when using legacy interface"
when using commands "A B C to access" etc etc.
I wrote a bunch of comments below.
> ---
> changelog:
> v5->v6:
> - aligned pci capability to 4B as required by PCI spec
> - added text for the PCI capability for the group member device
> v4->v5:
> - split pci transport and generic command section to new patch
> - removed multiple references to the VF
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> v3->v4:
> - moved noted to the conformance section details in next patch
> - removed queue notify address query AQ command on Michael's suggestion,
> though it is fine. Instead replaced with extending virtio_pci_notify_cap
> to indicate that legacy queue notifications can be done on the
> notification location.
> - fixed spelling errors.
> - replaced administrative virtqueue to administration virtqueue
> - added queue notification capability register to indicate legacy q
> notification supported
> v2->v3:
> - adddressed Jason and Michael's comment to split single register
> access command to common config and device specific commands.
> - dropped the suggetion to introduce enable/disable command as
> admin command cap bit already covers it.
> v1->v2:
> - addressed comments from Michael
> - added theory of operation
> - grammar corrections
> - removed group fields description from individual commands as
> it is already present in generic section
> - added endianness normative for legacy device registers region
> - renamed the file to drop vf and add legacy prefix
>
> - added overview in commit log
> - renamed subsection to reflect command
> ---
> conformance.tex | 3 +++
> transport-pci-legacy-regs.tex | 48 +++++++++++++++++++++++++++++++++++
> transport-pci.tex | 20 +++++++++++++++
> 3 files changed, 71 insertions(+)
> create mode 100644 transport-pci-legacy-regs.tex
>
> diff --git a/conformance.tex b/conformance.tex
> index 6bac4b0..93f409c 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -267,6 +267,9 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtio Device Configuration Layout Detection / Legacy Interface: A Note on Device Layout Detection}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Device Initialization / Virtqueue Configuration / Legacy Interface: A Note on Virtqueue Configuration}
> +\item Section \ref{devicenormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +\item Section \ref{drivernormative:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +\item Section \ref{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over MMIO / Legacy interface}
> \item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Setting the Virtio Revision / Legacy Interfaces: A Note on Setting the Virtio Revision}
> \item Section \ref{sec:Virtio Transport Options / Virtio over channel I/O / Device Initialization / Configuring a Virtqueue / Legacy Interface: A Note on Configuring a Virtqueue}
> diff --git a/transport-pci-legacy-regs.tex b/transport-pci-legacy-regs.tex
> new file mode 100644
> index 0000000..b671b3e
> --- /dev/null
> +++ b/transport-pci-legacy-regs.tex
> @@ -0,0 +1,48 @@
> +\subsection{Legacy Interfaces: Group member device Configuration Region Access}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces: Group Member Device Configuration Region Access}
> +
> +As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]} PCI VFs
> +do not support the I/O space BAR. Due to this limitation, a PCI VF hardware
> +device that supports legacy interfaces cannot be attached as passthrough
> +device to the guest virtual machine.
That's not the only reason. Let's keep things generic and start by
saying something like "Some configurations with a need to
support legacy drivers can not implement a transitional
device. For example .... ".
> +
> +To have a transitional or legacy virtio device in the guest virtual machine,
> +a PCI PF device as a group owner may
may -> can outside conformance statements.
> support accessing its group member
Pls keep it generic. A group owner ....
> +device's legacy configuration region using the group administration commands
> +listed at \ref{sec:Basic Facilities of a Virtio Device / Device groups / Group
> +administration commands / Legacy Interfaces: legacy configuration access commands}.
> +These legacy configuration access commands are transported via the
> +administration virtqueue.
are these called legacy configuration access commands then?
why do you introduce this term? it does not look like it's used
anywhere else.
I would prefer it if you just use the term "legacy interface"
it is well defined and widely used in the spec.
again leave virtqueue out of it pls we'll have other ways
down the road. what matters is that they are transferred
to a group member by sending legacy configuration access commands
to the group owner.
> This mechanism enables minimal involvement of the
> +hypervisor software only for the purpose of the legacy configuration region access.
Leave marketing out of it pls, kill last sentence.
> +
> +When a guest virtual machine requests legacy configuration and device
> +specific regions
wait so now it's legacy configuration and device specific
regions and these two are distinct?
> of group member device, a hypervisor software accesses
> +it using an administration virtqueue
please talk of administration commands. virtqueue does not matter
at all.
> on behalf of the guest virtual machine.
To put it in our terms, something like this:
when a legacy driver accesses a member device of
a group using the
legacy interface, a modern driver can intercept
the access and forward it to the owner device.
> +
> +Even though virtqueue driver notifications can be communicated through
> +administration virtqueue, if the group member device support such
supports?
> +notifications using a memory-mapped operation, such driver notifications
> +are sent using a group member device defined notification region.
what does "are sent" imply here? is there a matching conformance
statement? who sends them?
> +
> +The group owner device
this is the 1st you mention any group or its owner.
> should not expose PCI BAR0
Not PCI BAR0. That is in header. You mean VF BAR0.
> for the group member
> +devices when it prefers to support legacy interface for legacy configuration
> +access using its group owner.
don't use should outside conformance
I think this specific case actually should be more specific.
Something like:
- For PCI SRIOV groups, when group owner device implements commands
A,B,C the group owner's VF BAR 0 is hardwired to 0
in its PCI SRIOV capability.
> +This facilitates hypervisor software to operate with least amount of
> +complexities
complexity is its own plural
> to emulate the legacy interface I/O space BAR and passthrough
> +other PCI BARs and PCI device capabilities to the guest virtual machine
> +without any translation.
> +
> +The group member device should not expose PCI BAR0 in various PCI capabilities.
> +
> +\devicenormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +
> +When a PCI SR-IOV group owner device supports
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE, VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group owner device SHOULD
> +NOT expose PCI BAR0 in the SR-IOV Extended capability. This is to facilitate the
> +software to emulate I/O region BAR0 for supporting the legacy interface.
not PCI BAR0 - VF BAR0. Check the PCI spec
you can not "not expose it". if you want to
register can be "unimplemented".
Base Address registers are hardwired to zero
But it is better to be specific on what should happen. hardwire VF BAR0 to 0, right?
> +
> +\drivernormative{\subsubsection}{Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}{Virtio Transport Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group Member Device Legacy Configuration Access}
> +
> +The driver SHOULD NOT emulate I/O region BAR0 if a device group member exposes
> +a PCI BAR0.
what does "emulate" mean here? which driver it is?
> diff --git a/transport-pci.tex b/transport-pci.tex
> index a5c6719..3647485 100644
> --- a/transport-pci.tex
> +++ b/transport-pci.tex
> @@ -541,6 +541,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> struct virtio_pci_notify_cap {
> struct virtio_pci_cap cap;
> le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
> + u8 legacy_q_notify_supported;
I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> + u8 reserved[3];
align with spaces pls.
> };
> \end{lstlisting}
>
> @@ -560,6 +562,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> the same Queue Notify address for all queues.
> \end{note}
>
> +\field{legacy_q_notify_supported} when set to 1,
Do you mean something like:
When \field{legacy_q_notify_supported} this indicates that ...
> indicates that the device
> +supports legacy queue notifications at this notification location.
And what location? this refers to what? the location described by
this capability maybe?
More specifically, assume a transitional device (with an IO BAR
as normal). Does presence of this flag in the capability
imply we can use a legacy queue but use this
notification with this capability?
I worry that if we allow that, it opens floodgates of people
who are too lazy to upgrade to 1.0 but just hack
this notification in there.
> Legacy Queue
> +Notification address is derived within a BAR for a virtqueue:
> +
> +\begin{lstlisting}
> + cap.offset
> +\end{lstlisting}
> +
> \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
> The device MUST present at least one notification capability.
>
I don't get what this is saying. I *think* you really mean is
that queue_notify_off is 0.
> @@ -596,6 +606,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> cap.length >= queue_notify_off * notify_off_multiplier + 4
> \end{lstlisting}
>
> +\paragraph{Legacy Interfaces: Notification capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: Notification capability}
> +
> +The device SHOULD set \field{legacy_q_notify_supported}
why SHOULD and not MUST here - if driver checks for it then it won't work without no?
> when the device
> +notification location supports legacy driver notifications.
legacy driver notifications is ambigous.
what you want here and elsewhere is something like
"when used through a legacy interface, device supports driver
> +
> +The driver in the hypervisor MAY use notification location for accessing
> +legacy queue notify address when the device supports it.
> +
> \subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
>
> The VIRTIO_PCI_CAP_ISR_CFG capability
> @@ -1212,3 +1230,5 @@ \subsubsection{Driver Handling Interrupts}\label{sec:Virtio Transport Options /
> re-examine the configuration space to see what changed.
> \end{itemize}
> \end{itemize}
> +
> +\input{transport-pci-legacy-regs.tex}
> --
> 2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 16:20 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:20 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:14PM +0300, Parav Pandit wrote:
> Introduce group member legacy common configuration and legacy device
> configuration access read/write commands.
>
> Group member legacy registers access commands enable group owner driver
> software to access legacy registers on behalf of the guest virtual
> machine.
>
> Usecase:
> ========
> 1. A hypervisor/system needs to provide transitional
> virtio devices to the guest VM at scale of thousands,
> typically, one to eight devices per VM.
>
> 2. A hypervisor/system needs to provide such devices using a
> vendor agnostic driver in the hypervisor system.
>
> 3. A hypervisor system prefers to have single stack regardless of
> virtio device type (net/blk) and be future compatible with a
> single vfio stack using SR-IOV or other scalable device
> virtualization technology to map PCI devices to the guest VM.
> (as transitional or otherwise)
>
> Motivation/Background:
> =====================
> The existing virtio transitional PCI device is missing support for
> PCI SR-IOV based devices. Currently it does not work beyond
> PCI PF, or as software emulated device in reality. Currently it
> has below cited system level limitations:
>
> [a] PCIe spec citation:
> VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
>
> [b] cpu arch citiation:
> Intel 64 and IA-32 Architectures Software Developer’s Manual:
> The processor’s I/O address space is separate and distinct from
> the physical-memory address space. The I/O address space consists
> of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
>
> [c] PCIe spec citation:
> If a bridge implements an I/O address range,...I/O address range will be
> aligned to a 4 KB boundary.
>
> Overview:
> =========
> Above usecase requirements is solved by PCI PF group owner accessing
> its group member PCI VFs legacy registers using an admin virtqueue of
> the group owner PCI PF.
>
> Two types of administration commands are added which read/write PCI VF
> registers.
>
> Software usage example:
> =======================
>
> 1. One way to use and map to the guest VM is by using vfio driver
> framework in Linux kernel.
>
> +----------------------+
> |pci_dev_id = 0x100X |
> +---------------|pci_rev_id = 0x0 |-----+
> |vfio device |BAR0 = I/O region | |
> | |Other attributes | |
> | +----------------------+ |
> | |
> + +--------------+ +-----------------+ |
> | |I/O BAR to AQ | | Other vfio | |
> | |rd/wr mapper& | | functionalities | |
> | | forwarder | | | |
> | +--------------+ +-----------------+ |
> | |
> +------+-------------------------+-----------+
> | |
> Config region |
> access Driver notifications
> | |
> +----+------------+ +----+------------+
> | +-----+ | | PCI VF device A |
> | | AQ |-------------+---->+-------------+ |
> | +-----+ | | | | legacy regs | |
> | PCI PF device | | | +-------------+ |
> +-----------------+ | +-----------------+
> |
> | +----+------------+
> | | PCI VF device N |
> +---->+-------------+ |
> | | legacy regs | |
> | +-------------+ |
> +-----------------+
>
> 2. Continue to use the virtio pci driver to bind to the
> listed device id and use it as in the host.
>
> 3. Use it in a light weight hypervisor to run bare-metal OS.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
> ---
> changelog:
> v5->v6:
> - fixed previous missed abbreviation of LCC and LD
> v4->v5:
> - split from pci transport specific patch
> - split conformance to transport and generic sections
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - rename fields from register to region
> - avoided abbreviation for legacy, device and config
> ---
> admin-cmds-legacy-access.tex | 138 +++++++++++++++++++++++++++++++++++
> admin.tex | 12 ++-
> conformance.tex | 2 +
> 3 files changed, 151 insertions(+), 1 deletion(-)
> create mode 100644 admin-cmds-legacy-access.tex
>
> diff --git a/admin-cmds-legacy-access.tex b/admin-cmds-legacy-access.tex
> new file mode 100644
> index 0000000..89a4f49
> --- /dev/null
> +++ b/admin-cmds-legacy-access.tex
> @@ -0,0 +1,138 @@
> +\subsubsection{Legacy Interfaces: legacy configuration access commands}\label{sec:Basic Facilities of a Virtio Device / Device groups / Group
> +administration commands / Legacy Interfaces: legacy configuration access commands}
I think this is where all the theory of operation text should go.
> +
> +The group owner device can optionally enable the driver to access its member devices legacy common
> +configuration and device configuration region
we can just say "the legacy interface of member
devices"
> using following listed
using the following
> +administrative commands:
administration not administrative
> +
> +\begin{enumerate}
> +\item Legacy Common Configuration Region Write Command
> +\item Legacy common Configuration Region Read Command
> +\item Legacy Device Configuration Region Write Command
> +\item Legacy Device Configuration Region Read Command
> +\end{enumerate}
> +
> +\paragraph{Legacy Common Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group
> +administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Write Command}
> +
> +This command writes in a legacy common configuration region of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +This command uses following structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_common_cfg_wr_data {
> + u8 offset; /* Starting byte offset of the common configuration region to write */
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE.
> +The driver sets valid \field{offset} and associated \field{region_data} bytes to
> +write to the common configuration region.
> +
> +This command does not have any command specific result.
> +
> +\paragraph{Legacy Common Configuration Region Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Read Command}
> +
> +This command reads from a legacy common configuration region of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +This command uses following listed structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_common_cfg_rd_data {
> + u8 offset; /* Starting byte offset of the common configuration region to read */
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ.
> +
> +The driver sets valid \field{offset} of the region from where to read
> +\field{region_data}.
> +
> +When command completes successfully, \field{command_specific_result}
> +uses following listed structure:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_common_cfg_rd_result {
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +\paragraph{Legacy Device Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Write Command}
> +
> +This command writes in a legacy device configuration registers of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +This command uses following structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_ld_reg_wr_data {
> + u8 offset; /* Starting byte offset of the device configuration region to write */
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_WRITE.
> +The driver sets valid \field{offset} and associated \field{region_data} bytes to
> +the device configuration region.
> +
> +This command does not have any command specific result.
> +
> +\paragraph{Legacy Device Registers Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Read Command}
> +
> +This command reads from a legacy device configuration region of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +
> +This command uses following listed structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_dev_cfg_rd_data {
> + u8 offset; /* Starting byte offset of the device configuration region to read */
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_READ.
> +The driver sets valid \field{offset} of the device configurationi region from
> +where to read \field{region_data}.
> +
> +When command completes successfully, \field{command_specific_result}
> +uses following listed structure:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_ld_reg_rd_result {
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +\devicenormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> +
> +If the group owner device supports legacy registers access for its group members,
> +the device MUST set all corresponding bits for commands
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ in
> +the command result of VIRTIO_ADMIN_CMD_LIST_QUERY in
> +\field{device_admin_cmd_opcodes}.
> +
> +The device MUST encode and decode legacy device specific registers using
> +little-endian format.
> +
> +The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands for the invalid offset
> +which is outside the legacy common configuration register's address range.
> +
> +The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE,
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ commands for the invalid offset
> +which is outside the legacy device specific register's address range.
> +
> +\drivernormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> +
> +The driver MUST encode and decode legacy device specific registers using
> +little-endian format.
> +
> +The driver SHOULD send VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands with a valid offset which
> +is in the legacy common configuration region address range.
> +
> +The driver SHOULD send commands VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ with a valid offset which is in the legacy
> +device specific configuration region address range.
> diff --git a/admin.tex b/admin.tex
> index fd3b97d..49a0b6f 100644
> --- a/admin.tex
> +++ b/admin.tex
> @@ -117,7 +117,15 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
> \hline
> 0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\
> \hline
> -0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
> +0x0002 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE & Write legacy common configuration region of a member device \\
> +\hline
> +0x0003 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ & Read legacy common configuration region of a member device \\
> +\hline
> +0x0004 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE & Write legacy device configuration region of a member device \\
> +\hline
> +0x0005 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ & Read legacy device configuration region of a member device \\
> +\hline
> +0x0006 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
> \hline
> 0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\
> \hline
> @@ -286,6 +294,8 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
> supporting multiple group types, the list of supported commands
> might differ between different group types.
>
> +\input{admin-cmds-legacy-access.tex}
> +
> \devicenormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
>
> The device MUST validate \field{opcode}, \field{group_type} and
> diff --git a/conformance.tex b/conformance.tex
> index 01ccd69..6bac4b0 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -260,6 +260,8 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> \item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Layout}
> \item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Endianness}
> \item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Message Framing / Legacy Interface: Message Framing}
> +\item Section \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> +\item Section \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> \item Section \ref{sec:General Initialization And Device Operation / Device Initialization / Legacy Interface: Device Initialization}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery / Legacy Interfaces: A Note on PCI Device Discovery}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
> --
> 2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 16:20 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:20 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:14PM +0300, Parav Pandit wrote:
> Introduce group member legacy common configuration and legacy device
> configuration access read/write commands.
>
> Group member legacy registers access commands enable group owner driver
> software to access legacy registers on behalf of the guest virtual
> machine.
>
> Usecase:
> ========
> 1. A hypervisor/system needs to provide transitional
> virtio devices to the guest VM at scale of thousands,
> typically, one to eight devices per VM.
>
> 2. A hypervisor/system needs to provide such devices using a
> vendor agnostic driver in the hypervisor system.
>
> 3. A hypervisor system prefers to have single stack regardless of
> virtio device type (net/blk) and be future compatible with a
> single vfio stack using SR-IOV or other scalable device
> virtualization technology to map PCI devices to the guest VM.
> (as transitional or otherwise)
>
> Motivation/Background:
> =====================
> The existing virtio transitional PCI device is missing support for
> PCI SR-IOV based devices. Currently it does not work beyond
> PCI PF, or as software emulated device in reality. Currently it
> has below cited system level limitations:
>
> [a] PCIe spec citation:
> VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
>
> [b] cpu arch citiation:
> Intel 64 and IA-32 Architectures Software Developer’s Manual:
> The processor’s I/O address space is separate and distinct from
> the physical-memory address space. The I/O address space consists
> of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
>
> [c] PCIe spec citation:
> If a bridge implements an I/O address range,...I/O address range will be
> aligned to a 4 KB boundary.
>
> Overview:
> =========
> Above usecase requirements is solved by PCI PF group owner accessing
> its group member PCI VFs legacy registers using an admin virtqueue of
> the group owner PCI PF.
>
> Two types of administration commands are added which read/write PCI VF
> registers.
>
> Software usage example:
> =======================
>
> 1. One way to use and map to the guest VM is by using vfio driver
> framework in Linux kernel.
>
> +----------------------+
> |pci_dev_id = 0x100X |
> +---------------|pci_rev_id = 0x0 |-----+
> |vfio device |BAR0 = I/O region | |
> | |Other attributes | |
> | +----------------------+ |
> | |
> + +--------------+ +-----------------+ |
> | |I/O BAR to AQ | | Other vfio | |
> | |rd/wr mapper& | | functionalities | |
> | | forwarder | | | |
> | +--------------+ +-----------------+ |
> | |
> +------+-------------------------+-----------+
> | |
> Config region |
> access Driver notifications
> | |
> +----+------------+ +----+------------+
> | +-----+ | | PCI VF device A |
> | | AQ |-------------+---->+-------------+ |
> | +-----+ | | | | legacy regs | |
> | PCI PF device | | | +-------------+ |
> +-----------------+ | +-----------------+
> |
> | +----+------------+
> | | PCI VF device N |
> +---->+-------------+ |
> | | legacy regs | |
> | +-------------+ |
> +-----------------+
>
> 2. Continue to use the virtio pci driver to bind to the
> listed device id and use it as in the host.
>
> 3. Use it in a light weight hypervisor to run bare-metal OS.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
> ---
> changelog:
> v5->v6:
> - fixed previous missed abbreviation of LCC and LD
> v4->v5:
> - split from pci transport specific patch
> - split conformance to transport and generic sections
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - rename fields from register to region
> - avoided abbreviation for legacy, device and config
> ---
> admin-cmds-legacy-access.tex | 138 +++++++++++++++++++++++++++++++++++
> admin.tex | 12 ++-
> conformance.tex | 2 +
> 3 files changed, 151 insertions(+), 1 deletion(-)
> create mode 100644 admin-cmds-legacy-access.tex
>
> diff --git a/admin-cmds-legacy-access.tex b/admin-cmds-legacy-access.tex
> new file mode 100644
> index 0000000..89a4f49
> --- /dev/null
> +++ b/admin-cmds-legacy-access.tex
> @@ -0,0 +1,138 @@
> +\subsubsection{Legacy Interfaces: legacy configuration access commands}\label{sec:Basic Facilities of a Virtio Device / Device groups / Group
> +administration commands / Legacy Interfaces: legacy configuration access commands}
I think this is where all the theory of operation text should go.
> +
> +The group owner device can optionally enable the driver to access its member devices legacy common
> +configuration and device configuration region
we can just say "the legacy interface of member
devices"
> using following listed
using the following
> +administrative commands:
administration not administrative
> +
> +\begin{enumerate}
> +\item Legacy Common Configuration Region Write Command
> +\item Legacy common Configuration Region Read Command
> +\item Legacy Device Configuration Region Write Command
> +\item Legacy Device Configuration Region Read Command
> +\end{enumerate}
> +
> +\paragraph{Legacy Common Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group
> +administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Write Command}
> +
> +This command writes in a legacy common configuration region of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +This command uses following structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_common_cfg_wr_data {
> + u8 offset; /* Starting byte offset of the common configuration region to write */
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE.
> +The driver sets valid \field{offset} and associated \field{region_data} bytes to
> +write to the common configuration region.
> +
> +This command does not have any command specific result.
> +
> +\paragraph{Legacy Common Configuration Region Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Common Configuration Region Read Command}
> +
> +This command reads from a legacy common configuration region of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +This command uses following listed structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_common_cfg_rd_data {
> + u8 offset; /* Starting byte offset of the common configuration region to read */
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ.
> +
> +The driver sets valid \field{offset} of the region from where to read
> +\field{region_data}.
> +
> +When command completes successfully, \field{command_specific_result}
> +uses following listed structure:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_common_cfg_rd_result {
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +\paragraph{Legacy Device Configuration Region Write Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Write Command}
> +
> +This command writes in a legacy device configuration registers of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +This command uses following structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_ld_reg_wr_data {
> + u8 offset; /* Starting byte offset of the device configuration region to write */
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_WRITE.
> +The driver sets valid \field{offset} and associated \field{region_data} bytes to
> +the device configuration region.
> +
> +This command does not have any command specific result.
> +
> +\paragraph{Legacy Device Registers Read Command}\label{par:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands / Legacy Device Configuration Region Read Command}
> +
> +This command reads from a legacy device configuration region of a group member device.
> +This command follows \field{struct virtio_admin_cmd}.
> +
> +This command uses following listed structure for \field{command_specific_data}:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_legacy_dev_cfg_rd_data {
> + u8 offset; /* Starting byte offset of the device configuration region to read */
> +};
> +\end{lstlisting}
> +
> +The driver sets command \field{opcode} to VIRTIO_ADMIN_CMD_LEGACY_DEV_REG_READ.
> +The driver sets valid \field{offset} of the device configurationi region from
> +where to read \field{region_data}.
> +
> +When command completes successfully, \field{command_specific_result}
> +uses following listed structure:
> +
> +\begin{lstlisting}
> +struct virtio_admin_cmd_ld_reg_rd_result {
> + u8 region_data[];
> +};
> +\end{lstlisting}
> +
> +\devicenormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> +
> +If the group owner device supports legacy registers access for its group members,
> +the device MUST set all corresponding bits for commands
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ in
> +the command result of VIRTIO_ADMIN_CMD_LIST_QUERY in
> +\field{device_admin_cmd_opcodes}.
> +
> +The device MUST encode and decode legacy device specific registers using
> +little-endian format.
> +
> +The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands for the invalid offset
> +which is outside the legacy common configuration register's address range.
> +
> +The device MUST fail VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE,
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ commands for the invalid offset
> +which is outside the legacy device specific register's address range.
> +
> +\drivernormative{\paragraph}{Legacy Interfaces: legacy configuration access commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> +
> +The driver MUST encode and decode legacy device specific registers using
> +little-endian format.
> +
> +The driver SHOULD send VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE and
> +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ commands with a valid offset which
> +is in the legacy common configuration region address range.
> +
> +The driver SHOULD send commands VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE and
> +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ with a valid offset which is in the legacy
> +device specific configuration region address range.
> diff --git a/admin.tex b/admin.tex
> index fd3b97d..49a0b6f 100644
> --- a/admin.tex
> +++ b/admin.tex
> @@ -117,7 +117,15 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
> \hline
> 0x0001 & VIRTIO_ADMIN_CMD_LIST_USE & Provides to device list of commands used for this group type \\
> \hline
> -0x0002 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
> +0x0002 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE & Write legacy common configuration region of a member device \\
> +\hline
> +0x0003 & VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ & Read legacy common configuration region of a member device \\
> +\hline
> +0x0004 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE & Write legacy device configuration region of a member device \\
> +\hline
> +0x0005 & VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ & Read legacy device configuration region of a member device \\
> +\hline
> +0x0006 - 0x7FFF & - & Commands using \field{struct virtio_admin_cmd} \\
> \hline
> 0x8000 - 0xFFFF & - & Reserved for future commands (possibly using a different structure) \\
> \hline
> @@ -286,6 +294,8 @@ \subsection{Group administration commands}\label{sec:Basic Facilities of a Virti
> supporting multiple group types, the list of supported commands
> might differ between different group types.
>
> +\input{admin-cmds-legacy-access.tex}
> +
> \devicenormative{\subsubsection}{Group administration commands}{Basic Facilities of a Virtio Device / Device groups / Group administration commands}
>
> The device MUST validate \field{opcode}, \field{group_type} and
> diff --git a/conformance.tex b/conformance.tex
> index 01ccd69..6bac4b0 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -260,6 +260,8 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> \item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Layout}
> \item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Legacy Interfaces: A Note on Virtqueue Endianness}
> \item Section \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Message Framing / Legacy Interface: Message Framing}
> +\item Section \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> +\item Section \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Legacy Interfaces: legacy configuration access commands}
> \item Section \ref{sec:General Initialization And Device Operation / Device Initialization / Legacy Interface: Device Initialization}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery / Legacy Interfaces: A Note on PCI Device Discovery}
> \item Section \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Legacy Interfaces: A Note on PCI Device Layout}
> --
> 2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 16:28 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:28 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:11PM +0300, Parav Pandit wrote:
> This short series introduces legacy registers access commands for the owner
> group member PCI PF to access the legacy registers of the member VFs.
>
> If in future any SIOV devices to support legacy registers, they
> can be easily supported using same commands by using the group
> member identifiers of the future SIOV devices.
>
> More details as overview, motivation, use case are further described
> below.
>
> Patch summary:
> --------------
> patch-1 split rows of admin opcode tables by a line
> patch-2 adds administrative virtuqueue commands
> patch-3 adds its conformance section and links
numbering seems to have changed?
> It uses the newly introduced administrative virtqueue facility with 4 new
> commands which uses the existing virtio_admin_cmd.
>
> Usecase:
> --------
> 1. A hypervisor/system needs to provide transitional
> virtio devices to the guest VM at scale of thousands,
> typically, one to eight devices per VM.
>
> 2. A hypervisor/system needs to provide such devices using a
> vendor agnostic driver in the hypervisor system.
>
> 3. A hypervisor system prefers to have single stack regardless of
> virtio device type (net/blk) and be future compatible with a
> single vfio stack using SR-IOV or other scalable device
> virtualization technology to map PCI devices to the guest VM.
> (as transitional or otherwise)
>
> Motivation/Background:
> ----------------------
> The existing virtio transitional PCI device is missing support for
> PCI SR-IOV based devices. Currently it does not work beyond
> PCI PF, or as software emulated device in reality. Currently it
> has below cited system level limitations:
>
> [a] PCIe spec citation:
> VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
>
> [b] cpu arch citiation:
> Intel 64 and IA-32 Architectures Software Developer’s Manual:
> The processor’s I/O address space is separate and distinct from
> the physical-memory address space. The I/O address space consists
> of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
>
> [c] PCIe spec citation:
> If a bridge implements an I/O address range,...I/O address range will be
> aligned to a 4 KB boundary.
>
> Overview:
> ---------
> Above usecase requirements is solved by PCI PF group owner accessing
> its group member PCI VFs legacy registers using an admin virtqueue of
> the group owner PCI PF.
>
> Two new admin virtqueue commands are added which read/write PCI VF
> registers.
>
> Software usage example:
> -----------------------
> One way to use and map to the guest VM is by using vfio driver
> framework in Linux kernel.
>
> +----------------------+
> |pci_dev_id = 0x100X |
> +---------------|pci_rev_id = 0x0 |-----+
> |vfio device |BAR0 = I/O region | |
> | |Other attributes | |
> | +----------------------+ |
> | |
> + +--------------+ +-----------------+ |
> | |I/O BAR to AQ | | Other vfio | |
> | |rd/wr mapper | | functionalities | |
> | +--------------+ +-----------------+ |
> | |
> +------+-------------------------+-----------+
> | |
> | Driver notification
> | |
> | |
> +----+------------+ +----+------------+
> | +-----+ | | PCI VF device A |
> | | AQ |-------------+---->+-------------+ |
> | +-----+ | | | | legacy regs | |
> | PCI PF device | | | +-------------+ |
> +-----------------+ | +-----------------+
> |
> | +----+------------+
> | | PCI VF device N |
> +---->+-------------+ |
> | | legacy regs | |
> | +-------------+ |
> +-----------------+
>
> 2. Virtio pci driver to bind to the listed device id and
> use it in the host.
>
> 3. Use it in a light weight hypervisor to run bare-metal OS.
>
> Please review.
>
> Alternatives considered:
> ========================
> 1. Exposing BAR0 as MMIO BAR that follows legacy registers template
> Pros:
> a. Kind of works with legacy drivers as some of them have used API
> which is agnostic to MMIO vs IOBAR.
> b. Does not require hypervisor intervantion
> Cons:
> a. Device reset is extremely hard to implement in device at scale as
> driver does not wait for device reset completion
> b. Device register width related problems persist that hypervisor if
> wishes, it cannot be fixed.
>
> 2. Accessing VF registers by tunneling it through new legacy PCI capability
> Pros:
> a. Self contained, but cannot work with future PCI SIOV devices
> Cons:
> a. Equally slow as AQ access
> b. Still requires new capability for notification access
> c. Requires hardware to build low level registers access which is not worth
> for long term future
>
> 3. Accessing VF notification region using new PF BAR
> Cons:
> a. Requires hardware to build new PCI steering logic per PF to forward
> notification from the PF to VF, requires double the amount of logic
> compared to today
> b. Requires very large additional PF BAR whose size must be max_Vfs * BAR size.
>
> 4. Trapping CVQ, configuration region, LEGACY_HDR
> Cons:
> a. This does not fullfil the very basic requirement to not trap the
> 1.x objects (configuration registers, vqs)
> b. Requires feature negotiations mediation in hypervisor software
> c. Requires constant device type specific knowledge in hypervisor driver
> (Does not scale for 30+ device types)
>
> conclusion for picking AQ approach:
> ==================================
> 1. Overall AQ based access is simpler to implement with combination of
> best from software and device so that legacy registers do not get baked
> in the device hardware
> 2. AQ allows hypervisor software to intercept legacy registers and make
> corrections if needed
> 3. Provides trade-off between performance, device complexity vs spec,
> while still maintaining passthrough mode for the VFs with minimal
> hypervisor intercepts only for legacy registers access
> 4. AQ mechanism is designed for accessing other member devices registers
> as noted in AQ submission, it utilizes the existing infrastructure over
> other alternatives.
> 5. Uses existing driver notification region similar to legacy notification
> saves hardware resources
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
>
> ---
> changelog:
> v5->v6:
> - fixed previous missed abbreviation of LCC and LD
> - added text for the PCI capability for the group member device
> v4->v5:
> - split pci transport and generic command section to new patch
> - removed multiple references to the VF
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - split from pci transport specific patch
> - split conformance to transport and generic sections
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - rename fields from register to region
> - avoided abbreviation for legacy, device and config
> v3->v4:
> - moved noted to the conformance section details in next patch
> - removed queue notify address query AQ command on Michael's suggestion,
> though it is fine. Instead replaced with extending virtio_pci_notify_cap
> to indicate that legacy queue notifications can be done on the
> notification location
> - fixed spelling errors
> - replaced administrative virtqueue to administration virtqueue
> - moved legacy interface normative references to legacy conformance
> section
> v2->v3:
> - added new patch to split raws of admin vq opcode table
> - adddressed Jason and Michael's comment to split single register
> access command to common config and device specific commands.
> - dropped the suggetion to introduce enable/disable command as
> admin command cap bit already covers it.
> - added other alternative design considered and discussed in detail in v0, v1 and v2
> v1->v2:
> - addressed comments from Michael
> - added theory of operation
> - grammar corrections
> - removed group fields description from individual commands as
> it is already present in generic section
> - added endianness normative for legacy device registers region
> - renamed the file to drop vf and add legacy prefix
> - added overview in commit log
> - renamed subsection to reflect command
> v0->v1:
> - addressed comments, suggesetions and ideas from Michael Tsirkin and Jason Wang
> - far more simpler design than MMR access
> - removed complexities of MMR device ids
> - removed complexities of MMR registers and extended capabilities
> - dropped adding new extended capabilities because if if they are
> added, a pci device still needs to have existing capabilities
> in the legacy configuration space and hypervisor driver do not
> need to access them
>
>
> Parav Pandit (4):
> admin: Split opcode table rows with a line
> admin: Fix section numbering
> admin: Add group member legacy register access commands
> transport-pci: Introduce group legacy group member config region
> access
>
> admin-cmds-legacy-access.tex | 138 ++++++++++++++++++++++++++++++++++
> admin.tex | 18 ++++-
> conformance.tex | 5 ++
> transport-pci-legacy-regs.tex | 48 ++++++++++++
> transport-pci.tex | 20 +++++
> 5 files changed, 226 insertions(+), 3 deletions(-)
> create mode 100644 admin-cmds-legacy-access.tex
> create mode 100644 transport-pci-legacy-regs.tex
>
> --
> 2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ
@ 2023-06-19 16:28 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:28 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:11PM +0300, Parav Pandit wrote:
> This short series introduces legacy registers access commands for the owner
> group member PCI PF to access the legacy registers of the member VFs.
>
> If in future any SIOV devices to support legacy registers, they
> can be easily supported using same commands by using the group
> member identifiers of the future SIOV devices.
>
> More details as overview, motivation, use case are further described
> below.
>
> Patch summary:
> --------------
> patch-1 split rows of admin opcode tables by a line
> patch-2 adds administrative virtuqueue commands
> patch-3 adds its conformance section and links
numbering seems to have changed?
> It uses the newly introduced administrative virtqueue facility with 4 new
> commands which uses the existing virtio_admin_cmd.
>
> Usecase:
> --------
> 1. A hypervisor/system needs to provide transitional
> virtio devices to the guest VM at scale of thousands,
> typically, one to eight devices per VM.
>
> 2. A hypervisor/system needs to provide such devices using a
> vendor agnostic driver in the hypervisor system.
>
> 3. A hypervisor system prefers to have single stack regardless of
> virtio device type (net/blk) and be future compatible with a
> single vfio stack using SR-IOV or other scalable device
> virtualization technology to map PCI devices to the guest VM.
> (as transitional or otherwise)
>
> Motivation/Background:
> ----------------------
> The existing virtio transitional PCI device is missing support for
> PCI SR-IOV based devices. Currently it does not work beyond
> PCI PF, or as software emulated device in reality. Currently it
> has below cited system level limitations:
>
> [a] PCIe spec citation:
> VFs do not support I/O Space and thus VF BARs shall not indicate I/O Space.
>
> [b] cpu arch citiation:
> Intel 64 and IA-32 Architectures Software Developer’s Manual:
> The processor’s I/O address space is separate and distinct from
> the physical-memory address space. The I/O address space consists
> of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
>
> [c] PCIe spec citation:
> If a bridge implements an I/O address range,...I/O address range will be
> aligned to a 4 KB boundary.
>
> Overview:
> ---------
> Above usecase requirements is solved by PCI PF group owner accessing
> its group member PCI VFs legacy registers using an admin virtqueue of
> the group owner PCI PF.
>
> Two new admin virtqueue commands are added which read/write PCI VF
> registers.
>
> Software usage example:
> -----------------------
> One way to use and map to the guest VM is by using vfio driver
> framework in Linux kernel.
>
> +----------------------+
> |pci_dev_id = 0x100X |
> +---------------|pci_rev_id = 0x0 |-----+
> |vfio device |BAR0 = I/O region | |
> | |Other attributes | |
> | +----------------------+ |
> | |
> + +--------------+ +-----------------+ |
> | |I/O BAR to AQ | | Other vfio | |
> | |rd/wr mapper | | functionalities | |
> | +--------------+ +-----------------+ |
> | |
> +------+-------------------------+-----------+
> | |
> | Driver notification
> | |
> | |
> +----+------------+ +----+------------+
> | +-----+ | | PCI VF device A |
> | | AQ |-------------+---->+-------------+ |
> | +-----+ | | | | legacy regs | |
> | PCI PF device | | | +-------------+ |
> +-----------------+ | +-----------------+
> |
> | +----+------------+
> | | PCI VF device N |
> +---->+-------------+ |
> | | legacy regs | |
> | +-------------+ |
> +-----------------+
>
> 2. Virtio pci driver to bind to the listed device id and
> use it in the host.
>
> 3. Use it in a light weight hypervisor to run bare-metal OS.
>
> Please review.
>
> Alternatives considered:
> ========================
> 1. Exposing BAR0 as MMIO BAR that follows legacy registers template
> Pros:
> a. Kind of works with legacy drivers as some of them have used API
> which is agnostic to MMIO vs IOBAR.
> b. Does not require hypervisor intervantion
> Cons:
> a. Device reset is extremely hard to implement in device at scale as
> driver does not wait for device reset completion
> b. Device register width related problems persist that hypervisor if
> wishes, it cannot be fixed.
>
> 2. Accessing VF registers by tunneling it through new legacy PCI capability
> Pros:
> a. Self contained, but cannot work with future PCI SIOV devices
> Cons:
> a. Equally slow as AQ access
> b. Still requires new capability for notification access
> c. Requires hardware to build low level registers access which is not worth
> for long term future
>
> 3. Accessing VF notification region using new PF BAR
> Cons:
> a. Requires hardware to build new PCI steering logic per PF to forward
> notification from the PF to VF, requires double the amount of logic
> compared to today
> b. Requires very large additional PF BAR whose size must be max_Vfs * BAR size.
>
> 4. Trapping CVQ, configuration region, LEGACY_HDR
> Cons:
> a. This does not fullfil the very basic requirement to not trap the
> 1.x objects (configuration registers, vqs)
> b. Requires feature negotiations mediation in hypervisor software
> c. Requires constant device type specific knowledge in hypervisor driver
> (Does not scale for 30+ device types)
>
> conclusion for picking AQ approach:
> ==================================
> 1. Overall AQ based access is simpler to implement with combination of
> best from software and device so that legacy registers do not get baked
> in the device hardware
> 2. AQ allows hypervisor software to intercept legacy registers and make
> corrections if needed
> 3. Provides trade-off between performance, device complexity vs spec,
> while still maintaining passthrough mode for the VFs with minimal
> hypervisor intercepts only for legacy registers access
> 4. AQ mechanism is designed for accessing other member devices registers
> as noted in AQ submission, it utilizes the existing infrastructure over
> other alternatives.
> 5. Uses existing driver notification region similar to legacy notification
> saves hardware resources
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/167
> Signed-off-by: Parav Pandit <parav@nvidia.com>
>
> ---
> changelog:
> v5->v6:
> - fixed previous missed abbreviation of LCC and LD
> - added text for the PCI capability for the group member device
> v4->v5:
> - split pci transport and generic command section to new patch
> - removed multiple references to the VF
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - split from pci transport specific patch
> - split conformance to transport and generic sections
> - written the description of the command as generic with member
> and group device terminology
> - reflected many section names to remove VF
> - rename fields from register to region
> - avoided abbreviation for legacy, device and config
> v3->v4:
> - moved noted to the conformance section details in next patch
> - removed queue notify address query AQ command on Michael's suggestion,
> though it is fine. Instead replaced with extending virtio_pci_notify_cap
> to indicate that legacy queue notifications can be done on the
> notification location
> - fixed spelling errors
> - replaced administrative virtqueue to administration virtqueue
> - moved legacy interface normative references to legacy conformance
> section
> v2->v3:
> - added new patch to split raws of admin vq opcode table
> - adddressed Jason and Michael's comment to split single register
> access command to common config and device specific commands.
> - dropped the suggetion to introduce enable/disable command as
> admin command cap bit already covers it.
> - added other alternative design considered and discussed in detail in v0, v1 and v2
> v1->v2:
> - addressed comments from Michael
> - added theory of operation
> - grammar corrections
> - removed group fields description from individual commands as
> it is already present in generic section
> - added endianness normative for legacy device registers region
> - renamed the file to drop vf and add legacy prefix
> - added overview in commit log
> - renamed subsection to reflect command
> v0->v1:
> - addressed comments, suggesetions and ideas from Michael Tsirkin and Jason Wang
> - far more simpler design than MMR access
> - removed complexities of MMR device ids
> - removed complexities of MMR registers and extended capabilities
> - dropped adding new extended capabilities because if if they are
> added, a pci device still needs to have existing capabilities
> in the legacy configuration space and hypervisor driver do not
> need to access them
>
>
> Parav Pandit (4):
> admin: Split opcode table rows with a line
> admin: Fix section numbering
> admin: Add group member legacy register access commands
> transport-pci: Introduce group legacy group member config region
> access
>
> admin-cmds-legacy-access.tex | 138 ++++++++++++++++++++++++++++++++++
> admin.tex | 18 ++++-
> conformance.tex | 5 ++
> transport-pci-legacy-regs.tex | 48 ++++++++++++
> transport-pci.tex | 20 +++++
> 5 files changed, 226 insertions(+), 3 deletions(-)
> create mode 100644 admin-cmds-legacy-access.tex
> create mode 100644 transport-pci-legacy-regs.tex
>
> --
> 2.26.2
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 16:20 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 16:29 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 16:29 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:21 PM
> > diff --git a/admin-cmds-legacy-access.tex
> > b/admin-cmds-legacy-access.tex new file mode 100644 index
> > 0000000..89a4f49
> > --- /dev/null
> > +++ b/admin-cmds-legacy-access.tex
> > @@ -0,0 +1,138 @@
> > +\subsubsection{Legacy Interfaces: legacy configuration access
> > +commands}\label{sec:Basic Facilities of a Virtio Device / Device
> > +groups / Group administration commands / Legacy Interfaces: legacy
> > +configuration access commands}
>
> I think this is where all the theory of operation text should go.
>
This is where most of the theory of operation is documented which is agnostic of the transport.
PCI specific things are left in pci transport section.
> > +
> > +The group owner device can optionally enable the driver to access its
> > +member devices legacy common configuration and device configuration
> > +region
>
> we can just say "the legacy interface of member devices"
>
What is the harm in telling explicitly what is currently supported, specially when it theory of operation description section.
> > using following listed
>
> using the following
>
> > +administrative commands:
>
> administration not administrative
>
Ack.
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 16:29 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 16:29 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:21 PM
> > diff --git a/admin-cmds-legacy-access.tex
> > b/admin-cmds-legacy-access.tex new file mode 100644 index
> > 0000000..89a4f49
> > --- /dev/null
> > +++ b/admin-cmds-legacy-access.tex
> > @@ -0,0 +1,138 @@
> > +\subsubsection{Legacy Interfaces: legacy configuration access
> > +commands}\label{sec:Basic Facilities of a Virtio Device / Device
> > +groups / Group administration commands / Legacy Interfaces: legacy
> > +configuration access commands}
>
> I think this is where all the theory of operation text should go.
>
This is where most of the theory of operation is documented which is agnostic of the transport.
PCI specific things are left in pci transport section.
> > +
> > +The group owner device can optionally enable the driver to access its
> > +member devices legacy common configuration and device configuration
> > +region
>
> we can just say "the legacy interface of member devices"
>
What is the harm in telling explicitly what is currently supported, specially when it theory of operation description section.
> > using following listed
>
> using the following
>
> > +administrative commands:
>
> administration not administrative
>
Ack.
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 16:37 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:37 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> --- a/transport-pci.tex
> +++ b/transport-pci.tex
> @@ -541,6 +541,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> struct virtio_pci_notify_cap {
> struct virtio_pci_cap cap;
> le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
> + u8 legacy_q_notify_supported;
> + u8 reserved[3];
> };
> \end{lstlisting}
>
> @@ -560,6 +562,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> the same Queue Notify address for all queues.
> \end{note}
>
> +\field{legacy_q_notify_supported} when set to 1, indicates that the device
> +supports legacy queue notifications at this notification location. Legacy Queue
> +Notification address is derived within a BAR for a virtqueue:
> +
> +\begin{lstlisting}
> + cap.offset
> +\end{lstlisting}
> +
> \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
> The device MUST present at least one notification capability.
>
I was hoping that we can avoid touching transport-pci.tex completely,
this is why I wanted to reuse VIRTIO_PCI_CAP_NOTIFY_CFG.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 16:37 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:37 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> --- a/transport-pci.tex
> +++ b/transport-pci.tex
> @@ -541,6 +541,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> struct virtio_pci_notify_cap {
> struct virtio_pci_cap cap;
> le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
> + u8 legacy_q_notify_supported;
> + u8 reserved[3];
> };
> \end{lstlisting}
>
> @@ -560,6 +562,14 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> the same Queue Notify address for all queues.
> \end{note}
>
> +\field{legacy_q_notify_supported} when set to 1, indicates that the device
> +supports legacy queue notifications at this notification location. Legacy Queue
> +Notification address is derived within a BAR for a virtqueue:
> +
> +\begin{lstlisting}
> + cap.offset
> +\end{lstlisting}
> +
> \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
> The device MUST present at least one notification capability.
>
I was hoping that we can avoid touching transport-pci.tex completely,
this is why I wanted to reuse VIRTIO_PCI_CAP_NOTIFY_CFG.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 16:37 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 16:39 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 16:39 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:38 PM
> >
> > +\field{legacy_q_notify_supported} when set to 1, indicates that the
> > +device supports legacy queue notifications at this notification
> > +location. Legacy Queue Notification address is derived within a BAR for a
> virtqueue:
> > +
> > +\begin{lstlisting}
> > + cap.offset
> > +\end{lstlisting}
> > +
> > \devicenormative{\paragraph}{Notification capability}{Virtio
> > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> capability} The device MUST present at least one notification capability.
> >
>
> I was hoping that we can avoid touching transport-pci.tex completely, this is
> why I wanted to reuse VIRTIO_PCI_CAP_NOTIFY_CFG.
It is reused, but it doesnt come for free.
Do you want to re-consider AQ notify query command?
Or continue as proposed in v6?
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 16:39 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 16:39 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:38 PM
> >
> > +\field{legacy_q_notify_supported} when set to 1, indicates that the
> > +device supports legacy queue notifications at this notification
> > +location. Legacy Queue Notification address is derived within a BAR for a
> virtqueue:
> > +
> > +\begin{lstlisting}
> > + cap.offset
> > +\end{lstlisting}
> > +
> > \devicenormative{\paragraph}{Notification capability}{Virtio
> > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> capability} The device MUST present at least one notification capability.
> >
>
> I was hoping that we can avoid touching transport-pci.tex completely, this is
> why I wanted to reuse VIRTIO_PCI_CAP_NOTIFY_CFG.
It is reused, but it doesnt come for free.
Do you want to re-consider AQ notify query command?
Or continue as proposed in v6?
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 16:29 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 16:40 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:40 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 04:29:43PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:21 PM
>
> > > diff --git a/admin-cmds-legacy-access.tex
> > > b/admin-cmds-legacy-access.tex new file mode 100644 index
> > > 0000000..89a4f49
> > > --- /dev/null
> > > +++ b/admin-cmds-legacy-access.tex
> > > @@ -0,0 +1,138 @@
> > > +\subsubsection{Legacy Interfaces: legacy configuration access
> > > +commands}\label{sec:Basic Facilities of a Virtio Device / Device
> > > +groups / Group administration commands / Legacy Interfaces: legacy
> > > +configuration access commands}
> >
> > I think this is where all the theory of operation text should go.
> >
> This is where most of the theory of operation is documented which is agnostic of the transport.
> PCI specific things are left in pci transport section.
what I am saying is that it's cleaner here at least for now.
> > > +
> > > +The group owner device can optionally enable the driver to access its
> > > +member devices legacy common configuration and device configuration
> > > +region
> >
> > we can just say "the legacy interface of member devices"
> >
> What is the harm in telling explicitly what is currently supported, specially when it theory of operation description section.
The harm is that
- making things generic on the 1st try is the only stick we have
as contributors always push to make their solutions as
specific to their needs as possible
- people are lazy and they will copy text and then
I get to repeat this argument all over again.
If you want to also have a type specific text, be my guest
"specifically, for an SRIOV type" ....
> > > using following listed
> >
> > using the following
> >
> > > +administrative commands:
> >
> > administration not administrative
> >
> Ack.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 16:40 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 16:40 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 04:29:43PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:21 PM
>
> > > diff --git a/admin-cmds-legacy-access.tex
> > > b/admin-cmds-legacy-access.tex new file mode 100644 index
> > > 0000000..89a4f49
> > > --- /dev/null
> > > +++ b/admin-cmds-legacy-access.tex
> > > @@ -0,0 +1,138 @@
> > > +\subsubsection{Legacy Interfaces: legacy configuration access
> > > +commands}\label{sec:Basic Facilities of a Virtio Device / Device
> > > +groups / Group administration commands / Legacy Interfaces: legacy
> > > +configuration access commands}
> >
> > I think this is where all the theory of operation text should go.
> >
> This is where most of the theory of operation is documented which is agnostic of the transport.
> PCI specific things are left in pci transport section.
what I am saying is that it's cleaner here at least for now.
> > > +
> > > +The group owner device can optionally enable the driver to access its
> > > +member devices legacy common configuration and device configuration
> > > +region
> >
> > we can just say "the legacy interface of member devices"
> >
> What is the harm in telling explicitly what is currently supported, specially when it theory of operation description section.
The harm is that
- making things generic on the 1st try is the only stick we have
as contributors always push to make their solutions as
specific to their needs as possible
- people are lazy and they will copy text and then
I get to repeat this argument all over again.
If you want to also have a type specific text, be my guest
"specifically, for an SRIOV type" ....
> > > using following listed
> >
> > using the following
> >
> > > +administrative commands:
> >
> > administration not administrative
> >
> Ack.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 16:40 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 16:45 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 16:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:40 PM
> > >
> > > we can just say "the legacy interface of member devices"
> > >
> > What is the harm in telling explicitly what is currently supported, specially
> when it theory of operation description section.
>
> The harm is that
> - making things generic on the 1st try is the only stick we have
> as contributors always push to make their solutions as
> specific to their needs as possible
> - people are lazy and they will copy text and then I get to repeat this argument
> all over again.
>
The "legacy interface of member device" is a vague term. As implementer, reader I don't understand what that means.
The commands clearly communicate the scope.
> If you want to also have a type specific text, be my guest "specifically, for an
> SRIOV type" ....
>
SRIOV in generic section doesn't make sense. It is covered in the pci transport section.
>
> > > > using following listed
> > >
> > > using the following
> > >
> > > > +administrative commands:
> > >
> > > administration not administrative
> > >
> > Ack.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 16:45 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 16:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:40 PM
> > >
> > > we can just say "the legacy interface of member devices"
> > >
> > What is the harm in telling explicitly what is currently supported, specially
> when it theory of operation description section.
>
> The harm is that
> - making things generic on the 1st try is the only stick we have
> as contributors always push to make their solutions as
> specific to their needs as possible
> - people are lazy and they will copy text and then I get to repeat this argument
> all over again.
>
The "legacy interface of member device" is a vague term. As implementer, reader I don't understand what that means.
The commands clearly communicate the scope.
> If you want to also have a type specific text, be my guest "specifically, for an
> SRIOV type" ....
>
SRIOV in generic section doesn't make sense. It is covered in the pci transport section.
>
> > > > using following listed
> > >
> > > using the following
> > >
> > > > +administrative commands:
> > >
> > > administration not administrative
> > >
> > Ack.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:04 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:04 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> +Even though virtqueue driver notifications can be communicated through
> +administration virtqueue, if the group member device support such
> +notifications using a memory-mapped operation, such driver notifications
> +are sent using a group member device defined notification region.
I still feel we should support sending notifications to owner
at least optionally. For example:
\begin{lstlisting}
struct virtio_pci_owner_legacy_notify_cap {
struct virtio_pci_cap cap;
le32 notify_off_multiplier; /* Multiplier for member id. */
};
\end{lstlisting}
\field{notify_off_multiplier} is combined with the \field{member id} to
derive an address within a BAR.
\begin{lstlisting}
cap.offset + member_id * notify_off_multiplier
\end{lstlisting}
We also need, as part of theory of operation, text that reads
something like "accessing the member device legacy interface
using one of XYZ commands has the same effect as accessing
it using the legacy interface".
Also, do all VFs support legacy interface with these
commands or none at all?
Also these devices will use non-transitional ID but they in fact
do have a legacy interface so using this definition they are
transitional devices. Maybe we need to add
when we describe the device ID text like "non transitional
devices and transitional devices utilizing commands XYZ" ...?
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:04 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:04 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, yishaih, maorg, shahafs
On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> +Even though virtqueue driver notifications can be communicated through
> +administration virtqueue, if the group member device support such
> +notifications using a memory-mapped operation, such driver notifications
> +are sent using a group member device defined notification region.
I still feel we should support sending notifications to owner
at least optionally. For example:
\begin{lstlisting}
struct virtio_pci_owner_legacy_notify_cap {
struct virtio_pci_cap cap;
le32 notify_off_multiplier; /* Multiplier for member id. */
};
\end{lstlisting}
\field{notify_off_multiplier} is combined with the \field{member id} to
derive an address within a BAR.
\begin{lstlisting}
cap.offset + member_id * notify_off_multiplier
\end{lstlisting}
We also need, as part of theory of operation, text that reads
something like "accessing the member device legacy interface
using one of XYZ commands has the same effect as accessing
it using the legacy interface".
Also, do all VFs support legacy interface with these
commands or none at all?
Also these devices will use non-transitional ID but they in fact
do have a legacy interface so using this definition they are
transitional devices. Maybe we need to add
when we describe the device ID text like "non transitional
devices and transitional devices utilizing commands XYZ" ...?
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 16:45 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:10 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:10 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 04:45:23PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:40 PM
> > > >
> > > > we can just say "the legacy interface of member devices"
> > > >
> > > What is the harm in telling explicitly what is currently supported, specially
> > when it theory of operation description section.
> >
> > The harm is that
> > - making things generic on the 1st try is the only stick we have
> > as contributors always push to make their solutions as
> > specific to their needs as possible
> > - people are lazy and they will copy text and then I get to repeat this argument
> > all over again.
> >
>
> The "legacy interface of member device" is a vague term. As implementer, reader I don't understand what that means.
> The commands clearly communicate the scope.
Add this as an example then.
> > If you want to also have a type specific text, be my guest "specifically, for an
> > SRIOV type" ....
> >
> SRIOV in generic section doesn't make sense. It is covered in the pci transport section.
When it's convenient to you you argue that it's clearer, when it's not
you want it generic. Just mention it quickly and link to the specific
text in the new section.
Look the only reason I personally am even entertaining this over
objections of several members is that this stuff has the potentional to
be well contained. That's the only argument, otherwise individual
features like _F_LEGACY_MAC would win hands down - this is how we always
did it. If you are intent on spreading this all over the spec
we'll need to rethink the whole approach.
> >
> > > > > using following listed
> > > >
> > > > using the following
> > > >
> > > > > +administrative commands:
> > > >
> > > > administration not administrative
> > > >
> > > Ack.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 17:10 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:10 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 04:45:23PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:40 PM
> > > >
> > > > we can just say "the legacy interface of member devices"
> > > >
> > > What is the harm in telling explicitly what is currently supported, specially
> > when it theory of operation description section.
> >
> > The harm is that
> > - making things generic on the 1st try is the only stick we have
> > as contributors always push to make their solutions as
> > specific to their needs as possible
> > - people are lazy and they will copy text and then I get to repeat this argument
> > all over again.
> >
>
> The "legacy interface of member device" is a vague term. As implementer, reader I don't understand what that means.
> The commands clearly communicate the scope.
Add this as an example then.
> > If you want to also have a type specific text, be my guest "specifically, for an
> > SRIOV type" ....
> >
> SRIOV in generic section doesn't make sense. It is covered in the pci transport section.
When it's convenient to you you argue that it's clearer, when it's not
you want it generic. Just mention it quickly and link to the specific
text in the new section.
Look the only reason I personally am even entertaining this over
objections of several members is that this stuff has the potentional to
be well contained. That's the only argument, otherwise individual
features like _F_LEGACY_MAC would win hands down - this is how we always
did it. If you are intent on spreading this all over the spec
we'll need to rethink the whole approach.
> >
> > > > > using following listed
> > > >
> > > > using the following
> > > >
> > > > > +administrative commands:
> > > >
> > > > administration not administrative
> > > >
> > > Ack.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:04 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 17:11 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:11 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> Behalf Of Michael S. Tsirkin
> Sent: Monday, June 19, 2023 1:04 PM
> On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> > +Even though virtqueue driver notifications can be communicated
> > +through administration virtqueue, if the group member device support
> > +such notifications using a memory-mapped operation, such driver
> > +notifications are sent using a group member device defined notification
> region.
>
> I still feel we should support sending notifications to owner at least optionally.
> For example:
>
Let's not please keep discussing this yet again.
I won't be able to.
We have captured this alternative already in the cover letter in the alternatives section.
This optional capability can be added when there is a _real_ implementation by some hw.
>
> \begin{lstlisting}
> struct virtio_pci_owner_legacy_notify_cap {
> struct virtio_pci_cap cap;
> le32 notify_off_multiplier; /* Multiplier for member id. */ }; \end{lstlisting}
>
> \field{notify_off_multiplier} is combined with the \field{member id} to derive
> an address within a BAR.
>
> \begin{lstlisting}
> cap.offset + member_id * notify_off_multiplier \end{lstlisting}
>
>
>
> We also need, as part of theory of operation, text that reads something like
> "accessing the member device legacy interface using one of XYZ commands has
> the same effect as accessing it using the legacy interface".
>
Ack. I will add this.
>
>
> Also, do all VFs support legacy interface with these commands or none at all?
>
Right.
>
> Also these devices will use non-transitional ID but they in fact do have a legacy
> interface so using this definition they are transitional devices. Maybe we need
> to add when we describe the device ID text like "non transitional devices and
> transitional devices utilizing commands XYZ" ...?
Transitional device has specific meaning, I am not sure we should muddy it.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:11 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:11 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> Behalf Of Michael S. Tsirkin
> Sent: Monday, June 19, 2023 1:04 PM
> On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> > +Even though virtqueue driver notifications can be communicated
> > +through administration virtqueue, if the group member device support
> > +such notifications using a memory-mapped operation, such driver
> > +notifications are sent using a group member device defined notification
> region.
>
> I still feel we should support sending notifications to owner at least optionally.
> For example:
>
Let's not please keep discussing this yet again.
I won't be able to.
We have captured this alternative already in the cover letter in the alternatives section.
This optional capability can be added when there is a _real_ implementation by some hw.
>
> \begin{lstlisting}
> struct virtio_pci_owner_legacy_notify_cap {
> struct virtio_pci_cap cap;
> le32 notify_off_multiplier; /* Multiplier for member id. */ }; \end{lstlisting}
>
> \field{notify_off_multiplier} is combined with the \field{member id} to derive
> an address within a BAR.
>
> \begin{lstlisting}
> cap.offset + member_id * notify_off_multiplier \end{lstlisting}
>
>
>
> We also need, as part of theory of operation, text that reads something like
> "accessing the member device legacy interface using one of XYZ commands has
> the same effect as accessing it using the legacy interface".
>
Ack. I will add this.
>
>
> Also, do all VFs support legacy interface with these commands or none at all?
>
Right.
>
> Also these devices will use non-transitional ID but they in fact do have a legacy
> interface so using this definition they are transitional devices. Maybe we need
> to add when we describe the device ID text like "non transitional devices and
> transitional devices utilizing commands XYZ" ...?
Transitional device has specific meaning, I am not sure we should muddy it.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 16:39 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:19 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:19 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 04:39:16PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:38 PM
>
> > >
> > > +\field{legacy_q_notify_supported} when set to 1, indicates that the
> > > +device supports legacy queue notifications at this notification
> > > +location. Legacy Queue Notification address is derived within a BAR for a
> > virtqueue:
> > > +
> > > +\begin{lstlisting}
> > > + cap.offset
> > > +\end{lstlisting}
> > > +
> > > \devicenormative{\paragraph}{Notification capability}{Virtio
> > > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> > capability} The device MUST present at least one notification capability.
> > >
> >
> > I was hoping that we can avoid touching transport-pci.tex completely, this is
> > why I wanted to reuse VIRTIO_PCI_CAP_NOTIFY_CFG.
>
> It is reused, but it doesnt come for free.
> Do you want to re-consider AQ notify query command?
> Or continue as proposed in v6?
If we can't just make it come for free then maybe
VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the number
in the common section and then link to the description in the new
section.
I also feel we want ability to have such a capability in the owner too.
Would prefer including it now though I guess we can add it as an
extension.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:19 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:19 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 04:39:16PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:38 PM
>
> > >
> > > +\field{legacy_q_notify_supported} when set to 1, indicates that the
> > > +device supports legacy queue notifications at this notification
> > > +location. Legacy Queue Notification address is derived within a BAR for a
> > virtqueue:
> > > +
> > > +\begin{lstlisting}
> > > + cap.offset
> > > +\end{lstlisting}
> > > +
> > > \devicenormative{\paragraph}{Notification capability}{Virtio
> > > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> > capability} The device MUST present at least one notification capability.
> > >
> >
> > I was hoping that we can avoid touching transport-pci.tex completely, this is
> > why I wanted to reuse VIRTIO_PCI_CAP_NOTIFY_CFG.
>
> It is reused, but it doesnt come for free.
> Do you want to re-consider AQ notify query command?
> Or continue as proposed in v6?
If we can't just make it come for free then maybe
VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the number
in the common section and then link to the description in the new
section.
I also feel we want ability to have such a capability in the owner too.
Would prefer including it now though I guess we can add it as an
extension.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 17:10 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 17:21 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:21 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> Behalf Of Michael S. Tsirkin
> Sent: Monday, June 19, 2023 1:11 PM
> > The "legacy interface of member device" is a vague term. As implementer,
> reader I don't understand what that means.
> > The commands clearly communicate the scope.
>
> Add this as an example then.
>
I don't know what this means.
The section speaks for itself.
We currently have,
Section 2.12.1 as "Group administration commands".
Under this I added,
2.12.1.1 -> Legacy interface: legacy configuration access commands"
Above is no different than
"3.1.2 Legacy Interface: Device Initialization"
Tomorrow,
2.12.1.2 -> features config commands
2.12.1.3 ->
> > > If you want to also have a type specific text, be my guest
> > > "specifically, for an SRIOV type" ....
> > >
> > SRIOV in generic section doesn't make sense. It is covered in the pci transport
> section.
>
> When it's convenient to you you argue that it's clearer, when it's not you want it
> generic. Just mention it quickly and link to the specific text in the new section.
>
If you reviewer makes point "it is suggestion".
If patch proposer makes a point its "argue" is not good way to discuss.
So can you please avoid quoting "arguing".
It is not my convenience.
In the spec transport specific parts are not added.
Hence, I said SR-IOV in generic to be avoided. It is better kept in the pci transport section.
> If you are intent on
> spreading this all over the spec we'll need to rethink the whole approach.
>
Where is this spread all over?
In v5 it was self contained mainly in pci transport section.
You asked to split to generic and pci.
Fine, I split to two.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 17:21 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:21 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> Behalf Of Michael S. Tsirkin
> Sent: Monday, June 19, 2023 1:11 PM
> > The "legacy interface of member device" is a vague term. As implementer,
> reader I don't understand what that means.
> > The commands clearly communicate the scope.
>
> Add this as an example then.
>
I don't know what this means.
The section speaks for itself.
We currently have,
Section 2.12.1 as "Group administration commands".
Under this I added,
2.12.1.1 -> Legacy interface: legacy configuration access commands"
Above is no different than
"3.1.2 Legacy Interface: Device Initialization"
Tomorrow,
2.12.1.2 -> features config commands
2.12.1.3 ->
> > > If you want to also have a type specific text, be my guest
> > > "specifically, for an SRIOV type" ....
> > >
> > SRIOV in generic section doesn't make sense. It is covered in the pci transport
> section.
>
> When it's convenient to you you argue that it's clearer, when it's not you want it
> generic. Just mention it quickly and link to the specific text in the new section.
>
If you reviewer makes point "it is suggestion".
If patch proposer makes a point its "argue" is not good way to discuss.
So can you please avoid quoting "arguing".
It is not my convenience.
In the spec transport specific parts are not added.
Hence, I said SR-IOV in generic to be avoided. It is better kept in the pci transport section.
> If you are intent on
> spreading this all over the spec we'll need to rethink the whole approach.
>
Where is this spread all over?
In v5 it was self contained mainly in pci transport section.
You asked to split to generic and pci.
Fine, I split to two.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:11 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:26 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:26 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:11:19PM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> > Behalf Of Michael S. Tsirkin
> > Sent: Monday, June 19, 2023 1:04 PM
>
> > On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> > > +Even though virtqueue driver notifications can be communicated
> > > +through administration virtqueue, if the group member device support
> > > +such notifications using a memory-mapped operation, such driver
> > > +notifications are sent using a group member device defined notification
> > region.
> >
> > I still feel we should support sending notifications to owner at least optionally.
> > For example:
> >
> Let's not please keep discussing this yet again.
> I won't be able to.
> We have captured this alternative already in the cover letter in the alternatives section.
>
> This optional capability can be added when there is a _real_ implementation by some hw.
>
> >
> > \begin{lstlisting}
> > struct virtio_pci_owner_legacy_notify_cap {
> > struct virtio_pci_cap cap;
> > le32 notify_off_multiplier; /* Multiplier for member id. */ }; \end{lstlisting}
> >
> > \field{notify_off_multiplier} is combined with the \field{member id} to derive
> > an address within a BAR.
> >
> > \begin{lstlisting}
> > cap.offset + member_id * notify_off_multiplier \end{lstlisting}
> >
> >
> >
> > We also need, as part of theory of operation, text that reads something like
> > "accessing the member device legacy interface using one of XYZ commands has
> > the same effect as accessing it using the legacy interface".
> >
> Ack. I will add this.
>
> >
> >
> > Also, do all VFs support legacy interface with these commands or none at all?
> >
> Right.
>
> >
> > Also these devices will use non-transitional ID but they in fact do have a legacy
> > interface so using this definition they are transitional devices. Maybe we need
> > to add when we describe the device ID text like "non transitional devices and
> > transitional devices utilizing commands XYZ" ...?
>
> Transitional device has specific meaning, I am not sure we should muddy it.
To simplify transition from these earlier draft interfaces,
a device MAY implement:
\begin{description}
\item[Transitional Device]
a device supporting both drivers conforming to this
specification, and allowing legacy drivers.
\end{description}
I agree it can be read this way. The issue is a lot of text
in the spec just assumes that "has legacy interface == transitional
device".
For example:
When using the legacy interface the driver MAY access
the device-specific configuration region using any width accesses, and
a transitional device MUST present driver with the same results as
when accessed using the ``natural'' access method (i.e.
32-bit accesses for 32-bit fields, etc).
If we break the assumption we need to audit the spec for this
assumption and again, I really would rather not.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:26 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:26 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:11:19PM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> > Behalf Of Michael S. Tsirkin
> > Sent: Monday, June 19, 2023 1:04 PM
>
> > On Tue, Jun 13, 2023 at 08:30:15PM +0300, Parav Pandit wrote:
> > > +Even though virtqueue driver notifications can be communicated
> > > +through administration virtqueue, if the group member device support
> > > +such notifications using a memory-mapped operation, such driver
> > > +notifications are sent using a group member device defined notification
> > region.
> >
> > I still feel we should support sending notifications to owner at least optionally.
> > For example:
> >
> Let's not please keep discussing this yet again.
> I won't be able to.
> We have captured this alternative already in the cover letter in the alternatives section.
>
> This optional capability can be added when there is a _real_ implementation by some hw.
>
> >
> > \begin{lstlisting}
> > struct virtio_pci_owner_legacy_notify_cap {
> > struct virtio_pci_cap cap;
> > le32 notify_off_multiplier; /* Multiplier for member id. */ }; \end{lstlisting}
> >
> > \field{notify_off_multiplier} is combined with the \field{member id} to derive
> > an address within a BAR.
> >
> > \begin{lstlisting}
> > cap.offset + member_id * notify_off_multiplier \end{lstlisting}
> >
> >
> >
> > We also need, as part of theory of operation, text that reads something like
> > "accessing the member device legacy interface using one of XYZ commands has
> > the same effect as accessing it using the legacy interface".
> >
> Ack. I will add this.
>
> >
> >
> > Also, do all VFs support legacy interface with these commands or none at all?
> >
> Right.
>
> >
> > Also these devices will use non-transitional ID but they in fact do have a legacy
> > interface so using this definition they are transitional devices. Maybe we need
> > to add when we describe the device ID text like "non transitional devices and
> > transitional devices utilizing commands XYZ" ...?
>
> Transitional device has specific meaning, I am not sure we should muddy it.
To simplify transition from these earlier draft interfaces,
a device MAY implement:
\begin{description}
\item[Transitional Device]
a device supporting both drivers conforming to this
specification, and allowing legacy drivers.
\end{description}
I agree it can be read this way. The issue is a lot of text
in the spec just assumes that "has legacy interface == transitional
device".
For example:
When using the legacy interface the driver MAY access
the device-specific configuration region using any width accesses, and
a transitional device MUST present driver with the same results as
when accessed using the ``natural'' access method (i.e.
32-bit accesses for 32-bit fields, etc).
If we break the assumption we need to audit the spec for this
assumption and again, I really would rather not.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:19 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 17:26 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:20 PM
> > It is reused, but it doesnt come for free.
> > Do you want to re-consider AQ notify query command?
> > Or continue as proposed in v6?
>
>
> If we can't just make it come for free then maybe
> VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the number in
> the common section and then link to the description in the new section.
>
How is it better? It requires burning more bytes.
Is it better because it is self contained?
> I also feel we want ability to have such a capability in the owner too.
> Would prefer including it now though I guess we can add it as an extension.
Re-opening the past discussion.. amazing. :)
Why? How is it different and better than querying over AQ?
I see the AQ is better than group owner and group member specific cap.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:26 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:20 PM
> > It is reused, but it doesnt come for free.
> > Do you want to re-consider AQ notify query command?
> > Or continue as proposed in v6?
>
>
> If we can't just make it come for free then maybe
> VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the number in
> the common section and then link to the description in the new section.
>
How is it better? It requires burning more bytes.
Is it better because it is self contained?
> I also feel we want ability to have such a capability in the owner too.
> Would prefer including it now though I guess we can add it as an extension.
Re-opening the past discussion.. amazing. :)
Why? How is it different and better than querying over AQ?
I see the AQ is better than group owner and group member specific cap.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 17:21 ` Parav Pandit
@ 2023-06-19 17:33 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:33 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:21:20PM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> > Behalf Of Michael S. Tsirkin
> > Sent: Monday, June 19, 2023 1:11 PM
>
> > > The "legacy interface of member device" is a vague term. As implementer,
> > reader I don't understand what that means.
> > > The commands clearly communicate the scope.
> >
> > Add this as an example then.
> >
> I don't know what this means.
> The section speaks for itself.
> We currently have,
>
> Section 2.12.1 as "Group administration commands".
> Under this I added,
> 2.12.1.1 -> Legacy interface: legacy configuration access commands"
>
> Above is no different than
> "3.1.2 Legacy Interface: Device Initialization"
>
> Tomorrow,
> 2.12.1.2 -> features config commands
> 2.12.1.3 ->
>
> > > > If you want to also have a type specific text, be my guest
> > > > "specifically, for an SRIOV type" ....
> > > >
> > > SRIOV in generic section doesn't make sense. It is covered in the pci transport
> > section.
> >
> > When it's convenient to you you argue that it's clearer, when it's not you want it
> > generic. Just mention it quickly and link to the specific text in the new section.
> >
> If you reviewer makes point "it is suggestion".
> If patch proposer makes a point its "argue" is not good way to discuss.
> So can you please avoid quoting "arguing".
To argue
verb: give reasons or cite evidence in support of an idea,
action, or theory, typically with the aim of persuading others to share
one's view.
>
> It is not my convenience.
> In the spec transport specific parts are not added.
> Hence, I said SR-IOV in generic to be avoided. It is better kept in the pci transport section.
>
> > If you are intent on
> > spreading this all over the spec we'll need to rethink the whole approach.
> >
> Where is this spread all over?
> In v5 it was self contained mainly in pci transport section.
> You asked to split to generic and pci.
> Fine, I split to two.
OK I see, I guess I wasn't clear. I was talking about as much as possible
using terms "owner" "member" "commands" "member_id" and so on,
such that the commands apply as generally as possible.
From that POV this needs a bit more work I think we can make more
things generic.
But I am looking for a way to keep this stuff as self contained
as possible. I don't much care whether the new tex is
under pci and admin command section just lists commands and links there,
or it is under admin and then pci section links there.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 17:33 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:33 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:21:20PM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> > Behalf Of Michael S. Tsirkin
> > Sent: Monday, June 19, 2023 1:11 PM
>
> > > The "legacy interface of member device" is a vague term. As implementer,
> > reader I don't understand what that means.
> > > The commands clearly communicate the scope.
> >
> > Add this as an example then.
> >
> I don't know what this means.
> The section speaks for itself.
> We currently have,
>
> Section 2.12.1 as "Group administration commands".
> Under this I added,
> 2.12.1.1 -> Legacy interface: legacy configuration access commands"
>
> Above is no different than
> "3.1.2 Legacy Interface: Device Initialization"
>
> Tomorrow,
> 2.12.1.2 -> features config commands
> 2.12.1.3 ->
>
> > > > If you want to also have a type specific text, be my guest
> > > > "specifically, for an SRIOV type" ....
> > > >
> > > SRIOV in generic section doesn't make sense. It is covered in the pci transport
> > section.
> >
> > When it's convenient to you you argue that it's clearer, when it's not you want it
> > generic. Just mention it quickly and link to the specific text in the new section.
> >
> If you reviewer makes point "it is suggestion".
> If patch proposer makes a point its "argue" is not good way to discuss.
> So can you please avoid quoting "arguing".
To argue
verb: give reasons or cite evidence in support of an idea,
action, or theory, typically with the aim of persuading others to share
one's view.
>
> It is not my convenience.
> In the spec transport specific parts are not added.
> Hence, I said SR-IOV in generic to be avoided. It is better kept in the pci transport section.
>
> > If you are intent on
> > spreading this all over the spec we'll need to rethink the whole approach.
> >
> Where is this spread all over?
> In v5 it was self contained mainly in pci transport section.
> You asked to split to generic and pci.
> Fine, I split to two.
OK I see, I guess I wasn't clear. I was talking about as much as possible
using terms "owner" "member" "commands" "member_id" and so on,
such that the commands apply as generally as possible.
From that POV this needs a bit more work I think we can make more
things generic.
But I am looking for a way to keep this stuff as self contained
as possible. I don't much care whether the new tex is
under pci and admin command section just lists commands and links there,
or it is under admin and then pci section links there.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:26 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 17:35 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:35 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:26 PM
> > > Also these devices will use non-transitional ID but they in fact do
> > > have a legacy interface so using this definition they are
> > > transitional devices. Maybe we need to add when we describe the
> > > device ID text like "non transitional devices and transitional devices utilizing
> commands XYZ" ...?
> >
> > Transitional device has specific meaning, I am not sure we should muddy it.
>
>
>
> To simplify transition from these earlier draft interfaces, a device MAY
> implement:
>
> \begin{description}
> \item[Transitional Device]
> a device supporting both drivers conforming to this
> specification, and allowing legacy drivers.
> \end{description}
>
>
> I agree it can be read this way. The issue is a lot of text in the spec just assumes
> that "has legacy interface == transitional device".
>
>
>
> For example:
> When using the legacy interface the driver MAY access the device-specific
> configuration region using any width accesses, and a transitional device MUST
> present driver with the same results as when accessed using the ``natural''
> access method (i.e.
> 32-bit accesses for 32-bit fields, etc).
>
>
> If we break the assumption we need to audit the spec for this
> assumption and again, I really would rather not.
We are not breaking the assumption. Above listed requirement is already captured in the legacy interface conformance section.
So I am not sure what extra to write here.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:35 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:35 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:26 PM
> > > Also these devices will use non-transitional ID but they in fact do
> > > have a legacy interface so using this definition they are
> > > transitional devices. Maybe we need to add when we describe the
> > > device ID text like "non transitional devices and transitional devices utilizing
> commands XYZ" ...?
> >
> > Transitional device has specific meaning, I am not sure we should muddy it.
>
>
>
> To simplify transition from these earlier draft interfaces, a device MAY
> implement:
>
> \begin{description}
> \item[Transitional Device]
> a device supporting both drivers conforming to this
> specification, and allowing legacy drivers.
> \end{description}
>
>
> I agree it can be read this way. The issue is a lot of text in the spec just assumes
> that "has legacy interface == transitional device".
>
>
>
> For example:
> When using the legacy interface the driver MAY access the device-specific
> configuration region using any width accesses, and a transitional device MUST
> present driver with the same results as when accessed using the ``natural''
> access method (i.e.
> 32-bit accesses for 32-bit fields, etc).
>
>
> If we break the assumption we need to audit the spec for this
> assumption and again, I really would rather not.
We are not breaking the assumption. Above listed requirement is already captured in the legacy interface conformance section.
So I am not sure what extra to write here.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:26 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:37 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:37 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:26:22PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:20 PM
>
> > > It is reused, but it doesnt come for free.
> > > Do you want to re-consider AQ notify query command?
> > > Or continue as proposed in v6?
> >
> >
> > If we can't just make it come for free then maybe
> > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the number in
> > the common section and then link to the description in the new section.
> >
> How is it better? It requires burning more bytes.
> Is it better because it is self contained?
Mostly yes. But it also adds a bit more flexibility, right?
I thought avoiding touching transport-pci is worth less
flexibility but if we can't avoid that...
> > I also feel we want ability to have such a capability in the owner too.
> > Would prefer including it now though I guess we can add it as an extension.
>
> Re-opening the past discussion.. amazing. :)
OK fair enough.. I guess both of us are not 100% happy but
it's a compromize for now. And maybe we'll add one of AQ
and / or PF cap down the road.
> Why? How is it different and better than querying over AQ?
Wait a second let me clarify. I am talking about notification
in the PCI BAR of the PF. *Not* offset in PF capability
but in the VF BAR.
> I see the AQ is better than group owner and group member specific cap.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:37 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:37 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:26:22PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:20 PM
>
> > > It is reused, but it doesnt come for free.
> > > Do you want to re-consider AQ notify query command?
> > > Or continue as proposed in v6?
> >
> >
> > If we can't just make it come for free then maybe
> > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the number in
> > the common section and then link to the description in the new section.
> >
> How is it better? It requires burning more bytes.
> Is it better because it is self contained?
Mostly yes. But it also adds a bit more flexibility, right?
I thought avoiding touching transport-pci is worth less
flexibility but if we can't avoid that...
> > I also feel we want ability to have such a capability in the owner too.
> > Would prefer including it now though I guess we can add it as an extension.
>
> Re-opening the past discussion.. amazing. :)
OK fair enough.. I guess both of us are not 100% happy but
it's a compromize for now. And maybe we'll add one of AQ
and / or PF cap down the road.
> Why? How is it different and better than querying over AQ?
Wait a second let me clarify. I am talking about notification
in the PCI BAR of the PF. *Not* offset in PF capability
but in the VF BAR.
> I see the AQ is better than group owner and group member specific cap.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
2023-06-19 17:33 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 17:38 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:38 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:33 PM
> But I am looking for a way to keep this stuff as self contained as possible. I don't
> much care whether the new tex is under pci and admin command section just
> lists commands and links there, or it is under admin and then pci section links
> there.
Yes. On it to move things to the generic admin section.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [virtio-dev] Re: [PATCH v6 3/4] admin: Add group member legacy register access commands
@ 2023-06-19 17:38 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:38 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:33 PM
> But I am looking for a way to keep this stuff as self contained as possible. I don't
> much care whether the new tex is under pci and admin command section just
> lists commands and links there, or it is under admin and then pci section links
> there.
Yes. On it to move things to the generic admin section.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:37 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 17:45 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:38 PM
> > > If we can't just make it come for free then maybe
> > > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the
> > > number in the common section and then link to the description in the new
> section.
> > >
> > How is it better? It requires burning more bytes.
> > Is it better because it is self contained?
>
> Mostly yes. But it also adds a bit more flexibility, right?
> I thought avoiding touching transport-pci is worth less flexibility but if we can't
> avoid that...
>
That flexibility is better gained via AQ instead of introducing new extended capability on the VF.
>
> > > I also feel we want ability to have such a capability in the owner too.
> > > Would prefer including it now though I guess we can add it as an extension.
> >
> > Re-opening the past discussion.. amazing. :)
>
> OK fair enough.. I guess both of us are not 100% happy but it's a compromize
> for now. And maybe we'll add one of AQ and / or PF cap down the road.
>
> > Why? How is it different and better than querying over AQ?
>
> Wait a second let me clarify. I am talking about notification in the PCI BAR of the
> PF. *Not* offset in PF capability but in the VF BAR.
>
Ok. question remains the same, why not discover such notification capability it over the AQ command?
AQ command on the PF is self-contained and extendible without baking things in very low level hw centric pci capabilities, which are hard to get rid of it.
>
> > I see the AQ is better than group owner and group member specific cap.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:45 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 17:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:38 PM
> > > If we can't just make it come for free then maybe
> > > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the
> > > number in the common section and then link to the description in the new
> section.
> > >
> > How is it better? It requires burning more bytes.
> > Is it better because it is self contained?
>
> Mostly yes. But it also adds a bit more flexibility, right?
> I thought avoiding touching transport-pci is worth less flexibility but if we can't
> avoid that...
>
That flexibility is better gained via AQ instead of introducing new extended capability on the VF.
>
> > > I also feel we want ability to have such a capability in the owner too.
> > > Would prefer including it now though I guess we can add it as an extension.
> >
> > Re-opening the past discussion.. amazing. :)
>
> OK fair enough.. I guess both of us are not 100% happy but it's a compromize
> for now. And maybe we'll add one of AQ and / or PF cap down the road.
>
> > Why? How is it different and better than querying over AQ?
>
> Wait a second let me clarify. I am talking about notification in the PCI BAR of the
> PF. *Not* offset in PF capability but in the VF BAR.
>
Ok. question remains the same, why not discover such notification capability it over the AQ command?
AQ command on the PF is self-contained and extendible without baking things in very low level hw centric pci capabilities, which are hard to get rid of it.
>
> > I see the AQ is better than group owner and group member specific cap.
>
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:35 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:46 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:46 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:35:04PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:26 PM
>
> > > > Also these devices will use non-transitional ID but they in fact do
> > > > have a legacy interface so using this definition they are
> > > > transitional devices. Maybe we need to add when we describe the
> > > > device ID text like "non transitional devices and transitional devices utilizing
> > commands XYZ" ...?
> > >
> > > Transitional device has specific meaning, I am not sure we should muddy it.
> >
> >
> >
> > To simplify transition from these earlier draft interfaces, a device MAY
> > implement:
> >
> > \begin{description}
> > \item[Transitional Device]
> > a device supporting both drivers conforming to this
> > specification, and allowing legacy drivers.
> > \end{description}
> >
> >
> > I agree it can be read this way. The issue is a lot of text in the spec just assumes
> > that "has legacy interface == transitional device".
> >
> >
> >
> > For example:
> > When using the legacy interface the driver MAY access the device-specific
> > configuration region using any width accesses, and a transitional device MUST
> > present driver with the same results as when accessed using the ``natural''
> > access method (i.e.
> > 32-bit accesses for 32-bit fields, etc).
> >
> >
> > If we break the assumption we need to audit the spec for this
> > assumption and again, I really would rather not.
>
> We are not breaking the assumption. Above listed requirement is already captured in the legacy interface conformance section.
> So I am not sure what extra to write here.
>
Hmm not sure what's unclear. I can try to explain the issue again.
These devices have a legacy interface yes?
So they should be transitional to avoid breaking assumption.
But they are not *exactly*
in that they don't have a transitional device ID.
At least the device id section needs extra text
then to explain this?
Or do you just want to make them have transitional ID?
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:46 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:46 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:35:04PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:26 PM
>
> > > > Also these devices will use non-transitional ID but they in fact do
> > > > have a legacy interface so using this definition they are
> > > > transitional devices. Maybe we need to add when we describe the
> > > > device ID text like "non transitional devices and transitional devices utilizing
> > commands XYZ" ...?
> > >
> > > Transitional device has specific meaning, I am not sure we should muddy it.
> >
> >
> >
> > To simplify transition from these earlier draft interfaces, a device MAY
> > implement:
> >
> > \begin{description}
> > \item[Transitional Device]
> > a device supporting both drivers conforming to this
> > specification, and allowing legacy drivers.
> > \end{description}
> >
> >
> > I agree it can be read this way. The issue is a lot of text in the spec just assumes
> > that "has legacy interface == transitional device".
> >
> >
> >
> > For example:
> > When using the legacy interface the driver MAY access the device-specific
> > configuration region using any width accesses, and a transitional device MUST
> > present driver with the same results as when accessed using the ``natural''
> > access method (i.e.
> > 32-bit accesses for 32-bit fields, etc).
> >
> >
> > If we break the assumption we need to audit the spec for this
> > assumption and again, I really would rather not.
>
> We are not breaking the assumption. Above listed requirement is already captured in the legacy interface conformance section.
> So I am not sure what extra to write here.
>
Hmm not sure what's unclear. I can try to explain the issue again.
These devices have a legacy interface yes?
So they should be transitional to avoid breaking assumption.
But they are not *exactly*
in that they don't have a transitional device ID.
At least the device id section needs extra text
then to explain this?
Or do you just want to make them have transitional ID?
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:45 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 17:57 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:57 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:45:17PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:38 PM
>
> > > > If we can't just make it come for free then maybe
> > > > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the
> > > > number in the common section and then link to the description in the new
> > section.
> > > >
> > > How is it better? It requires burning more bytes.
> > > Is it better because it is self contained?
> >
> > Mostly yes. But it also adds a bit more flexibility, right?
> > I thought avoiding touching transport-pci is worth less flexibility but if we can't
> > avoid that...
> >
> That flexibility is better gained via AQ instead of introducing new extended capability on the VF.
>
> >
> > > > I also feel we want ability to have such a capability in the owner too.
> > > > Would prefer including it now though I guess we can add it as an extension.
> > >
> > > Re-opening the past discussion.. amazing. :)
> >
> > OK fair enough.. I guess both of us are not 100% happy but it's a compromize
> > for now. And maybe we'll add one of AQ and / or PF cap down the road.
> >
> > > Why? How is it different and better than querying over AQ?
> >
> > Wait a second let me clarify. I am talking about notification in the PCI BAR of the
> > PF. *Not* offset in PF capability but in the VF BAR.
> >
> Ok. question remains the same, why not discover such notification capability it over the AQ command?
> AQ command on the PF is self-contained and extendible without baking things in very low level hw centric pci capabilities, which are hard to get rid of it.
I worry about systems where there's value in having VF driver
map memory without talking to PF driver.
Current VF capability is kind of ok I guess ...
If you want admin command to query then it would need
to map in PF memory.
I understand two concerns with this:
- you worry that this forces ordering requirements - it's true and I
don't know of a good fix. So if possible use VF BAR by preference?
- you worry about wasting physical memory space
this we can fix by sticking VF# in the kick.
As an alternative, if we can make the new command
able to communicate offset in PCI BAR or in VF BAR
or both then I think that's also an acceptable compromize:
drivers that have trouble mapping VF BAR after querying
PF can just map PF BAR. WDYT?
> >
> > > I see the AQ is better than group owner and group member specific cap.
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 17:57 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 17:57 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:45:17PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:38 PM
>
> > > > If we can't just make it come for free then maybe
> > > > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the
> > > > number in the common section and then link to the description in the new
> > section.
> > > >
> > > How is it better? It requires burning more bytes.
> > > Is it better because it is self contained?
> >
> > Mostly yes. But it also adds a bit more flexibility, right?
> > I thought avoiding touching transport-pci is worth less flexibility but if we can't
> > avoid that...
> >
> That flexibility is better gained via AQ instead of introducing new extended capability on the VF.
>
> >
> > > > I also feel we want ability to have such a capability in the owner too.
> > > > Would prefer including it now though I guess we can add it as an extension.
> > >
> > > Re-opening the past discussion.. amazing. :)
> >
> > OK fair enough.. I guess both of us are not 100% happy but it's a compromize
> > for now. And maybe we'll add one of AQ and / or PF cap down the road.
> >
> > > Why? How is it different and better than querying over AQ?
> >
> > Wait a second let me clarify. I am talking about notification in the PCI BAR of the
> > PF. *Not* offset in PF capability but in the VF BAR.
> >
> Ok. question remains the same, why not discover such notification capability it over the AQ command?
> AQ command on the PF is self-contained and extendible without baking things in very low level hw centric pci capabilities, which are hard to get rid of it.
I worry about systems where there's value in having VF driver
map memory without talking to PF driver.
Current VF capability is kind of ok I guess ...
If you want admin command to query then it would need
to map in PF memory.
I understand two concerns with this:
- you worry that this forces ordering requirements - it's true and I
don't know of a good fix. So if possible use VF BAR by preference?
- you worry about wasting physical memory space
this we can fix by sticking VF# in the kick.
As an alternative, if we can make the new command
able to communicate offset in PCI BAR or in VF BAR
or both then I think that's also an acceptable compromize:
drivers that have trouble mapping VF BAR after querying
PF can just map PF BAR. WDYT?
> >
> > > I see the AQ is better than group owner and group member specific cap.
> >
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:45 ` [virtio-comment] " Parav Pandit
@ 2023-06-19 18:00 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 18:00 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:45:17PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:38 PM
>
> > > > If we can't just make it come for free then maybe
> > > > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the
> > > > number in the common section and then link to the description in the new
> > section.
> > > >
> > > How is it better? It requires burning more bytes.
> > > Is it better because it is self contained?
> >
> > Mostly yes. But it also adds a bit more flexibility, right?
> > I thought avoiding touching transport-pci is worth less flexibility but if we can't
> > avoid that...
> >
> That flexibility is better gained via AQ instead of introducing new extended capability on the VF.
To be frank for e.g. transport vq we'll need to do it with admin command
(can we stop saying AQ please? it's a habit that carries over
when you write spec later. I know AC is a weird abbreviation though
;) )
So yes, if we can find a way to make everyone happy with admin commands
then it is better, for sure.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 18:00 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 18:00 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 05:45:17PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 1:38 PM
>
> > > > If we can't just make it come for free then maybe
> > > > VIRTIO_PCI_CAP_LEGACY_NOTIFY_CFG is better, we can just list the
> > > > number in the common section and then link to the description in the new
> > section.
> > > >
> > > How is it better? It requires burning more bytes.
> > > Is it better because it is self contained?
> >
> > Mostly yes. But it also adds a bit more flexibility, right?
> > I thought avoiding touching transport-pci is worth less flexibility but if we can't
> > avoid that...
> >
> That flexibility is better gained via AQ instead of introducing new extended capability on the VF.
To be frank for e.g. transport vq we'll need to do it with admin command
(can we stop saying AQ please? it's a habit that carries over
when you write spec later. I know AC is a weird abbreviation though
;) )
So yes, if we can find a way to make everyone happy with admin commands
then it is better, for sure.
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:57 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 18:07 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 18:07 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:58 PM
> > AQ command on the PF is self-contained and extendible without baking things
> in very low level hw centric pci capabilities, which are hard to get rid of it.
>
> I worry about systems where there's value in having VF driver map memory
> without talking to PF driver.
Such driver will not be able to use AQ legacy commands anyway.
When such driver arrive, someone, somehow will build self contained device.
And hence, adding such pci capability in that future makes sense.
We don't need to do it now.
> Current VF capability is kind of ok I guess ...
>
Already proposed one?
> If you want admin command to query then it would need to map in PF memory.
No need.
It is similar to rest of the configuration access commands.
> I understand two concerns with this:
> - you worry that this forces ordering requirements - it's true and I
> don't know of a good fix. So if possible use VF BAR by preference?
> - you worry about wasting physical memory space
> this we can fix by sticking VF# in the kick.
>
> As an alternative, if we can make the new command able to communicate offset
> in PCI BAR or in VF BAR or both then I think that's also an acceptable
A new command proposed was for VF BAR that member driver can ask to its group owner driver (like rest of the config commands).
A future command can tell for the PF BAR if needed and some device wants to do such extra implementation.
> compromize:
> drivers that have trouble mapping VF BAR after querying PF can just map PF
> BAR. WDYT?
PF BAR for notification can be done later when there is some device who really have the limitation and wants to do that.
We don't see that additional need as driver notifications are present on the VF already as listed in the alternative.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 18:07 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 18:07 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 1:58 PM
> > AQ command on the PF is self-contained and extendible without baking things
> in very low level hw centric pci capabilities, which are hard to get rid of it.
>
> I worry about systems where there's value in having VF driver map memory
> without talking to PF driver.
Such driver will not be able to use AQ legacy commands anyway.
When such driver arrive, someone, somehow will build self contained device.
And hence, adding such pci capability in that future makes sense.
We don't need to do it now.
> Current VF capability is kind of ok I guess ...
>
Already proposed one?
> If you want admin command to query then it would need to map in PF memory.
No need.
It is similar to rest of the configuration access commands.
> I understand two concerns with this:
> - you worry that this forces ordering requirements - it's true and I
> don't know of a good fix. So if possible use VF BAR by preference?
> - you worry about wasting physical memory space
> this we can fix by sticking VF# in the kick.
>
> As an alternative, if we can make the new command able to communicate offset
> in PCI BAR or in VF BAR or both then I think that's also an acceptable
A new command proposed was for VF BAR that member driver can ask to its group owner driver (like rest of the config commands).
A future command can tell for the PF BAR if needed and some device wants to do such extra implementation.
> compromize:
> drivers that have trouble mapping VF BAR after querying PF can just map PF
> BAR. WDYT?
PF BAR for notification can be done later when there is some device who really have the limitation and wants to do that.
We don't see that additional need as driver notifications are present on the VF already as listed in the alternative.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 18:00 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 18:12 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 18:12 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 2:01 PM
> > That flexibility is better gained via AQ instead of introducing new extended
> capability on the VF.
>
> To be frank for e.g. transport vq we'll need to do it with admin command (can
> we stop saying AQ please? it's a habit that carries over when you write spec
> later. I know AC is a weird abbreviation though
> ;) )
>
Yes, this was mentioned already, but you ignored or missed in past, not sure why.
Yes, admin command.
And yes lets please stop calling the transport vq as well, as it is not going to "transport everything".
It is also just bunch of commands... this was discussed in a thread with Jason and want to park that discussion at later point.
> So yes, if we can find a way to make everyone happy with admin commands
> then it is better, for sure.
From device side we are ok with Admin command to query the notification offset in the VF BAR.
So far you had the concern with admin command for query.
Let me know how you want to proceed as we have both the options proposed now.
So in v7 I want to write the final option that you don't need to re-think on.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 18:12 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 18:12 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 2:01 PM
> > That flexibility is better gained via AQ instead of introducing new extended
> capability on the VF.
>
> To be frank for e.g. transport vq we'll need to do it with admin command (can
> we stop saying AQ please? it's a habit that carries over when you write spec
> later. I know AC is a weird abbreviation though
> ;) )
>
Yes, this was mentioned already, but you ignored or missed in past, not sure why.
Yes, admin command.
And yes lets please stop calling the transport vq as well, as it is not going to "transport everything".
It is also just bunch of commands... this was discussed in a thread with Jason and want to park that discussion at later point.
> So yes, if we can find a way to make everyone happy with admin commands
> then it is better, for sure.
From device side we are ok with Admin command to query the notification offset in the VF BAR.
So far you had the concern with admin command for query.
Let me know how you want to proceed as we have both the options proposed now.
So in v7 I want to write the final option that you don't need to re-think on.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 16:16 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-19 21:07 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 21:07 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:17 PM
> I'd love to just put this stuff in a separate tex document and just link there from
> the generic thing.
> Or if it's too hard, please create new sections for this stuff. But in this case each
> of these sections has to start with repeating the description when things apply.
> E.g. "when using legacy interface"
> when using commands "A B C to access" etc etc.
>
> I wrote a bunch of comments below.
>
I will stick to them as most things are taken out to generic section anyway in patch 3.
Only PCI transport part stays here.
I will move some more text to generic section.
> > +As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]}
> > +PCI VFs do not support the I/O space BAR. Due to this limitation, a
> > +PCI VF hardware device that supports legacy interfaces cannot be
> > +attached as passthrough device to the guest virtual machine.
>
> That's not the only reason. Let's keep things generic and start by saying
> something like "Some configurations with a need to support legacy drivers can
> not implement a transitional device. For example .... ".
>
I will avoid For example part and move this to generic section.
> > +
> > +To have a transitional or legacy virtio device in the guest virtual
> > +machine, a PCI PF device as a group owner may
>
> may -> can outside conformance statements.
>
Ack.
> > support accessing its group member
>
> Pls keep it generic. A group owner ....
>
Ack
>
> > +device's legacy configuration region using the group administration
> > +commands listed at \ref{sec:Basic Facilities of a Virtio Device /
> > +Device groups / Group administration commands / Legacy Interfaces: legacy
> configuration access commands}.
> > +These legacy configuration access commands are transported via the
> > +administration virtqueue.
>
> are these called legacy configuration access commands then?
> why do you introduce this term? it does not look like it's used anywhere else.
>
It is used and referred PCI section " Legacy Interfaces: Group member device Configuration Region Access".
>
> I would prefer it if you just use the term "legacy interface"
> it is well defined and widely used in the spec.
>
It is "legacy interface", with suffix for group member config access.
In future, there may be a section for other things than config access.
> again leave virtqueue out of it pls we'll have other ways down the road. what
> matters is that they are transferred to a group member by sending legacy
> configuration access commands to the group owner.
>
Ack.
It is theory of operation, not a normative.
Tomorrow if other than aq exists, one will write as "AQ or xyz"..
But fine to me to reduce changes from 216 to less lines. :)
>
> > This mechanism enables minimal involvement of the
> > +hypervisor software only for the purpose of the legacy configuration region
> access.
>
> Leave marketing out of it pls, kill last sentence.
>
Huh, I don't see theory of operation and scope of the hypervisor as marketing at all.
I will remove as it makes this patch shorter.
>
>
> > +
> > +When a guest virtual machine requests legacy configuration and device
> > +specific regions
>
> wait so now it's legacy configuration and device specific regions and these two
> are distinct?
>
> > of group member device, a hypervisor software accesses
> > +it using an administration virtqueue
>
> please talk of administration commands. virtqueue does not matter at all.
>
Ack.
> > on behalf of the guest virtual machine.
>
>
>
> To put it in our terms, something like this:
> when a legacy driver accesses a member device of
> a group using the
> legacy interface, a modern driver can intercept
> the access and forward it to the owner device.
>
I will not mention "modern driver" as it has zero reference in spec and don't want to bring Linux terms here.
"the driver can intercept" is enough.
>
>
> > +
> > +Even though virtqueue driver notifications can be communicated
> > +through administration virtqueue, if the group member device support
> > +such
>
> supports?
>
Ack.
> > +notifications using a memory-mapped operation, such driver
> > +notifications are sent using a group member device defined notification
> region.
>
> what does "are sent" imply here? is there a matching conformance statement?
> who sends them?
>
It has the same meaning as "Driver notifications" section first line has.
Isnt it implicit that driver notifications are sent by driver based on "Driver notifications" section?
> > +
> > +The group owner device
>
> this is the 1st you mention any group or its owner.
>
> > should not expose PCI BAR0
>
> Not PCI BAR0. That is in header. You mean VF BAR0.
>
I will rewrite it as,
The group owner device should not expose PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it prefers to
support legacy interface for legacy configuration access using this group owner.
> > for the group member
> > +devices when it prefers to support legacy interface for legacy
> > +configuration access using its group owner.
>
> don't use should outside conformance
>
What should be used instead of "should"?
>
> I think this specific case actually should be more specific.
> Something like:
>
> - For PCI SRIOV groups, when group owner device implements commands
> A,B,C the group owner's VF BAR 0 is hardwired to 0
> in its PCI SRIOV capability.
>
I wrote it slightly differently above.
>
> > +This facilitates hypervisor software to operate with least amount of
> > +complexities
>
> complexity is its own plural
>
> > to emulate the legacy interface I/O space BAR and passthrough
> > +other PCI BARs and PCI device capabilities to the guest virtual
> > +machine without any translation.
> > +
> > +The group member device should not expose PCI BAR0 in various PCI
> capabilities.
> > +
> > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > +Member Device Legacy Configuration Access}
> > +
> > +When a PCI SR-IOV group owner device supports
> > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> owner
> > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > +This is to facilitate the software to emulate I/O region BAR0 for supporting
> the legacy interface.
>
> not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose it". if you
> want to register can be "unimplemented".
> Base Address registers are hardwired to zero
>
> But it is better to be specific on what should happen. hardwire VF BAR0 to 0,
> right?
>
Hardwire to zero and not expose is same thing to me.
>
> > +
> > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > +Member Device Legacy Configuration Access}
> > +
> > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > +member exposes a PCI BAR0.
>
> what does "emulate" mean here? which driver it is?
>
I will add the line to theory of operation, but I recollect "emulate" line came from you.
> > diff --git a/transport-pci.tex b/transport-pci.tex index
> > a5c6719..3647485 100644
> > --- a/transport-pci.tex
> > +++ b/transport-pci.tex
> > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > struct virtio_pci_cap cap;
> > le32 notify_off_multiplier; /* Multiplier for
> > queue_notify_off. */
> > + u8 legacy_q_notify_supported;
>
> I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
>
>
> > + u8 reserved[3];
>
>
> align with spaces pls.
>
Ack.
>
> > };
> > \end{lstlisting}
> >
> > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > layout}\label{sec:Virtio Transport Options the same Queue Notify address for
> all queues.
> > \end{note}
> >
> > +\field{legacy_q_notify_supported} when set to 1,
>
>
> Do you mean something like:
> When \field{legacy_q_notify_supported} this indicates that ...
>
Yes will fix.
> > indicates that the device
> > +supports legacy queue notifications at this notification location.
>
> And what location? this refers to what? the location described by this capability
> maybe?
>
Ack.
>
> More specifically, assume a transitional device (with an IO BAR as normal). Does
> presence of this flag in the capability imply we can use a legacy queue but use
> this notification with this capability?
I don't know how this combination can be even possible by the device other than a buggy software who will do both.
> I worry that if we allow that, it opens floodgates of people who are too lazy to
> upgrade to 1.0 but just hack this notification in there.
>
Don't understand the concern.
If a device has transitional device, it is 1.0 device supporting legacy with IOBAR and newer interface.
So...
>
>
>
>
> > Legacy Queue
> > +Notification address is derived within a BAR for a virtqueue:
> > +
> > +\begin{lstlisting}
> > + cap.offset
> > +\end{lstlisting}
> > +
> > \devicenormative{\paragraph}{Notification capability}{Virtio
> > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> capability} The device MUST present at least one notification capability.
> >
>
> I don't get what this is saying. I *think* you really mean is that
> queue_notify_off is 0.
>
Please refer to the current notification address definition.
Above is written in similar way where it doesnt need to describe queue_notify_off.
>
> > @@ -596,6 +606,14 @@ \subsubsection{Notification structure
> > layout}\label{sec:Virtio Transport Options cap.length >=
> > queue_notify_off * notify_off_multiplier + 4 \end{lstlisting}
> >
> > +\paragraph{Legacy Interfaces: Notification
> > +capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus
> > +/ PCI Device Layout / Legacy Interfaces: Notification capability}
> > +
> > +The device SHOULD set \field{legacy_q_notify_supported}
>
> why SHOULD and not MUST here - if driver checks for it then it won't work
> without no?
>
If device doesn't want to support queue notifications, it has ability to set to 0.
> > when the device
> > +notification location supports legacy driver notifications.
>
> legacy driver notifications is ambigous.
> what you want here and elsewhere is something like "when used through a
> legacy interface, device supports driver
>
Ack.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-19 21:07 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-19 21:07 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 12:17 PM
> I'd love to just put this stuff in a separate tex document and just link there from
> the generic thing.
> Or if it's too hard, please create new sections for this stuff. But in this case each
> of these sections has to start with repeating the description when things apply.
> E.g. "when using legacy interface"
> when using commands "A B C to access" etc etc.
>
> I wrote a bunch of comments below.
>
I will stick to them as most things are taken out to generic section anyway in patch 3.
Only PCI transport part stays here.
I will move some more text to generic section.
> > +As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]}
> > +PCI VFs do not support the I/O space BAR. Due to this limitation, a
> > +PCI VF hardware device that supports legacy interfaces cannot be
> > +attached as passthrough device to the guest virtual machine.
>
> That's not the only reason. Let's keep things generic and start by saying
> something like "Some configurations with a need to support legacy drivers can
> not implement a transitional device. For example .... ".
>
I will avoid For example part and move this to generic section.
> > +
> > +To have a transitional or legacy virtio device in the guest virtual
> > +machine, a PCI PF device as a group owner may
>
> may -> can outside conformance statements.
>
Ack.
> > support accessing its group member
>
> Pls keep it generic. A group owner ....
>
Ack
>
> > +device's legacy configuration region using the group administration
> > +commands listed at \ref{sec:Basic Facilities of a Virtio Device /
> > +Device groups / Group administration commands / Legacy Interfaces: legacy
> configuration access commands}.
> > +These legacy configuration access commands are transported via the
> > +administration virtqueue.
>
> are these called legacy configuration access commands then?
> why do you introduce this term? it does not look like it's used anywhere else.
>
It is used and referred PCI section " Legacy Interfaces: Group member device Configuration Region Access".
>
> I would prefer it if you just use the term "legacy interface"
> it is well defined and widely used in the spec.
>
It is "legacy interface", with suffix for group member config access.
In future, there may be a section for other things than config access.
> again leave virtqueue out of it pls we'll have other ways down the road. what
> matters is that they are transferred to a group member by sending legacy
> configuration access commands to the group owner.
>
Ack.
It is theory of operation, not a normative.
Tomorrow if other than aq exists, one will write as "AQ or xyz"..
But fine to me to reduce changes from 216 to less lines. :)
>
> > This mechanism enables minimal involvement of the
> > +hypervisor software only for the purpose of the legacy configuration region
> access.
>
> Leave marketing out of it pls, kill last sentence.
>
Huh, I don't see theory of operation and scope of the hypervisor as marketing at all.
I will remove as it makes this patch shorter.
>
>
> > +
> > +When a guest virtual machine requests legacy configuration and device
> > +specific regions
>
> wait so now it's legacy configuration and device specific regions and these two
> are distinct?
>
> > of group member device, a hypervisor software accesses
> > +it using an administration virtqueue
>
> please talk of administration commands. virtqueue does not matter at all.
>
Ack.
> > on behalf of the guest virtual machine.
>
>
>
> To put it in our terms, something like this:
> when a legacy driver accesses a member device of
> a group using the
> legacy interface, a modern driver can intercept
> the access and forward it to the owner device.
>
I will not mention "modern driver" as it has zero reference in spec and don't want to bring Linux terms here.
"the driver can intercept" is enough.
>
>
> > +
> > +Even though virtqueue driver notifications can be communicated
> > +through administration virtqueue, if the group member device support
> > +such
>
> supports?
>
Ack.
> > +notifications using a memory-mapped operation, such driver
> > +notifications are sent using a group member device defined notification
> region.
>
> what does "are sent" imply here? is there a matching conformance statement?
> who sends them?
>
It has the same meaning as "Driver notifications" section first line has.
Isnt it implicit that driver notifications are sent by driver based on "Driver notifications" section?
> > +
> > +The group owner device
>
> this is the 1st you mention any group or its owner.
>
> > should not expose PCI BAR0
>
> Not PCI BAR0. That is in header. You mean VF BAR0.
>
I will rewrite it as,
The group owner device should not expose PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it prefers to
support legacy interface for legacy configuration access using this group owner.
> > for the group member
> > +devices when it prefers to support legacy interface for legacy
> > +configuration access using its group owner.
>
> don't use should outside conformance
>
What should be used instead of "should"?
>
> I think this specific case actually should be more specific.
> Something like:
>
> - For PCI SRIOV groups, when group owner device implements commands
> A,B,C the group owner's VF BAR 0 is hardwired to 0
> in its PCI SRIOV capability.
>
I wrote it slightly differently above.
>
> > +This facilitates hypervisor software to operate with least amount of
> > +complexities
>
> complexity is its own plural
>
> > to emulate the legacy interface I/O space BAR and passthrough
> > +other PCI BARs and PCI device capabilities to the guest virtual
> > +machine without any translation.
> > +
> > +The group member device should not expose PCI BAR0 in various PCI
> capabilities.
> > +
> > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > +Member Device Legacy Configuration Access}
> > +
> > +When a PCI SR-IOV group owner device supports
> > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> owner
> > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > +This is to facilitate the software to emulate I/O region BAR0 for supporting
> the legacy interface.
>
> not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose it". if you
> want to register can be "unimplemented".
> Base Address registers are hardwired to zero
>
> But it is better to be specific on what should happen. hardwire VF BAR0 to 0,
> right?
>
Hardwire to zero and not expose is same thing to me.
>
> > +
> > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > +Member Device Legacy Configuration Access}
> > +
> > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > +member exposes a PCI BAR0.
>
> what does "emulate" mean here? which driver it is?
>
I will add the line to theory of operation, but I recollect "emulate" line came from you.
> > diff --git a/transport-pci.tex b/transport-pci.tex index
> > a5c6719..3647485 100644
> > --- a/transport-pci.tex
> > +++ b/transport-pci.tex
> > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > struct virtio_pci_cap cap;
> > le32 notify_off_multiplier; /* Multiplier for
> > queue_notify_off. */
> > + u8 legacy_q_notify_supported;
>
> I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
>
>
> > + u8 reserved[3];
>
>
> align with spaces pls.
>
Ack.
>
> > };
> > \end{lstlisting}
> >
> > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > layout}\label{sec:Virtio Transport Options the same Queue Notify address for
> all queues.
> > \end{note}
> >
> > +\field{legacy_q_notify_supported} when set to 1,
>
>
> Do you mean something like:
> When \field{legacy_q_notify_supported} this indicates that ...
>
Yes will fix.
> > indicates that the device
> > +supports legacy queue notifications at this notification location.
>
> And what location? this refers to what? the location described by this capability
> maybe?
>
Ack.
>
> More specifically, assume a transitional device (with an IO BAR as normal). Does
> presence of this flag in the capability imply we can use a legacy queue but use
> this notification with this capability?
I don't know how this combination can be even possible by the device other than a buggy software who will do both.
> I worry that if we allow that, it opens floodgates of people who are too lazy to
> upgrade to 1.0 but just hack this notification in there.
>
Don't understand the concern.
If a device has transitional device, it is 1.0 device supporting legacy with IOBAR and newer interface.
So...
>
>
>
>
> > Legacy Queue
> > +Notification address is derived within a BAR for a virtqueue:
> > +
> > +\begin{lstlisting}
> > + cap.offset
> > +\end{lstlisting}
> > +
> > \devicenormative{\paragraph}{Notification capability}{Virtio
> > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> capability} The device MUST present at least one notification capability.
> >
>
> I don't get what this is saying. I *think* you really mean is that
> queue_notify_off is 0.
>
Please refer to the current notification address definition.
Above is written in similar way where it doesnt need to describe queue_notify_off.
>
> > @@ -596,6 +606,14 @@ \subsubsection{Notification structure
> > layout}\label{sec:Virtio Transport Options cap.length >=
> > queue_notify_off * notify_off_multiplier + 4 \end{lstlisting}
> >
> > +\paragraph{Legacy Interfaces: Notification
> > +capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus
> > +/ PCI Device Layout / Legacy Interfaces: Notification capability}
> > +
> > +The device SHOULD set \field{legacy_q_notify_supported}
>
> why SHOULD and not MUST here - if driver checks for it then it won't work
> without no?
>
If device doesn't want to support queue notifications, it has ability to set to 0.
> > when the device
> > +notification location supports legacy driver notifications.
>
> legacy driver notifications is ambigous.
> what you want here and elsewhere is something like "when used through a
> legacy interface, device supports driver
>
Ack.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 17:46 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-20 0:14 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-20 0:14 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> Behalf Of Michael S. Tsirkin
> Sent: Monday, June 19, 2023 1:46 PM
> These devices have a legacy interface yes?
Yes. partially.
> So they should be transitional to avoid breaking assumption.
>
>
> But they are not *exactly*
> in that they don't have a transitional device ID.
>
Right. They do not have transitional device ID.
> At least the device id section needs extra text then to explain this?
>
We don't modify any of the transitional device attributes.
In respective conformance section, it is described what requirements of legacy interface to follow.
> Or do you just want to make them have transitional ID?
Don't want to do that.
Non transitional device id with the extension is just fine.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-20 0:14 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-20 0:14 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> Behalf Of Michael S. Tsirkin
> Sent: Monday, June 19, 2023 1:46 PM
> These devices have a legacy interface yes?
Yes. partially.
> So they should be transitional to avoid breaking assumption.
>
>
> But they are not *exactly*
> in that they don't have a transitional device ID.
>
Right. They do not have transitional device ID.
> At least the device id section needs extra text then to explain this?
>
We don't modify any of the transitional device attributes.
In respective conformance section, it is described what requirements of legacy interface to follow.
> Or do you just want to make them have transitional ID?
Don't want to do that.
Non transitional device id with the extension is just fine.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-20 0:14 ` [virtio-comment] " Parav Pandit
@ 2023-06-20 10:21 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-20 10:21 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Tue, Jun 20, 2023 at 12:14:16AM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> > Behalf Of Michael S. Tsirkin
> > Sent: Monday, June 19, 2023 1:46 PM
>
> > These devices have a legacy interface yes?
> Yes. partially.
>
> > So they should be transitional to avoid breaking assumption.
> >
> >
> > But they are not *exactly*
> > in that they don't have a transitional device ID.
> >
> Right. They do not have transitional device ID.
I was trying to think whether we need a conformance statement
stating so. I guess this is up to the device?
Then let's make it clear. Something like:
For the SR-IOV group type,
the owner device supporting legacy configuration access commands
[assuming this is the term - do we define it somewhere? or just
list the commands]
MAY follow the rules for the PCI Device ID, Revision ID
and Subsystem Device ID for the non-transitional devices
documented in
{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery}
or do you want to make it a SHOULD?
> > At least the device id section needs extra text then to explain this?
> >
> We don't modify any of the transitional device attributes.
> In respective conformance section, it is described what requirements of legacy interface to follow.
> > Or do you just want to make them have transitional ID?
> Don't want to do that.
> Non transitional device id with the extension is just fine.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-20 10:21 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-20 10:21 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Tue, Jun 20, 2023 at 12:14:16AM +0000, Parav Pandit wrote:
>
>
> > From: virtio-dev@lists.oasis-open.org <virtio-dev@lists.oasis-open.org> On
> > Behalf Of Michael S. Tsirkin
> > Sent: Monday, June 19, 2023 1:46 PM
>
> > These devices have a legacy interface yes?
> Yes. partially.
>
> > So they should be transitional to avoid breaking assumption.
> >
> >
> > But they are not *exactly*
> > in that they don't have a transitional device ID.
> >
> Right. They do not have transitional device ID.
I was trying to think whether we need a conformance statement
stating so. I guess this is up to the device?
Then let's make it clear. Something like:
For the SR-IOV group type,
the owner device supporting legacy configuration access commands
[assuming this is the term - do we define it somewhere? or just
list the commands]
MAY follow the rules for the PCI Device ID, Revision ID
and Subsystem Device ID for the non-transitional devices
documented in
{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery}
or do you want to make it a SHOULD?
> > At least the device id section needs extra text then to explain this?
> >
> We don't modify any of the transitional device attributes.
> In respective conformance section, it is described what requirements of legacy interface to follow.
> > Or do you just want to make them have transitional ID?
> Don't want to do that.
> Non transitional device id with the extension is just fine.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 18:07 ` [virtio-comment] " Parav Pandit
@ 2023-06-20 14:12 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-20 14:12 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
Michael,
> From: Parav Pandit <parav@nvidia.com>
> Sent: Monday, June 19, 2023 2:07 PM
> We don't need to do it now.
>
> > Current VF capability is kind of ok I guess ...
> >
> Already proposed one?
>
> > If you want admin command to query then it would need to map in PF
> memory.
> No need.
> It is similar to rest of the configuration access commands.
>
> > I understand two concerns with this:
> > - you worry that this forces ordering requirements - it's true and I
> > don't know of a good fix. So if possible use VF BAR by preference?
> > - you worry about wasting physical memory space
> > this we can fix by sticking VF# in the kick.
> >
> > As an alternative, if we can make the new command able to communicate
> > offset in PCI BAR or in VF BAR or both then I think that's also an
> > acceptable
>
> A new command proposed was for VF BAR that member driver can ask to its
> group owner driver (like rest of the config commands).
>
Are you ok with the admin command to query the notification address of the member device bar?
I presented both the proposals.
We need to conclude this now without keep flipping between two from version to version.
a. v3 with admin command
pros:
1. self-contained via AQ for rest of the control path operations.
2. Can work with future siov device who may want to support legacy
3. Doesn't bake things in low level pci capabilities which is hard to get rid of
4. Able to add new command or new response if some device wants to support this on PF (very unlikely)
b. and v6 with capability extension.
Cons:
1. Does not have above advantages
Pros:
1. Uses existing PCI capability for the extension
> A future command can tell for the PF BAR if needed and some device wants to
> do such extra implementation.
>
> > compromize:
> > drivers that have trouble mapping VF BAR after querying PF can just
> > map PF BAR. WDYT?
>
> PF BAR for notification can be done later when there is some device who really
> have the limitation and wants to do that.
> We don't see that additional need as driver notifications are present on the VF
> already as listed in the alternative.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-20 14:12 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-20 14:12 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
Michael,
> From: Parav Pandit <parav@nvidia.com>
> Sent: Monday, June 19, 2023 2:07 PM
> We don't need to do it now.
>
> > Current VF capability is kind of ok I guess ...
> >
> Already proposed one?
>
> > If you want admin command to query then it would need to map in PF
> memory.
> No need.
> It is similar to rest of the configuration access commands.
>
> > I understand two concerns with this:
> > - you worry that this forces ordering requirements - it's true and I
> > don't know of a good fix. So if possible use VF BAR by preference?
> > - you worry about wasting physical memory space
> > this we can fix by sticking VF# in the kick.
> >
> > As an alternative, if we can make the new command able to communicate
> > offset in PCI BAR or in VF BAR or both then I think that's also an
> > acceptable
>
> A new command proposed was for VF BAR that member driver can ask to its
> group owner driver (like rest of the config commands).
>
Are you ok with the admin command to query the notification address of the member device bar?
I presented both the proposals.
We need to conclude this now without keep flipping between two from version to version.
a. v3 with admin command
pros:
1. self-contained via AQ for rest of the control path operations.
2. Can work with future siov device who may want to support legacy
3. Doesn't bake things in low level pci capabilities which is hard to get rid of
4. Able to add new command or new response if some device wants to support this on PF (very unlikely)
b. and v6 with capability extension.
Cons:
1. Does not have above advantages
Pros:
1. Uses existing PCI capability for the extension
> A future command can tell for the PF BAR if needed and some device wants to
> do such extra implementation.
>
> > compromize:
> > drivers that have trouble mapping VF BAR after querying PF can just
> > map PF BAR. WDYT?
>
> PF BAR for notification can be done later when there is some device who really
> have the limitation and wants to do that.
> We don't see that additional need as driver notifications are present on the VF
> already as listed in the alternative.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-20 10:21 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-21 1:09 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 1:09 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Tuesday, June 20, 2023 6:22 AM
>
> On Tue, Jun 20, 2023 at 12:14:16AM +0000, Parav Pandit wrote:
> >
> >
> > > From: virtio-dev@lists.oasis-open.org
> > > <virtio-dev@lists.oasis-open.org> On Behalf Of Michael S. Tsirkin
> > > Sent: Monday, June 19, 2023 1:46 PM
> >
> > > These devices have a legacy interface yes?
> > Yes. partially.
> >
> > > So they should be transitional to avoid breaking assumption.
> > >
> > >
> > > But they are not *exactly*
> > > in that they don't have a transitional device ID.
> > >
> > Right. They do not have transitional device ID.
>
> I was trying to think whether we need a conformance statement stating so. I
> guess this is up to the device?
> Then let's make it clear. Something like:
> For the SR-IOV group type,
> the owner device supporting legacy configuration access commands
> [assuming this is the term - do we define it somewhere? or just
> list the commands]
This term is largely defined as section name currently.
> MAY follow the rules for the PCI Device ID, Revision ID
> and Subsystem Device ID for the non-transitional devices
> documented in
> {Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery}
>
> or do you want to make it a SHOULD?
>
SHOULD seems fine to me as one cannot build PCI VF hw practically with IOBAR anyway.
So better to extended above line as,
For the SR-IOV group type, the owner device ... and member device SHOULD fllow the rules for...
>
> > > At least the device id section needs extra text then to explain this?
> > >
> > We don't modify any of the transitional device attributes.
> > In respective conformance section, it is described what requirements of
> legacy interface to follow.
> > > Or do you just want to make them have transitional ID?
> > Don't want to do that.
> > Non transitional device id with the extension is just fine.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 1:09 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 1:09 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Tuesday, June 20, 2023 6:22 AM
>
> On Tue, Jun 20, 2023 at 12:14:16AM +0000, Parav Pandit wrote:
> >
> >
> > > From: virtio-dev@lists.oasis-open.org
> > > <virtio-dev@lists.oasis-open.org> On Behalf Of Michael S. Tsirkin
> > > Sent: Monday, June 19, 2023 1:46 PM
> >
> > > These devices have a legacy interface yes?
> > Yes. partially.
> >
> > > So they should be transitional to avoid breaking assumption.
> > >
> > >
> > > But they are not *exactly*
> > > in that they don't have a transitional device ID.
> > >
> > Right. They do not have transitional device ID.
>
> I was trying to think whether we need a conformance statement stating so. I
> guess this is up to the device?
> Then let's make it clear. Something like:
> For the SR-IOV group type,
> the owner device supporting legacy configuration access commands
> [assuming this is the term - do we define it somewhere? or just
> list the commands]
This term is largely defined as section name currently.
> MAY follow the rules for the PCI Device ID, Revision ID
> and Subsystem Device ID for the non-transitional devices
> documented in
> {Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery}
>
> or do you want to make it a SHOULD?
>
SHOULD seems fine to me as one cannot build PCI VF hw practically with IOBAR anyway.
So better to extended above line as,
For the SR-IOV group type, the owner device ... and member device SHOULD fllow the rules for...
>
> > > At least the device id section needs extra text then to explain this?
> > >
> > We don't modify any of the transitional device attributes.
> > In respective conformance section, it is described what requirements of
> legacy interface to follow.
> > > Or do you just want to make them have transitional ID?
> > Don't want to do that.
> > Non transitional device id with the extension is just fine.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 1:09 ` [virtio-comment] " Parav Pandit
@ 2023-06-21 5:05 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 5:05 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 01:09:40AM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Tuesday, June 20, 2023 6:22 AM
> >
> > On Tue, Jun 20, 2023 at 12:14:16AM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: virtio-dev@lists.oasis-open.org
> > > > <virtio-dev@lists.oasis-open.org> On Behalf Of Michael S. Tsirkin
> > > > Sent: Monday, June 19, 2023 1:46 PM
> > >
> > > > These devices have a legacy interface yes?
> > > Yes. partially.
> > >
> > > > So they should be transitional to avoid breaking assumption.
> > > >
> > > >
> > > > But they are not *exactly*
> > > > in that they don't have a transitional device ID.
> > > >
> > > Right. They do not have transitional device ID.
> >
> > I was trying to think whether we need a conformance statement stating so. I
> > guess this is up to the device?
> > Then let's make it clear. Something like:
> > For the SR-IOV group type,
> > the owner device supporting legacy configuration access commands
> > [assuming this is the term - do we define it somewhere? or just
> > list the commands]
> This term is largely defined as section name currently.
Right. A sentence saying "the following legacy configuration access
commands allow access to the legacy interface of a member device through
the owner device ..." at the beginning of the section might be a good idea.
> > MAY follow the rules for the PCI Device ID, Revision ID
> > and Subsystem Device ID for the non-transitional devices
> > documented in
> > {Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery}
> >
> > or do you want to make it a SHOULD?
> >
> SHOULD seems fine to me as one cannot build PCI VF hw practically with IOBAR anyway.
> So better to extended above line as,
>
> For the SR-IOV group type, the owner device ... and member device SHOULD fllow the rules for...
ok
> >
> > > > At least the device id section needs extra text then to explain this?
> > > >
> > > We don't modify any of the transitional device attributes.
> > > In respective conformance section, it is described what requirements of
> > legacy interface to follow.
> > > > Or do you just want to make them have transitional ID?
> > > Don't want to do that.
> > > Non transitional device id with the extension is just fine.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 5:05 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 5:05 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 01:09:40AM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Tuesday, June 20, 2023 6:22 AM
> >
> > On Tue, Jun 20, 2023 at 12:14:16AM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: virtio-dev@lists.oasis-open.org
> > > > <virtio-dev@lists.oasis-open.org> On Behalf Of Michael S. Tsirkin
> > > > Sent: Monday, June 19, 2023 1:46 PM
> > >
> > > > These devices have a legacy interface yes?
> > > Yes. partially.
> > >
> > > > So they should be transitional to avoid breaking assumption.
> > > >
> > > >
> > > > But they are not *exactly*
> > > > in that they don't have a transitional device ID.
> > > >
> > > Right. They do not have transitional device ID.
> >
> > I was trying to think whether we need a conformance statement stating so. I
> > guess this is up to the device?
> > Then let's make it clear. Something like:
> > For the SR-IOV group type,
> > the owner device supporting legacy configuration access commands
> > [assuming this is the term - do we define it somewhere? or just
> > list the commands]
> This term is largely defined as section name currently.
Right. A sentence saying "the following legacy configuration access
commands allow access to the legacy interface of a member device through
the owner device ..." at the beginning of the section might be a good idea.
> > MAY follow the rules for the PCI Device ID, Revision ID
> > and Subsystem Device ID for the non-transitional devices
> > documented in
> > {Virtio Transport Options / Virtio Over PCI Bus / PCI Device Discovery}
> >
> > or do you want to make it a SHOULD?
> >
> SHOULD seems fine to me as one cannot build PCI VF hw practically with IOBAR anyway.
> So better to extended above line as,
>
> For the SR-IOV group type, the owner device ... and member device SHOULD fllow the rules for...
ok
> >
> > > > At least the device id section needs extra text then to explain this?
> > > >
> > > We don't modify any of the transitional device attributes.
> > > In respective conformance section, it is described what requirements of
> > legacy interface to follow.
> > > > Or do you just want to make them have transitional ID?
> > > Don't want to do that.
> > > Non transitional device id with the extension is just fine.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-20 14:12 ` [virtio-comment] " Parav Pandit
@ 2023-06-21 15:50 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 15:50 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Parav Pandit
> Sent: Tuesday, June 20, 2023 10:12 AM
>
> Michael,
>
> > From: Parav Pandit <parav@nvidia.com>
> > Sent: Monday, June 19, 2023 2:07 PM
> > We don't need to do it now.
> >
> > > Current VF capability is kind of ok I guess ...
> > >
> > Already proposed one?
> >
> > > If you want admin command to query then it would need to map in PF
> > memory.
> > No need.
> > It is similar to rest of the configuration access commands.
> >
> > > I understand two concerns with this:
> > > - you worry that this forces ordering requirements - it's true and I
> > > don't know of a good fix. So if possible use VF BAR by preference?
> > > - you worry about wasting physical memory space
> > > this we can fix by sticking VF# in the kick.
> > >
> > > As an alternative, if we can make the new command able to
> > > communicate offset in PCI BAR or in VF BAR or both then I think
> > > that's also an acceptable
> >
> > A new command proposed was for VF BAR that member driver can ask to
> > its group owner driver (like rest of the config commands).
> >
>
> Are you ok with the admin command to query the notification address of the
> member device bar?
>
> I presented both the proposals.
> We need to conclude this now without keep flipping between two from version
> to version.
>
> a. v3 with admin command
> pros:
> 1. self-contained via AQ for rest of the control path operations.
> 2. Can work with future siov device who may want to support legacy 3. Doesn't
> bake things in low level pci capabilities which is hard to get rid of 4. Able to add
> new command or new response if some device wants to support this on PF
> (very unlikely)
>
Given that this has more pros and no objection to it,
I will revise v7 today with query admin command like v3.
> b. and v6 with capability extension.
> Cons:
> 1. Does not have above advantages
>
> Pros:
> 1. Uses existing PCI capability for the extension
>
>
> > A future command can tell for the PF BAR if needed and some device
> > wants to do such extra implementation.
> >
> > > compromize:
> > > drivers that have trouble mapping VF BAR after querying PF can just
> > > map PF BAR. WDYT?
> >
> > PF BAR for notification can be done later when there is some device
> > who really have the limitation and wants to do that.
> > We don't see that additional need as driver notifications are present
> > on the VF already as listed in the alternative.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 15:50 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 15:50 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Parav Pandit
> Sent: Tuesday, June 20, 2023 10:12 AM
>
> Michael,
>
> > From: Parav Pandit <parav@nvidia.com>
> > Sent: Monday, June 19, 2023 2:07 PM
> > We don't need to do it now.
> >
> > > Current VF capability is kind of ok I guess ...
> > >
> > Already proposed one?
> >
> > > If you want admin command to query then it would need to map in PF
> > memory.
> > No need.
> > It is similar to rest of the configuration access commands.
> >
> > > I understand two concerns with this:
> > > - you worry that this forces ordering requirements - it's true and I
> > > don't know of a good fix. So if possible use VF BAR by preference?
> > > - you worry about wasting physical memory space
> > > this we can fix by sticking VF# in the kick.
> > >
> > > As an alternative, if we can make the new command able to
> > > communicate offset in PCI BAR or in VF BAR or both then I think
> > > that's also an acceptable
> >
> > A new command proposed was for VF BAR that member driver can ask to
> > its group owner driver (like rest of the config commands).
> >
>
> Are you ok with the admin command to query the notification address of the
> member device bar?
>
> I presented both the proposals.
> We need to conclude this now without keep flipping between two from version
> to version.
>
> a. v3 with admin command
> pros:
> 1. self-contained via AQ for rest of the control path operations.
> 2. Can work with future siov device who may want to support legacy 3. Doesn't
> bake things in low level pci capabilities which is hard to get rid of 4. Able to add
> new command or new response if some device wants to support this on PF
> (very unlikely)
>
Given that this has more pros and no objection to it,
I will revise v7 today with query admin command like v3.
> b. and v6 with capability extension.
> Cons:
> 1. Does not have above advantages
>
> Pros:
> 1. Uses existing PCI capability for the extension
>
>
> > A future command can tell for the PF BAR if needed and some device
> > wants to do such extra implementation.
> >
> > > compromize:
> > > drivers that have trouble mapping VF BAR after querying PF can just
> > > map PF BAR. WDYT?
> >
> > PF BAR for notification can be done later when there is some device
> > who really have the limitation and wants to do that.
> > We don't see that additional need as driver notifications are present
> > on the VF already as listed in the alternative.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 15:50 ` [virtio-comment] " Parav Pandit
@ 2023-06-21 15:56 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 15:56 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 03:50:00PM +0000, Parav Pandit wrote:
>
>
> > From: Parav Pandit
> > Sent: Tuesday, June 20, 2023 10:12 AM
> >
> > Michael,
> >
> > > From: Parav Pandit <parav@nvidia.com>
> > > Sent: Monday, June 19, 2023 2:07 PM
> > > We don't need to do it now.
> > >
> > > > Current VF capability is kind of ok I guess ...
> > > >
> > > Already proposed one?
> > >
> > > > If you want admin command to query then it would need to map in PF
> > > memory.
> > > No need.
> > > It is similar to rest of the configuration access commands.
> > >
> > > > I understand two concerns with this:
> > > > - you worry that this forces ordering requirements - it's true and I
> > > > don't know of a good fix. So if possible use VF BAR by preference?
> > > > - you worry about wasting physical memory space
> > > > this we can fix by sticking VF# in the kick.
> > > >
> > > > As an alternative, if we can make the new command able to
> > > > communicate offset in PCI BAR or in VF BAR or both then I think
> > > > that's also an acceptable
> > >
> > > A new command proposed was for VF BAR that member driver can ask to
> > > its group owner driver (like rest of the config commands).
> > >
> >
> > Are you ok with the admin command to query the notification address of the
> > member device bar?
> >
> > I presented both the proposals.
> > We need to conclude this now without keep flipping between two from version
> > to version.
> >
> > a. v3 with admin command
> > pros:
> > 1. self-contained via AQ for rest of the control path operations.
> > 2. Can work with future siov device who may want to support legacy 3. Doesn't
> > bake things in low level pci capabilities which is hard to get rid of 4. Able to add
> > new command or new response if some device wants to support this on PF
> > (very unlikely)
> >
> Given that this has more pros and no objection to it,
> I will revise v7 today with query admin command like v3.
Not like v3 please. If you want to re-open this:
First I think we need multiple offsets just like notification
capaiblity, sorted by priority. Second I think we need ability to report
offset within owner not member.
So please add ability to report multiple offsets, and add e.g.
a flags field, with bits for owner, member.
> > b. and v6 with capability extension.
> > Cons:
> > 1. Does not have above advantages
> >
> > Pros:
> > 1. Uses existing PCI capability for the extension
> >
> >
> > > A future command can tell for the PF BAR if needed and some device
> > > wants to do such extra implementation.
> > >
> > > > compromize:
> > > > drivers that have trouble mapping VF BAR after querying PF can just
> > > > map PF BAR. WDYT?
> > >
> > > PF BAR for notification can be done later when there is some device
> > > who really have the limitation and wants to do that.
> > > We don't see that additional need as driver notifications are present
> > > on the VF already as listed in the alternative.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] Re: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 15:56 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 15:56 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 03:50:00PM +0000, Parav Pandit wrote:
>
>
> > From: Parav Pandit
> > Sent: Tuesday, June 20, 2023 10:12 AM
> >
> > Michael,
> >
> > > From: Parav Pandit <parav@nvidia.com>
> > > Sent: Monday, June 19, 2023 2:07 PM
> > > We don't need to do it now.
> > >
> > > > Current VF capability is kind of ok I guess ...
> > > >
> > > Already proposed one?
> > >
> > > > If you want admin command to query then it would need to map in PF
> > > memory.
> > > No need.
> > > It is similar to rest of the configuration access commands.
> > >
> > > > I understand two concerns with this:
> > > > - you worry that this forces ordering requirements - it's true and I
> > > > don't know of a good fix. So if possible use VF BAR by preference?
> > > > - you worry about wasting physical memory space
> > > > this we can fix by sticking VF# in the kick.
> > > >
> > > > As an alternative, if we can make the new command able to
> > > > communicate offset in PCI BAR or in VF BAR or both then I think
> > > > that's also an acceptable
> > >
> > > A new command proposed was for VF BAR that member driver can ask to
> > > its group owner driver (like rest of the config commands).
> > >
> >
> > Are you ok with the admin command to query the notification address of the
> > member device bar?
> >
> > I presented both the proposals.
> > We need to conclude this now without keep flipping between two from version
> > to version.
> >
> > a. v3 with admin command
> > pros:
> > 1. self-contained via AQ for rest of the control path operations.
> > 2. Can work with future siov device who may want to support legacy 3. Doesn't
> > bake things in low level pci capabilities which is hard to get rid of 4. Able to add
> > new command or new response if some device wants to support this on PF
> > (very unlikely)
> >
> Given that this has more pros and no objection to it,
> I will revise v7 today with query admin command like v3.
Not like v3 please. If you want to re-open this:
First I think we need multiple offsets just like notification
capaiblity, sorted by priority. Second I think we need ability to report
offset within owner not member.
So please add ability to report multiple offsets, and add e.g.
a flags field, with bits for owner, member.
> > b. and v6 with capability extension.
> > Cons:
> > 1. Does not have above advantages
> >
> > Pros:
> > 1. Uses existing PCI capability for the extension
> >
> >
> > > A future command can tell for the PF BAR if needed and some device
> > > wants to do such extra implementation.
> > >
> > > > compromize:
> > > > drivers that have trouble mapping VF BAR after querying PF can just
> > > > map PF BAR. WDYT?
> > >
> > > PF BAR for notification can be done later when there is some device
> > > who really have the limitation and wants to do that.
> > > We don't see that additional need as driver notifications are present
> > > on the VF already as listed in the alternative.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 15:56 ` [virtio-comment] " Michael S. Tsirkin
@ 2023-06-21 16:01 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 16:01 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 11:56 AM
> Not like v3 please. If you want to re-open this:
>
>
> First I think we need multiple offsets just like notification capaiblity, sorted by
> priority. Second I think we need ability to report offset within owner not
> member.
>
> So please add ability to report multiple offsets, and add e.g.
> a flags field, with bits for owner, member.
What are these multiple offsets for?
Do you mean two offsets? One for owner and one for member?
Both bits are optional. Do you agree?
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 16:01 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 16:01 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 11:56 AM
> Not like v3 please. If you want to re-open this:
>
>
> First I think we need multiple offsets just like notification capaiblity, sorted by
> priority. Second I think we need ability to report offset within owner not
> member.
>
> So please add ability to report multiple offsets, and add e.g.
> a flags field, with bits for owner, member.
What are these multiple offsets for?
Do you mean two offsets? One for owner and one for member?
Both bits are optional. Do you agree?
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 16:01 ` [virtio-comment] " Parav Pandit
@ 2023-06-21 19:43 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 19:43 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 04:01:00PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, June 21, 2023 11:56 AM
>
> > Not like v3 please. If you want to re-open this:
> >
> >
> > First I think we need multiple offsets just like notification capaiblity, sorted by
> > priority. Second I think we need ability to report offset within owner not
> > member.
> >
> > So please add ability to report multiple offsets, and add e.g.
> > a flags field, with bits for owner, member.
>
> What are these multiple offsets for?
Different BARs. For example, IO versus memory. Yes I know VFs don't
support IO but PFs do. prefetch vs non-prefetch might matter too
(non-prefetch is mostly limited to 32 bit).
> Do you mean two offsets? One for owner and one for member?
> Both bits are optional. Do you agree?
I agree that everything should be optional sure. Let's not limit it to
two though. There could be multiple BARs or whatever. We have ability
to pass buffers of arbitrary length and device reports the length.
Maybe have a SHOULD recommending not more than 10 of these (1 for each
possible PCI BAR one for each possible VF BAR).
This is to match the functionality that notification capability has.
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 19:43 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 19:43 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 04:01:00PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, June 21, 2023 11:56 AM
>
> > Not like v3 please. If you want to re-open this:
> >
> >
> > First I think we need multiple offsets just like notification capaiblity, sorted by
> > priority. Second I think we need ability to report offset within owner not
> > member.
> >
> > So please add ability to report multiple offsets, and add e.g.
> > a flags field, with bits for owner, member.
>
> What are these multiple offsets for?
Different BARs. For example, IO versus memory. Yes I know VFs don't
support IO but PFs do. prefetch vs non-prefetch might matter too
(non-prefetch is mostly limited to 32 bit).
> Do you mean two offsets? One for owner and one for member?
> Both bits are optional. Do you agree?
I agree that everything should be optional sure. Let's not limit it to
two though. There could be multiple BARs or whatever. We have ability
to pass buffers of arbitrary length and device reports the length.
Maybe have a SHOULD recommending not more than 10 of these (1 for each
possible PCI BAR one for each possible VF BAR).
This is to match the functionality that notification capability has.
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 18:12 ` [virtio-comment] " Parav Pandit
@ 2023-06-21 19:47 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 19:47 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 06:12:11PM +0000, Parav Pandit wrote:
> And yes lets please stop calling the transport vq as well, as it is not going to "transport everything".
Good point, this transport is actually virtio over admin commands.
admin transport for short?
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 19:47 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 19:47 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 06:12:11PM +0000, Parav Pandit wrote:
> And yes lets please stop calling the transport vq as well, as it is not going to "transport everything".
Good point, this transport is actually virtio over admin commands.
admin transport for short?
--
MST
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 19:43 ` Michael S. Tsirkin
@ 2023-06-21 20:04 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 20:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 3:43 PM
>
> On Wed, Jun 21, 2023 at 04:01:00PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, June 21, 2023 11:56 AM
> >
> > > Not like v3 please. If you want to re-open this:
> > >
> > >
> > > First I think we need multiple offsets just like notification
> > > capaiblity, sorted by priority. Second I think we need ability to
> > > report offset within owner not member.
> > >
> > > So please add ability to report multiple offsets, and add e.g.
> > > a flags field, with bits for owner, member.
> >
> > What are these multiple offsets for?
>
> Different BARs. For example, IO versus memory. Yes I know VFs don't support
> IO but PFs do. prefetch vs non-prefetch might matter too (non-prefetch is
> mostly limited to 32 bit).
>
BAR type and its prefetch attributes are told by the PCI anyway so no duplication here anyway.
For owner and member I get it.
>
> > Do you mean two offsets? One for owner and one for member?
> > Both bits are optional. Do you agree?
>
> I agree that everything should be optional sure. Let's not limit it to two though.
> There could be multiple BARs or whatever. We have ability to pass buffers of
> arbitrary length and device reports the length.
> Maybe have a SHOULD recommending not more than 10 of these (1 for each
> possible PCI BAR one for each possible VF BAR).
>
> This is to match the functionality that notification capability has.
>
A device can expose multiple offsets within the single bar to a count of 100 entries;
Also allowed in the notification capability.
(I don't see why any sane device would do that anyway)
So I will avoid such extra restrictions and limit the scope of this patch to be minimal to have arbitrary number of BAR entries limited by the command result length.
>
>
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and to
> > minimize spam in the list archive, subscription is required before
> > posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License:
> > https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines:
> > https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 20:04 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 20:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 3:43 PM
>
> On Wed, Jun 21, 2023 at 04:01:00PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, June 21, 2023 11:56 AM
> >
> > > Not like v3 please. If you want to re-open this:
> > >
> > >
> > > First I think we need multiple offsets just like notification
> > > capaiblity, sorted by priority. Second I think we need ability to
> > > report offset within owner not member.
> > >
> > > So please add ability to report multiple offsets, and add e.g.
> > > a flags field, with bits for owner, member.
> >
> > What are these multiple offsets for?
>
> Different BARs. For example, IO versus memory. Yes I know VFs don't support
> IO but PFs do. prefetch vs non-prefetch might matter too (non-prefetch is
> mostly limited to 32 bit).
>
BAR type and its prefetch attributes are told by the PCI anyway so no duplication here anyway.
For owner and member I get it.
>
> > Do you mean two offsets? One for owner and one for member?
> > Both bits are optional. Do you agree?
>
> I agree that everything should be optional sure. Let's not limit it to two though.
> There could be multiple BARs or whatever. We have ability to pass buffers of
> arbitrary length and device reports the length.
> Maybe have a SHOULD recommending not more than 10 of these (1 for each
> possible PCI BAR one for each possible VF BAR).
>
> This is to match the functionality that notification capability has.
>
A device can expose multiple offsets within the single bar to a count of 100 entries;
Also allowed in the notification capability.
(I don't see why any sane device would do that anyway)
So I will avoid such extra restrictions and limit the scope of this patch to be minimal to have arbitrary number of BAR entries limited by the command result length.
>
>
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and to
> > minimize spam in the list archive, subscription is required before
> > posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License:
> > https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines:
> > https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-19 21:07 ` [virtio-comment] " Parav Pandit
@ 2023-06-21 20:05 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 20:05 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 09:07:56PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:17 PM
>
> > I'd love to just put this stuff in a separate tex document and just link there from
> > the generic thing.
> > Or if it's too hard, please create new sections for this stuff. But in this case each
> > of these sections has to start with repeating the description when things apply.
> > E.g. "when using legacy interface"
> > when using commands "A B C to access" etc etc.
> >
> > I wrote a bunch of comments below.
> >
> I will stick to them as most things are taken out to generic section anyway in patch 3.
> Only PCI transport part stays here.
> I will move some more text to generic section.
>
> > > +As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]}
> > > +PCI VFs do not support the I/O space BAR. Due to this limitation, a
> > > +PCI VF hardware device that supports legacy interfaces cannot be
> > > +attached as passthrough device to the guest virtual machine.
> >
> > That's not the only reason. Let's keep things generic and start by saying
> > something like "Some configurations with a need to support legacy drivers can
> > not implement a transitional device. For example .... ".
> >
> I will avoid For example part and move this to generic section.
>
> > > +
> > > +To have a transitional or legacy virtio device in the guest virtual
> > > +machine, a PCI PF device as a group owner may
> >
> > may -> can outside conformance statements.
> >
> Ack.
>
> > > support accessing its group member
> >
> > Pls keep it generic. A group owner ....
> >
> Ack
>
> >
> > > +device's legacy configuration region using the group administration
> > > +commands listed at \ref{sec:Basic Facilities of a Virtio Device /
> > > +Device groups / Group administration commands / Legacy Interfaces: legacy
> > configuration access commands}.
> > > +These legacy configuration access commands are transported via the
> > > +administration virtqueue.
> >
> > are these called legacy configuration access commands then?
> > why do you introduce this term? it does not look like it's used anywhere else.
> >
> It is used and referred PCI section " Legacy Interfaces: Group member device Configuration Region Access".
> >
> > I would prefer it if you just use the term "legacy interface"
> > it is well defined and widely used in the spec.
> >
> It is "legacy interface", with suffix for group member config access.
> In future, there may be a section for other things than config access.
>
> > again leave virtqueue out of it pls we'll have other ways down the road. what
> > matters is that they are transferred to a group member by sending legacy
> > configuration access commands to the group owner.
> >
> Ack.
> It is theory of operation, not a normative.
> Tomorrow if other than aq exists, one will write as "AQ or xyz"..
>
> But fine to me to reduce changes from 216 to less lines. :)
>
> >
> > > This mechanism enables minimal involvement of the
> > > +hypervisor software only for the purpose of the legacy configuration region
> > access.
> >
> > Leave marketing out of it pls, kill last sentence.
> >
> Huh, I don't see theory of operation and scope of the hypervisor as marketing at all.
> I will remove as it makes this patch shorter.
>
> >
> >
> > > +
> > > +When a guest virtual machine requests legacy configuration and device
> > > +specific regions
> >
> > wait so now it's legacy configuration and device specific regions and these two
> > are distinct?
> >
> > > of group member device, a hypervisor software accesses
> > > +it using an administration virtqueue
> >
> > please talk of administration commands. virtqueue does not matter at all.
> >
> Ack.
>
> > > on behalf of the guest virtual machine.
> >
> >
> >
> > To put it in our terms, something like this:
> > when a legacy driver accesses a member device of
> > a group using the
> > legacy interface, a modern driver can intercept
> > the access and forward it to the owner device.
> >
> I will not mention "modern driver" as it has zero reference in spec and don't want to bring Linux terms here.
> "the driver can intercept" is enough.
Good point but since you say "a legacy driver" then "the driver" seems
to refer to exactly that. How about:
when a legacy member device driver accesses a member device of
a group using the
legacy interface, an owner device driver can intercept
the access and forward it to the owner device.
>
> >
> >
> > > +
> > > +Even though virtqueue driver notifications can be communicated
> > > +through administration virtqueue, if the group member device support
> > > +such
> >
> > supports?
> >
> Ack.
>
> > > +notifications using a memory-mapped operation, such driver
> > > +notifications are sent using a group member device defined notification
> > region.
> >
> > what does "are sent" imply here? is there a matching conformance statement?
> > who sends them?
> >
> It has the same meaning as "Driver notifications" section first line has.
> Isnt it implicit that driver notifications are sent by driver based on "Driver notifications" section?
>
> > > +
> > > +The group owner device
> >
> > this is the 1st you mention any group or its owner.
> >
> > > should not expose PCI BAR0
> >
> > Not PCI BAR0. That is in header. You mean VF BAR0.
> >
> I will rewrite it as,
>
> The group owner device should not expose PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it prefers to
> support legacy interface for legacy configuration access using this group owner.
This seems to ignore all my comments.
> > > for the group member
> > > +devices when it prefers to support legacy interface for legacy
> > > +configuration access using its group owner.
> >
> > don't use should outside conformance
> >
> What should be used instead of "should"?
Depends on context, generally we just say what it does. E.g. "VF BAR0
is hardwired to zero".
> >
> > I think this specific case actually should be more specific.
> > Something like:
> >
> > - For PCI SRIOV groups, when group owner device implements commands
> > A,B,C the group owner's VF BAR 0 is hardwired to 0
> > in its PCI SRIOV capability.
> >
> I wrote it slightly differently above.
I don't see why what you wrote is any better than what you had to be
frank. You are coming up with our own terminology instead of just
using what's in the SR IOV spec. Please don't.
> >
> > > +This facilitates hypervisor software to operate with least amount of
> > > +complexities
> >
> > complexity is its own plural
> >
> > > to emulate the legacy interface I/O space BAR and passthrough
> > > +other PCI BARs and PCI device capabilities to the guest virtual
> > > +machine without any translation.
> > > +
> > > +The group member device should not expose PCI BAR0 in various PCI
> > capabilities.
> > > +
> > > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > > +Member Device Legacy Configuration Access}
> > > +
> > > +When a PCI SR-IOV group owner device supports
> > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> > owner
> > > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > > +This is to facilitate the software to emulate I/O region BAR0 for supporting
> > the legacy interface.
> >
> > not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose it". if you
> > want to register can be "unimplemented".
> > Base Address registers are hardwired to zero
> >
> > But it is better to be specific on what should happen. hardwire VF BAR0 to 0,
> > right?
> >
> Hardwire to zero and not expose is same thing to me.
Maybe, though previously you said it just means not put any capabilities
there. More importantly I don't see expose or not expose anywhere in
the PCI or SRIOV spec. When spec wants to say how not to have a given
BAR it says exactly hardwired to zero.
> >
> > > +
> > > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > > +Member Device Legacy Configuration Access}
> > > +
> > > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > > +member exposes a PCI BAR0.
> >
> > what does "emulate" mean here? which driver it is?
> >
> I will add the line to theory of operation, but I recollect "emulate" line came from you.
I generally am fine with using this term but we need to then
define it before use.
This specific thing I would just drop.
Instead of wasting time just say device MUST hardwire VF BAR0 to zero as
opposed to SHOULD, and we do not need to worry about how drivers
recover. If they want to they will validate, if they don't then they
won't.
> > > diff --git a/transport-pci.tex b/transport-pci.tex index
> > > a5c6719..3647485 100644
> > > --- a/transport-pci.tex
> > > +++ b/transport-pci.tex
> > > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > > struct virtio_pci_cap cap;
> > > le32 notify_off_multiplier; /* Multiplier for
> > > queue_notify_off. */
> > > + u8 legacy_q_notify_supported;
> >
> > I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> >
> >
> > > + u8 reserved[3];
> >
> >
> > align with spaces pls.
> >
> Ack.
>
> >
> > > };
> > > \end{lstlisting}
> > >
> > > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > > layout}\label{sec:Virtio Transport Options the same Queue Notify address for
> > all queues.
> > > \end{note}
> > >
> > > +\field{legacy_q_notify_supported} when set to 1,
> >
> >
> > Do you mean something like:
> > When \field{legacy_q_notify_supported} this indicates that ...
> >
> Yes will fix.
>
> > > indicates that the device
> > > +supports legacy queue notifications at this notification location.
> >
> > And what location? this refers to what? the location described by this capability
> > maybe?
> >
> Ack.
>
> >
> > More specifically, assume a transitional device (with an IO BAR as normal). Does
> > presence of this flag in the capability imply we can use a legacy queue but use
> > this notification with this capability?
> I don't know how this combination can be even possible by the device other than a buggy software who will do both.
>
> > I worry that if we allow that, it opens floodgates of people who are too lazy to
> > upgrade to 1.0 but just hack this notification in there.
> >
> Don't understand the concern.
> If a device has transitional device, it is 1.0 device supporting legacy with IOBAR and newer interface.
> So...
if we are switching to admin commands for this, then let's not argue
about it.
> >
> >
> >
> >
> > > Legacy Queue
> > > +Notification address is derived within a BAR for a virtqueue:
> > > +
> > > +\begin{lstlisting}
> > > + cap.offset
> > > +\end{lstlisting}
> > > +
> > > \devicenormative{\paragraph}{Notification capability}{Virtio
> > > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> > capability} The device MUST present at least one notification capability.
> > >
> >
> > I don't get what this is saying. I *think* you really mean is that
> > queue_notify_off is 0.
> >
> Please refer to the current notification address definition.
> Above is written in similar way where it doesnt need to describe queue_notify_off.
> >
> > > @@ -596,6 +606,14 @@ \subsubsection{Notification structure
> > > layout}\label{sec:Virtio Transport Options cap.length >=
> > > queue_notify_off * notify_off_multiplier + 4 \end{lstlisting}
> > >
> > > +\paragraph{Legacy Interfaces: Notification
> > > +capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus
> > > +/ PCI Device Layout / Legacy Interfaces: Notification capability}
> > > +
> > > +The device SHOULD set \field{legacy_q_notify_supported}
> >
> > why SHOULD and not MUST here - if driver checks for it then it won't work
> > without no?
> >
> If device doesn't want to support queue notifications, it has ability to set to 0.
>
> > > when the device
> > > +notification location supports legacy driver notifications.
> >
> > legacy driver notifications is ambigous.
> > what you want here and elsewhere is something like "when used through a
> > legacy interface, device supports driver
> >
> Ack.
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 20:05 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 20:05 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Mon, Jun 19, 2023 at 09:07:56PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, June 19, 2023 12:17 PM
>
> > I'd love to just put this stuff in a separate tex document and just link there from
> > the generic thing.
> > Or if it's too hard, please create new sections for this stuff. But in this case each
> > of these sections has to start with repeating the description when things apply.
> > E.g. "when using legacy interface"
> > when using commands "A B C to access" etc etc.
> >
> > I wrote a bunch of comments below.
> >
> I will stick to them as most things are taken out to generic section anyway in patch 3.
> Only PCI transport part stays here.
> I will move some more text to generic section.
>
> > > +As described in PCIe base specification \hyperref[intro:PCIe]{[PCIe]}
> > > +PCI VFs do not support the I/O space BAR. Due to this limitation, a
> > > +PCI VF hardware device that supports legacy interfaces cannot be
> > > +attached as passthrough device to the guest virtual machine.
> >
> > That's not the only reason. Let's keep things generic and start by saying
> > something like "Some configurations with a need to support legacy drivers can
> > not implement a transitional device. For example .... ".
> >
> I will avoid For example part and move this to generic section.
>
> > > +
> > > +To have a transitional or legacy virtio device in the guest virtual
> > > +machine, a PCI PF device as a group owner may
> >
> > may -> can outside conformance statements.
> >
> Ack.
>
> > > support accessing its group member
> >
> > Pls keep it generic. A group owner ....
> >
> Ack
>
> >
> > > +device's legacy configuration region using the group administration
> > > +commands listed at \ref{sec:Basic Facilities of a Virtio Device /
> > > +Device groups / Group administration commands / Legacy Interfaces: legacy
> > configuration access commands}.
> > > +These legacy configuration access commands are transported via the
> > > +administration virtqueue.
> >
> > are these called legacy configuration access commands then?
> > why do you introduce this term? it does not look like it's used anywhere else.
> >
> It is used and referred PCI section " Legacy Interfaces: Group member device Configuration Region Access".
> >
> > I would prefer it if you just use the term "legacy interface"
> > it is well defined and widely used in the spec.
> >
> It is "legacy interface", with suffix for group member config access.
> In future, there may be a section for other things than config access.
>
> > again leave virtqueue out of it pls we'll have other ways down the road. what
> > matters is that they are transferred to a group member by sending legacy
> > configuration access commands to the group owner.
> >
> Ack.
> It is theory of operation, not a normative.
> Tomorrow if other than aq exists, one will write as "AQ or xyz"..
>
> But fine to me to reduce changes from 216 to less lines. :)
>
> >
> > > This mechanism enables minimal involvement of the
> > > +hypervisor software only for the purpose of the legacy configuration region
> > access.
> >
> > Leave marketing out of it pls, kill last sentence.
> >
> Huh, I don't see theory of operation and scope of the hypervisor as marketing at all.
> I will remove as it makes this patch shorter.
>
> >
> >
> > > +
> > > +When a guest virtual machine requests legacy configuration and device
> > > +specific regions
> >
> > wait so now it's legacy configuration and device specific regions and these two
> > are distinct?
> >
> > > of group member device, a hypervisor software accesses
> > > +it using an administration virtqueue
> >
> > please talk of administration commands. virtqueue does not matter at all.
> >
> Ack.
>
> > > on behalf of the guest virtual machine.
> >
> >
> >
> > To put it in our terms, something like this:
> > when a legacy driver accesses a member device of
> > a group using the
> > legacy interface, a modern driver can intercept
> > the access and forward it to the owner device.
> >
> I will not mention "modern driver" as it has zero reference in spec and don't want to bring Linux terms here.
> "the driver can intercept" is enough.
Good point but since you say "a legacy driver" then "the driver" seems
to refer to exactly that. How about:
when a legacy member device driver accesses a member device of
a group using the
legacy interface, an owner device driver can intercept
the access and forward it to the owner device.
>
> >
> >
> > > +
> > > +Even though virtqueue driver notifications can be communicated
> > > +through administration virtqueue, if the group member device support
> > > +such
> >
> > supports?
> >
> Ack.
>
> > > +notifications using a memory-mapped operation, such driver
> > > +notifications are sent using a group member device defined notification
> > region.
> >
> > what does "are sent" imply here? is there a matching conformance statement?
> > who sends them?
> >
> It has the same meaning as "Driver notifications" section first line has.
> Isnt it implicit that driver notifications are sent by driver based on "Driver notifications" section?
>
> > > +
> > > +The group owner device
> >
> > this is the 1st you mention any group or its owner.
> >
> > > should not expose PCI BAR0
> >
> > Not PCI BAR0. That is in header. You mean VF BAR0.
> >
> I will rewrite it as,
>
> The group owner device should not expose PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it prefers to
> support legacy interface for legacy configuration access using this group owner.
This seems to ignore all my comments.
> > > for the group member
> > > +devices when it prefers to support legacy interface for legacy
> > > +configuration access using its group owner.
> >
> > don't use should outside conformance
> >
> What should be used instead of "should"?
Depends on context, generally we just say what it does. E.g. "VF BAR0
is hardwired to zero".
> >
> > I think this specific case actually should be more specific.
> > Something like:
> >
> > - For PCI SRIOV groups, when group owner device implements commands
> > A,B,C the group owner's VF BAR 0 is hardwired to 0
> > in its PCI SRIOV capability.
> >
> I wrote it slightly differently above.
I don't see why what you wrote is any better than what you had to be
frank. You are coming up with our own terminology instead of just
using what's in the SR IOV spec. Please don't.
> >
> > > +This facilitates hypervisor software to operate with least amount of
> > > +complexities
> >
> > complexity is its own plural
> >
> > > to emulate the legacy interface I/O space BAR and passthrough
> > > +other PCI BARs and PCI device capabilities to the guest virtual
> > > +machine without any translation.
> > > +
> > > +The group member device should not expose PCI BAR0 in various PCI
> > capabilities.
> > > +
> > > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > > +Member Device Legacy Configuration Access}
> > > +
> > > +When a PCI SR-IOV group owner device supports
> > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> > owner
> > > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > > +This is to facilitate the software to emulate I/O region BAR0 for supporting
> > the legacy interface.
> >
> > not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose it". if you
> > want to register can be "unimplemented".
> > Base Address registers are hardwired to zero
> >
> > But it is better to be specific on what should happen. hardwire VF BAR0 to 0,
> > right?
> >
> Hardwire to zero and not expose is same thing to me.
Maybe, though previously you said it just means not put any capabilities
there. More importantly I don't see expose or not expose anywhere in
the PCI or SRIOV spec. When spec wants to say how not to have a given
BAR it says exactly hardwired to zero.
> >
> > > +
> > > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements: Group
> > > +Member Device Legacy Configuration Access}
> > > +
> > > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > > +member exposes a PCI BAR0.
> >
> > what does "emulate" mean here? which driver it is?
> >
> I will add the line to theory of operation, but I recollect "emulate" line came from you.
I generally am fine with using this term but we need to then
define it before use.
This specific thing I would just drop.
Instead of wasting time just say device MUST hardwire VF BAR0 to zero as
opposed to SHOULD, and we do not need to worry about how drivers
recover. If they want to they will validate, if they don't then they
won't.
> > > diff --git a/transport-pci.tex b/transport-pci.tex index
> > > a5c6719..3647485 100644
> > > --- a/transport-pci.tex
> > > +++ b/transport-pci.tex
> > > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > > struct virtio_pci_cap cap;
> > > le32 notify_off_multiplier; /* Multiplier for
> > > queue_notify_off. */
> > > + u8 legacy_q_notify_supported;
> >
> > I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> >
> >
> > > + u8 reserved[3];
> >
> >
> > align with spaces pls.
> >
> Ack.
>
> >
> > > };
> > > \end{lstlisting}
> > >
> > > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > > layout}\label{sec:Virtio Transport Options the same Queue Notify address for
> > all queues.
> > > \end{note}
> > >
> > > +\field{legacy_q_notify_supported} when set to 1,
> >
> >
> > Do you mean something like:
> > When \field{legacy_q_notify_supported} this indicates that ...
> >
> Yes will fix.
>
> > > indicates that the device
> > > +supports legacy queue notifications at this notification location.
> >
> > And what location? this refers to what? the location described by this capability
> > maybe?
> >
> Ack.
>
> >
> > More specifically, assume a transitional device (with an IO BAR as normal). Does
> > presence of this flag in the capability imply we can use a legacy queue but use
> > this notification with this capability?
> I don't know how this combination can be even possible by the device other than a buggy software who will do both.
>
> > I worry that if we allow that, it opens floodgates of people who are too lazy to
> > upgrade to 1.0 but just hack this notification in there.
> >
> Don't understand the concern.
> If a device has transitional device, it is 1.0 device supporting legacy with IOBAR and newer interface.
> So...
if we are switching to admin commands for this, then let's not argue
about it.
> >
> >
> >
> >
> > > Legacy Queue
> > > +Notification address is derived within a BAR for a virtqueue:
> > > +
> > > +\begin{lstlisting}
> > > + cap.offset
> > > +\end{lstlisting}
> > > +
> > > \devicenormative{\paragraph}{Notification capability}{Virtio
> > > Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification
> > capability} The device MUST present at least one notification capability.
> > >
> >
> > I don't get what this is saying. I *think* you really mean is that
> > queue_notify_off is 0.
> >
> Please refer to the current notification address definition.
> Above is written in similar way where it doesnt need to describe queue_notify_off.
> >
> > > @@ -596,6 +606,14 @@ \subsubsection{Notification structure
> > > layout}\label{sec:Virtio Transport Options cap.length >=
> > > queue_notify_off * notify_off_multiplier + 4 \end{lstlisting}
> > >
> > > +\paragraph{Legacy Interfaces: Notification
> > > +capability}\label{par:Virtio Transport Options / Virtio Over PCI Bus
> > > +/ PCI Device Layout / Legacy Interfaces: Notification capability}
> > > +
> > > +The device SHOULD set \field{legacy_q_notify_supported}
> >
> > why SHOULD and not MUST here - if driver checks for it then it won't work
> > without no?
> >
> If device doesn't want to support queue notifications, it has ability to set to 0.
>
> > > when the device
> > > +notification location supports legacy driver notifications.
> >
> > legacy driver notifications is ambigous.
> > what you want here and elsewhere is something like "when used through a
> > legacy interface, device supports driver
> >
> Ack.
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 20:04 ` Parav Pandit
@ 2023-06-21 20:08 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 20:08 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 08:04:22PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, June 21, 2023 3:43 PM
> >
> > On Wed, Jun 21, 2023 at 04:01:00PM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, June 21, 2023 11:56 AM
> > >
> > > > Not like v3 please. If you want to re-open this:
> > > >
> > > >
> > > > First I think we need multiple offsets just like notification
> > > > capaiblity, sorted by priority. Second I think we need ability to
> > > > report offset within owner not member.
> > > >
> > > > So please add ability to report multiple offsets, and add e.g.
> > > > a flags field, with bits for owner, member.
> > >
> > > What are these multiple offsets for?
> >
> > Different BARs. For example, IO versus memory. Yes I know VFs don't support
> > IO but PFs do. prefetch vs non-prefetch might matter too (non-prefetch is
> > mostly limited to 32 bit).
> >
> BAR type and its prefetch attributes are told by the PCI anyway so no duplication here anyway.
yes they are! But for example, software PF device might prefer notification in IO
BAR if that can be mapped. If not - use MMIO.
> For owner and member I get it.
> >
> > > Do you mean two offsets? One for owner and one for member?
> > > Both bits are optional. Do you agree?
> >
> > I agree that everything should be optional sure. Let's not limit it to two though.
> > There could be multiple BARs or whatever. We have ability to pass buffers of
> > arbitrary length and device reports the length.
> > Maybe have a SHOULD recommending not more than 10 of these (1 for each
> > possible PCI BAR one for each possible VF BAR).
> >
> > This is to match the functionality that notification capability has.
> >
> A device can expose multiple offsets within the single bar to a count of 100 entries;
> Also allowed in the notification capability.
> (I don't see why any sane device would do that anyway)
Me neither.
> So I will avoid such extra restrictions and limit the scope of this patch to be minimal to have arbitrary number of BAR entries limited by the command result length.
That sounds OK.
> >
> >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and to
> > > minimize spam in the list archive, subscription is required before
> > > posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License:
> > > https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines:
> > > https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 20:08 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 20:08 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 08:04:22PM +0000, Parav Pandit wrote:
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, June 21, 2023 3:43 PM
> >
> > On Wed, Jun 21, 2023 at 04:01:00PM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, June 21, 2023 11:56 AM
> > >
> > > > Not like v3 please. If you want to re-open this:
> > > >
> > > >
> > > > First I think we need multiple offsets just like notification
> > > > capaiblity, sorted by priority. Second I think we need ability to
> > > > report offset within owner not member.
> > > >
> > > > So please add ability to report multiple offsets, and add e.g.
> > > > a flags field, with bits for owner, member.
> > >
> > > What are these multiple offsets for?
> >
> > Different BARs. For example, IO versus memory. Yes I know VFs don't support
> > IO but PFs do. prefetch vs non-prefetch might matter too (non-prefetch is
> > mostly limited to 32 bit).
> >
> BAR type and its prefetch attributes are told by the PCI anyway so no duplication here anyway.
yes they are! But for example, software PF device might prefer notification in IO
BAR if that can be mapped. If not - use MMIO.
> For owner and member I get it.
> >
> > > Do you mean two offsets? One for owner and one for member?
> > > Both bits are optional. Do you agree?
> >
> > I agree that everything should be optional sure. Let's not limit it to two though.
> > There could be multiple BARs or whatever. We have ability to pass buffers of
> > arbitrary length and device reports the length.
> > Maybe have a SHOULD recommending not more than 10 of these (1 for each
> > possible PCI BAR one for each possible VF BAR).
> >
> > This is to match the functionality that notification capability has.
> >
> A device can expose multiple offsets within the single bar to a count of 100 entries;
> Also allowed in the notification capability.
> (I don't see why any sane device would do that anyway)
Me neither.
> So I will avoid such extra restrictions and limit the scope of this patch to be minimal to have arbitrary number of BAR entries limited by the command result length.
That sounds OK.
> >
> >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and to
> > > minimize spam in the list archive, subscription is required before
> > > posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License:
> > > https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines:
> > > https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 20:05 ` Michael S. Tsirkin
@ 2023-06-21 20:22 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 20:22 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 4:06 PM
[..]
> > > To put it in our terms, something like this:
> > > when a legacy driver accesses a member device of
> > > a group using the
> > > legacy interface, a modern driver can intercept
> > > the access and forward it to the owner device.
> > >
> > I will not mention "modern driver" as it has zero reference in spec and don't
> want to bring Linux terms here.
> > "the driver can intercept" is enough.
>
> Good point but since you say "a legacy driver" then "the driver" seems to refer
> to exactly that. How about:
>
> when a legacy member device driver accesses a member device of
> a group using the
> legacy interface, an owner device driver can intercept
> the access and forward it to the owner device.
>
Above is not correct.
We have 3 drivers in picture.
1. guest driver
2. hypervisor level member device driver
3. group owner device driver
Trying to write without introducing guest and hypervisor term,
A group owner device driver provides the service to access member device's configuration region.
A member device driver avail this service when it wants to access the member device's configuration region.
> > I will rewrite it as,
> >
> > The group owner device should not expose PCI BAR0 in the PCI SR-IOV
> > extended capability for the group member PCI VF devices when it prefers to
> support legacy interface for legacy configuration access using this group owner.
>
>
> This seems to ignore all my comments.
>
I replied after that, probably missed in exchanges.
How about:
The group owner device MUST hardwire PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it supports legacy configuration access commands.
>
>
> > > > for the group member
> > > > +devices when it prefers to support legacy interface for legacy
> > > > +configuration access using its group owner.
> > >
> > > don't use should outside conformance
> > >
> > What should be used instead of "should"?
>
> Depends on context, generally we just say what it does. E.g. "VF BAR0 is
> hardwired to zero".
>
Ok.
>
>
> > >
> > > I think this specific case actually should be more specific.
> > > Something like:
> > >
> > > - For PCI SRIOV groups, when group owner device implements commands
> > > A,B,C the group owner's VF BAR 0 is hardwired to 0
> > > in its PCI SRIOV capability.
> > >
> > I wrote it slightly differently above.
>
> I don't see why what you wrote is any better than what you had to be frank. You
> are coming up with our own terminology instead of just using what's in the SR
> IOV spec. Please don't.
>
> > >
> > > > +This facilitates hypervisor software to operate with least amount
> > > > +of complexities
> > >
> > > complexity is its own plural
> > >
> > > > to emulate the legacy interface I/O space BAR and passthrough
> > > > +other PCI BARs and PCI device capabilities to the guest virtual
> > > > +machine without any translation.
> > > > +
> > > > +The group member device should not expose PCI BAR0 in various PCI
> > > capabilities.
> > > > +
> > > > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}
> > > > +
> > > > +When a PCI SR-IOV group owner device supports
> > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> > > owner
> > > > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > > > +This is to facilitate the software to emulate I/O region BAR0 for
> > > > +supporting
> > > the legacy interface.
> > >
> > > not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose
> > > it". if you want to register can be "unimplemented".
> > > Base Address registers are hardwired to zero
> > >
> > > But it is better to be specific on what should happen. hardwire VF
> > > BAR0 to 0, right?
> > >
> > Hardwire to zero and not expose is same thing to me.
>
> Maybe, though previously you said it just means not put any capabilities there.
> More importantly I don't see expose or not expose anywhere in the PCI or
> SRIOV spec. When spec wants to say how not to have a given BAR it says exactly
> hardwired to zero.
>
Hardwired to zero is fine, done above now.
> > >
> > > > +
> > > > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}
> > > > +
> > > > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > > > +member exposes a PCI BAR0.
> > >
> > > what does "emulate" mean here? which driver it is?
> > >
> > I will add the line to theory of operation, but I recollect "emulate" line came
> from you.
>
> I generally am fine with using this term but we need to then define it before
> use.
>
> This specific thing I would just drop.
>
> Instead of wasting time just say device MUST hardwire VF BAR0 to zero as
> opposed to SHOULD, and we do not need to worry about how drivers recover. If
> they want to they will validate, if they don't then they won't.
>
This limitation seems fine to me.
Will drop.
>
> > > > diff --git a/transport-pci.tex b/transport-pci.tex index
> > > > a5c6719..3647485 100644
> > > > --- a/transport-pci.tex
> > > > +++ b/transport-pci.tex
> > > > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > > > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > > > struct virtio_pci_cap cap;
> > > > le32 notify_off_multiplier; /* Multiplier for
> > > > queue_notify_off. */
> > > > + u8 legacy_q_notify_supported;
> > >
> > > I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> > >
> > >
> > > > + u8 reserved[3];
> > >
> > >
> > > align with spaces pls.
> > >
> > Ack.
> >
> > >
> > > > };
> > > > \end{lstlisting}
> > > >
> > > > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > > > layout}\label{sec:Virtio Transport Options the same Queue Notify
> > > > address for
> > > all queues.
> > > > \end{note}
> > > >
> > > > +\field{legacy_q_notify_supported} when set to 1,
> > >
> > >
> > > Do you mean something like:
> > > When \field{legacy_q_notify_supported} this indicates that ...
> > >
> > Yes will fix.
> >
> > > > indicates that the device
> > > > +supports legacy queue notifications at this notification location.
> > >
> > > And what location? this refers to what? the location described by
> > > this capability maybe?
> > >
> > Ack.
> >
> > >
> > > More specifically, assume a transitional device (with an IO BAR as
> > > normal). Does presence of this flag in the capability imply we can
> > > use a legacy queue but use this notification with this capability?
> > I don't know how this combination can be even possible by the device other
> than a buggy software who will do both.
> >
> > > I worry that if we allow that, it opens floodgates of people who are
> > > too lazy to upgrade to 1.0 but just hack this notification in there.
> > >
> > Don't understand the concern.
> > If a device has transitional device, it is 1.0 device supporting legacy with
> IOBAR and newer interface.
> > So...
>
>
> if we are switching to admin commands for this, then let's not argue about it.
>
Ok.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 20:22 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 20:22 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 4:06 PM
[..]
> > > To put it in our terms, something like this:
> > > when a legacy driver accesses a member device of
> > > a group using the
> > > legacy interface, a modern driver can intercept
> > > the access and forward it to the owner device.
> > >
> > I will not mention "modern driver" as it has zero reference in spec and don't
> want to bring Linux terms here.
> > "the driver can intercept" is enough.
>
> Good point but since you say "a legacy driver" then "the driver" seems to refer
> to exactly that. How about:
>
> when a legacy member device driver accesses a member device of
> a group using the
> legacy interface, an owner device driver can intercept
> the access and forward it to the owner device.
>
Above is not correct.
We have 3 drivers in picture.
1. guest driver
2. hypervisor level member device driver
3. group owner device driver
Trying to write without introducing guest and hypervisor term,
A group owner device driver provides the service to access member device's configuration region.
A member device driver avail this service when it wants to access the member device's configuration region.
> > I will rewrite it as,
> >
> > The group owner device should not expose PCI BAR0 in the PCI SR-IOV
> > extended capability for the group member PCI VF devices when it prefers to
> support legacy interface for legacy configuration access using this group owner.
>
>
> This seems to ignore all my comments.
>
I replied after that, probably missed in exchanges.
How about:
The group owner device MUST hardwire PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it supports legacy configuration access commands.
>
>
> > > > for the group member
> > > > +devices when it prefers to support legacy interface for legacy
> > > > +configuration access using its group owner.
> > >
> > > don't use should outside conformance
> > >
> > What should be used instead of "should"?
>
> Depends on context, generally we just say what it does. E.g. "VF BAR0 is
> hardwired to zero".
>
Ok.
>
>
> > >
> > > I think this specific case actually should be more specific.
> > > Something like:
> > >
> > > - For PCI SRIOV groups, when group owner device implements commands
> > > A,B,C the group owner's VF BAR 0 is hardwired to 0
> > > in its PCI SRIOV capability.
> > >
> > I wrote it slightly differently above.
>
> I don't see why what you wrote is any better than what you had to be frank. You
> are coming up with our own terminology instead of just using what's in the SR
> IOV spec. Please don't.
>
> > >
> > > > +This facilitates hypervisor software to operate with least amount
> > > > +of complexities
> > >
> > > complexity is its own plural
> > >
> > > > to emulate the legacy interface I/O space BAR and passthrough
> > > > +other PCI BARs and PCI device capabilities to the guest virtual
> > > > +machine without any translation.
> > > > +
> > > > +The group member device should not expose PCI BAR0 in various PCI
> > > capabilities.
> > > > +
> > > > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}
> > > > +
> > > > +When a PCI SR-IOV group owner device supports
> > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> > > owner
> > > > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > > > +This is to facilitate the software to emulate I/O region BAR0 for
> > > > +supporting
> > > the legacy interface.
> > >
> > > not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose
> > > it". if you want to register can be "unimplemented".
> > > Base Address registers are hardwired to zero
> > >
> > > But it is better to be specific on what should happen. hardwire VF
> > > BAR0 to 0, right?
> > >
> > Hardwire to zero and not expose is same thing to me.
>
> Maybe, though previously you said it just means not put any capabilities there.
> More importantly I don't see expose or not expose anywhere in the PCI or
> SRIOV spec. When spec wants to say how not to have a given BAR it says exactly
> hardwired to zero.
>
Hardwired to zero is fine, done above now.
> > >
> > > > +
> > > > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > +Group Member Device Legacy Configuration Access}
> > > > +
> > > > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > > > +member exposes a PCI BAR0.
> > >
> > > what does "emulate" mean here? which driver it is?
> > >
> > I will add the line to theory of operation, but I recollect "emulate" line came
> from you.
>
> I generally am fine with using this term but we need to then define it before
> use.
>
> This specific thing I would just drop.
>
> Instead of wasting time just say device MUST hardwire VF BAR0 to zero as
> opposed to SHOULD, and we do not need to worry about how drivers recover. If
> they want to they will validate, if they don't then they won't.
>
This limitation seems fine to me.
Will drop.
>
> > > > diff --git a/transport-pci.tex b/transport-pci.tex index
> > > > a5c6719..3647485 100644
> > > > --- a/transport-pci.tex
> > > > +++ b/transport-pci.tex
> > > > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > > > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > > > struct virtio_pci_cap cap;
> > > > le32 notify_off_multiplier; /* Multiplier for
> > > > queue_notify_off. */
> > > > + u8 legacy_q_notify_supported;
> > >
> > > I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> > >
> > >
> > > > + u8 reserved[3];
> > >
> > >
> > > align with spaces pls.
> > >
> > Ack.
> >
> > >
> > > > };
> > > > \end{lstlisting}
> > > >
> > > > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > > > layout}\label{sec:Virtio Transport Options the same Queue Notify
> > > > address for
> > > all queues.
> > > > \end{note}
> > > >
> > > > +\field{legacy_q_notify_supported} when set to 1,
> > >
> > >
> > > Do you mean something like:
> > > When \field{legacy_q_notify_supported} this indicates that ...
> > >
> > Yes will fix.
> >
> > > > indicates that the device
> > > > +supports legacy queue notifications at this notification location.
> > >
> > > And what location? this refers to what? the location described by
> > > this capability maybe?
> > >
> > Ack.
> >
> > >
> > > More specifically, assume a transitional device (with an IO BAR as
> > > normal). Does presence of this flag in the capability imply we can
> > > use a legacy queue but use this notification with this capability?
> > I don't know how this combination can be even possible by the device other
> than a buggy software who will do both.
> >
> > > I worry that if we allow that, it opens floodgates of people who are
> > > too lazy to upgrade to 1.0 but just hack this notification in there.
> > >
> > Don't understand the concern.
> > If a device has transitional device, it is 1.0 device supporting legacy with
> IOBAR and newer interface.
> > So...
>
>
> if we are switching to admin commands for this, then let's not argue about it.
>
Ok.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 20:22 ` Parav Pandit
@ 2023-06-21 20:31 ` Michael S. Tsirkin
-1 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 20:31 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 08:22:58PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, June 21, 2023 4:06 PM
>
> [..]
> > > > To put it in our terms, something like this:
> > > > when a legacy driver accesses a member device of
> > > > a group using the
> > > > legacy interface, a modern driver can intercept
> > > > the access and forward it to the owner device.
> > > >
> > > I will not mention "modern driver" as it has zero reference in spec and don't
> > want to bring Linux terms here.
> > > "the driver can intercept" is enough.
Maybe a (non legacy) driver can intercept? Would that be acceptable?
Just to clarify the confusion above.
> >
> > Good point but since you say "a legacy driver" then "the driver" seems to refer
> > to exactly that. How about:
> >
> > when a legacy member device driver accesses a member device of
> > a group using the
> > legacy interface, an owner device driver can intercept
> > the access and forward it to the owner device.
> >
> Above is not correct.
>
> We have 3 drivers in picture.
> 1. guest driver
this is legacy driver, so easy
> 2. hypervisor level member device driver
this is just for notifications, optionally, yes?
> 3. group owner device driver
>
> Trying to write without introducing guest and hypervisor term,
>
> A group owner device driver provides the service to access member device's configuration region.
> A member device driver avail this service when it wants to access the member device's configuration region.
I agree, it's getting complicated.
>
> > > I will rewrite it as,
> > >
> > > The group owner device should not expose PCI BAR0 in the PCI SR-IOV
> > > extended capability for the group member PCI VF devices when it prefers to
> > support legacy interface for legacy configuration access using this group owner.
> >
> >
> > This seems to ignore all my comments.
> >
> I replied after that, probably missed in exchanges.
>
> How about:
> The group owner device MUST hardwire PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it supports legacy configuration access commands.
>
better but it's not a PCI BAR0. let's add link to sriov spec,
and name it VF BAR0 same as in that spec.
> >
> >
> > > > > for the group member
> > > > > +devices when it prefers to support legacy interface for legacy
> > > > > +configuration access using its group owner.
> > > >
> > > > don't use should outside conformance
> > > >
> > > What should be used instead of "should"?
> >
> > Depends on context, generally we just say what it does. E.g. "VF BAR0 is
> > hardwired to zero".
> >
> Ok.
>
> >
> >
> > > >
> > > > I think this specific case actually should be more specific.
> > > > Something like:
> > > >
> > > > - For PCI SRIOV groups, when group owner device implements commands
> > > > A,B,C the group owner's VF BAR 0 is hardwired to 0
> > > > in its PCI SRIOV capability.
> > > >
> > > I wrote it slightly differently above.
> >
> > I don't see why what you wrote is any better than what you had to be frank. You
> > are coming up with our own terminology instead of just using what's in the SR
> > IOV spec. Please don't.
> >
> > > >
> > > > > +This facilitates hypervisor software to operate with least amount
> > > > > +of complexities
> > > >
> > > > complexity is its own plural
> > > >
> > > > > to emulate the legacy interface I/O space BAR and passthrough
> > > > > +other PCI BARs and PCI device capabilities to the guest virtual
> > > > > +machine without any translation.
> > > > > +
> > > > > +The group member device should not expose PCI BAR0 in various PCI
> > > > capabilities.
> > > > > +
> > > > > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}
> > > > > +
> > > > > +When a PCI SR-IOV group owner device supports
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> > > > owner
> > > > > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > > > > +This is to facilitate the software to emulate I/O region BAR0 for
> > > > > +supporting
> > > > the legacy interface.
> > > >
> > > > not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose
> > > > it". if you want to register can be "unimplemented".
> > > > Base Address registers are hardwired to zero
> > > >
> > > > But it is better to be specific on what should happen. hardwire VF
> > > > BAR0 to 0, right?
> > > >
> > > Hardwire to zero and not expose is same thing to me.
> >
> > Maybe, though previously you said it just means not put any capabilities there.
> > More importantly I don't see expose or not expose anywhere in the PCI or
> > SRIOV spec. When spec wants to say how not to have a given BAR it says exactly
> > hardwired to zero.
> >
> Hardwired to zero is fine, done above now.
>
> > > >
> > > > > +
> > > > > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}
> > > > > +
> > > > > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > > > > +member exposes a PCI BAR0.
> > > >
> > > > what does "emulate" mean here? which driver it is?
> > > >
> > > I will add the line to theory of operation, but I recollect "emulate" line came
> > from you.
> >
> > I generally am fine with using this term but we need to then define it before
> > use.
> >
> > This specific thing I would just drop.
> >
> > Instead of wasting time just say device MUST hardwire VF BAR0 to zero as
> > opposed to SHOULD, and we do not need to worry about how drivers recover. If
> > they want to they will validate, if they don't then they won't.
> >
> This limitation seems fine to me.
> Will drop.
>
> >
> > > > > diff --git a/transport-pci.tex b/transport-pci.tex index
> > > > > a5c6719..3647485 100644
> > > > > --- a/transport-pci.tex
> > > > > +++ b/transport-pci.tex
> > > > > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > > > > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > > > > struct virtio_pci_cap cap;
> > > > > le32 notify_off_multiplier; /* Multiplier for
> > > > > queue_notify_off. */
> > > > > + u8 legacy_q_notify_supported;
> > > >
> > > > I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> > > >
> > > >
> > > > > + u8 reserved[3];
> > > >
> > > >
> > > > align with spaces pls.
> > > >
> > > Ack.
> > >
> > > >
> > > > > };
> > > > > \end{lstlisting}
> > > > >
> > > > > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > > > > layout}\label{sec:Virtio Transport Options the same Queue Notify
> > > > > address for
> > > > all queues.
> > > > > \end{note}
> > > > >
> > > > > +\field{legacy_q_notify_supported} when set to 1,
> > > >
> > > >
> > > > Do you mean something like:
> > > > When \field{legacy_q_notify_supported} this indicates that ...
> > > >
> > > Yes will fix.
> > >
> > > > > indicates that the device
> > > > > +supports legacy queue notifications at this notification location.
> > > >
> > > > And what location? this refers to what? the location described by
> > > > this capability maybe?
> > > >
> > > Ack.
> > >
> > > >
> > > > More specifically, assume a transitional device (with an IO BAR as
> > > > normal). Does presence of this flag in the capability imply we can
> > > > use a legacy queue but use this notification with this capability?
> > > I don't know how this combination can be even possible by the device other
> > than a buggy software who will do both.
> > >
> > > > I worry that if we allow that, it opens floodgates of people who are
> > > > too lazy to upgrade to 1.0 but just hack this notification in there.
> > > >
> > > Don't understand the concern.
> > > If a device has transitional device, it is 1.0 device supporting legacy with
> > IOBAR and newer interface.
> > > So...
> >
> >
> > if we are switching to admin commands for this, then let's not argue about it.
> >
> Ok.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 20:31 ` Michael S. Tsirkin
0 siblings, 0 replies; 100+ messages in thread
From: Michael S. Tsirkin @ 2023-06-21 20:31 UTC (permalink / raw)
To: Parav Pandit
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
On Wed, Jun 21, 2023 at 08:22:58PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, June 21, 2023 4:06 PM
>
> [..]
> > > > To put it in our terms, something like this:
> > > > when a legacy driver accesses a member device of
> > > > a group using the
> > > > legacy interface, a modern driver can intercept
> > > > the access and forward it to the owner device.
> > > >
> > > I will not mention "modern driver" as it has zero reference in spec and don't
> > want to bring Linux terms here.
> > > "the driver can intercept" is enough.
Maybe a (non legacy) driver can intercept? Would that be acceptable?
Just to clarify the confusion above.
> >
> > Good point but since you say "a legacy driver" then "the driver" seems to refer
> > to exactly that. How about:
> >
> > when a legacy member device driver accesses a member device of
> > a group using the
> > legacy interface, an owner device driver can intercept
> > the access and forward it to the owner device.
> >
> Above is not correct.
>
> We have 3 drivers in picture.
> 1. guest driver
this is legacy driver, so easy
> 2. hypervisor level member device driver
this is just for notifications, optionally, yes?
> 3. group owner device driver
>
> Trying to write without introducing guest and hypervisor term,
>
> A group owner device driver provides the service to access member device's configuration region.
> A member device driver avail this service when it wants to access the member device's configuration region.
I agree, it's getting complicated.
>
> > > I will rewrite it as,
> > >
> > > The group owner device should not expose PCI BAR0 in the PCI SR-IOV
> > > extended capability for the group member PCI VF devices when it prefers to
> > support legacy interface for legacy configuration access using this group owner.
> >
> >
> > This seems to ignore all my comments.
> >
> I replied after that, probably missed in exchanges.
>
> How about:
> The group owner device MUST hardwire PCI BAR0 in the PCI SR-IOV extended capability for the group member PCI VF devices when it supports legacy configuration access commands.
>
better but it's not a PCI BAR0. let's add link to sriov spec,
and name it VF BAR0 same as in that spec.
> >
> >
> > > > > for the group member
> > > > > +devices when it prefers to support legacy interface for legacy
> > > > > +configuration access using its group owner.
> > > >
> > > > don't use should outside conformance
> > > >
> > > What should be used instead of "should"?
> >
> > Depends on context, generally we just say what it does. E.g. "VF BAR0 is
> > hardwired to zero".
> >
> Ok.
>
> >
> >
> > > >
> > > > I think this specific case actually should be more specific.
> > > > Something like:
> > > >
> > > > - For PCI SRIOV groups, when group owner device implements commands
> > > > A,B,C the group owner's VF BAR 0 is hardwired to 0
> > > > in its PCI SRIOV capability.
> > > >
> > > I wrote it slightly differently above.
> >
> > I don't see why what you wrote is any better than what you had to be frank. You
> > are coming up with our own terminology instead of just using what's in the SR
> > IOV spec. Please don't.
> >
> > > >
> > > > > +This facilitates hypervisor software to operate with least amount
> > > > > +of complexities
> > > >
> > > > complexity is its own plural
> > > >
> > > > > to emulate the legacy interface I/O space BAR and passthrough
> > > > > +other PCI BARs and PCI device capabilities to the guest virtual
> > > > > +machine without any translation.
> > > > > +
> > > > > +The group member device should not expose PCI BAR0 in various PCI
> > > > capabilities.
> > > > > +
> > > > > +\devicenormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}
> > > > > +
> > > > > +When a PCI SR-IOV group owner device supports
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ,
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE,
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ,
> > > > > +VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE commands, the group
> > > > owner
> > > > > +device SHOULD NOT expose PCI BAR0 in the SR-IOV Extended capability.
> > > > > +This is to facilitate the software to emulate I/O region BAR0 for
> > > > > +supporting
> > > > the legacy interface.
> > > >
> > > > not PCI BAR0 - VF BAR0. Check the PCI spec you can not "not expose
> > > > it". if you want to register can be "unimplemented".
> > > > Base Address registers are hardwired to zero
> > > >
> > > > But it is better to be specific on what should happen. hardwire VF
> > > > BAR0 to 0, right?
> > > >
> > > Hardwire to zero and not expose is same thing to me.
> >
> > Maybe, though previously you said it just means not put any capabilities there.
> > More importantly I don't see expose or not expose anywhere in the PCI or
> > SRIOV spec. When spec wants to say how not to have a given BAR it says exactly
> > hardwired to zero.
> >
> Hardwired to zero is fine, done above now.
>
> > > >
> > > > > +
> > > > > +\drivernormative{\subsubsection}{Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}{Virtio Transport
> > > > > +Options / Virtio Over PCI Bus / Legacy Interfaces Requirements:
> > > > > +Group Member Device Legacy Configuration Access}
> > > > > +
> > > > > +The driver SHOULD NOT emulate I/O region BAR0 if a device group
> > > > > +member exposes a PCI BAR0.
> > > >
> > > > what does "emulate" mean here? which driver it is?
> > > >
> > > I will add the line to theory of operation, but I recollect "emulate" line came
> > from you.
> >
> > I generally am fine with using this term but we need to then define it before
> > use.
> >
> > This specific thing I would just drop.
> >
> > Instead of wasting time just say device MUST hardwire VF BAR0 to zero as
> > opposed to SHOULD, and we do not need to worry about how drivers recover. If
> > they want to they will validate, if they don't then they won't.
> >
> This limitation seems fine to me.
> Will drop.
>
> >
> > > > > diff --git a/transport-pci.tex b/transport-pci.tex index
> > > > > a5c6719..3647485 100644
> > > > > --- a/transport-pci.tex
> > > > > +++ b/transport-pci.tex
> > > > > @@ -541,6 +541,8 @@ \subsubsection{Notification structure
> > > > > layout}\label{sec:Virtio Transport Options struct virtio_pci_notify_cap {
> > > > > struct virtio_pci_cap cap;
> > > > > le32 notify_off_multiplier; /* Multiplier for
> > > > > queue_notify_off. */
> > > > > + u8 legacy_q_notify_supported;
> > > >
> > > > I am still mulling whether VIRTIO_PCI_CAP_NOTIFY_CFG would be better.
> > > >
> > > >
> > > > > + u8 reserved[3];
> > > >
> > > >
> > > > align with spaces pls.
> > > >
> > > Ack.
> > >
> > > >
> > > > > };
> > > > > \end{lstlisting}
> > > > >
> > > > > @@ -560,6 +562,14 @@ \subsubsection{Notification structure
> > > > > layout}\label{sec:Virtio Transport Options the same Queue Notify
> > > > > address for
> > > > all queues.
> > > > > \end{note}
> > > > >
> > > > > +\field{legacy_q_notify_supported} when set to 1,
> > > >
> > > >
> > > > Do you mean something like:
> > > > When \field{legacy_q_notify_supported} this indicates that ...
> > > >
> > > Yes will fix.
> > >
> > > > > indicates that the device
> > > > > +supports legacy queue notifications at this notification location.
> > > >
> > > > And what location? this refers to what? the location described by
> > > > this capability maybe?
> > > >
> > > Ack.
> > >
> > > >
> > > > More specifically, assume a transitional device (with an IO BAR as
> > > > normal). Does presence of this flag in the capability imply we can
> > > > use a legacy queue but use this notification with this capability?
> > > I don't know how this combination can be even possible by the device other
> > than a buggy software who will do both.
> > >
> > > > I worry that if we allow that, it opens floodgates of people who are
> > > > too lazy to upgrade to 1.0 but just hack this notification in there.
> > > >
> > > Don't understand the concern.
> > > If a device has transitional device, it is 1.0 device supporting legacy with
> > IOBAR and newer interface.
> > > So...
> >
> >
> > if we are switching to admin commands for this, then let's not argue about it.
> >
> Ok.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
* [virtio-dev] RE: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
2023-06-21 20:31 ` Michael S. Tsirkin
@ 2023-06-21 20:43 ` Parav Pandit
-1 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 20:43 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 4:31 PM
> To: Parav Pandit <parav@nvidia.com>
> Cc: virtio-comment@lists.oasis-open.org; cohuck@redhat.com;
> david.edmondson@oracle.com; virtio-dev@lists.oasis-open.org;
> sburla@marvell.com; jasowang@redhat.com; Yishai Hadas
> <yishaih@nvidia.com>; Maor Gottlieb <maorg@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>
> Subject: Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group
> legacy group member config region access
>
> On Wed, Jun 21, 2023 at 08:22:58PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, June 21, 2023 4:06 PM
> >
> > [..]
> > > > > To put it in our terms, something like this:
> > > > > when a legacy driver accesses a member device of
> > > > > a group using the
> > > > > legacy interface, a modern driver can intercept
> > > > > the access and forward it to the owner device.
> > > > >
> > > > I will not mention "modern driver" as it has zero reference in
> > > > spec and don't
> > > want to bring Linux terms here.
> > > > "the driver can intercept" is enough.
>
> Maybe a (non legacy) driver can intercept? Would that be acceptable?
> Just to clarify the confusion above.
>
Non legacy driver wording is fine.
> > > when a legacy member device driver accesses a member device of
> > > a group using the
> > > legacy interface, an owner device driver can intercept
> > > the access and forward it to the owner device.
> > >
> > Above is not correct.
> >
> > We have 3 drivers in picture.
> > 1. guest driver
>
> this is legacy driver, so easy
>
> > 2. hypervisor level member device driver
>
> this is just for notifications, optionally, yes?
>
For notifications (optionally) and for configuration region access.
>
> > 3. group owner device driver
> >
> > Trying to write without introducing guest and hypervisor term,
> >
> > A group owner device driver provides the service to access member device's
> configuration region.
> > A member device driver avail this service when it wants to access the member
> device's configuration region.
>
>
> I agree, it's getting complicated.
>
> >
> > > > I will rewrite it as,
> > > >
> > > > The group owner device should not expose PCI BAR0 in the PCI
> > > > SR-IOV extended capability for the group member PCI VF devices
> > > > when it prefers to
> > > support legacy interface for legacy configuration access using this group
> owner.
> > >
> > >
> > > This seems to ignore all my comments.
> > >
> > I replied after that, probably missed in exchanges.
> >
> > How about:
> > The group owner device MUST hardwire PCI BAR0 in the PCI SR-IOV extended
> capability for the group member PCI VF devices when it supports legacy
> configuration access commands.
> >
>
> better but it's not a PCI BAR0. let's add link to sriov spec, and name it VF BAR0
> same as in that spec.
>
Yes, VF BAR0, will correct it.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 100+ messages in thread
* RE: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access
@ 2023-06-21 20:43 ` Parav Pandit
0 siblings, 0 replies; 100+ messages in thread
From: Parav Pandit @ 2023-06-21 20:43 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-comment, cohuck, david.edmondson, virtio-dev, sburla,
jasowang, Yishai Hadas, Maor Gottlieb, Shahaf Shuler
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, June 21, 2023 4:31 PM
> To: Parav Pandit <parav@nvidia.com>
> Cc: virtio-comment@lists.oasis-open.org; cohuck@redhat.com;
> david.edmondson@oracle.com; virtio-dev@lists.oasis-open.org;
> sburla@marvell.com; jasowang@redhat.com; Yishai Hadas
> <yishaih@nvidia.com>; Maor Gottlieb <maorg@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>
> Subject: Re: [virtio-comment] RE: [PATCH v6 4/4] transport-pci: Introduce group
> legacy group member config region access
>
> On Wed, Jun 21, 2023 at 08:22:58PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, June 21, 2023 4:06 PM
> >
> > [..]
> > > > > To put it in our terms, something like this:
> > > > > when a legacy driver accesses a member device of
> > > > > a group using the
> > > > > legacy interface, a modern driver can intercept
> > > > > the access and forward it to the owner device.
> > > > >
> > > > I will not mention "modern driver" as it has zero reference in
> > > > spec and don't
> > > want to bring Linux terms here.
> > > > "the driver can intercept" is enough.
>
> Maybe a (non legacy) driver can intercept? Would that be acceptable?
> Just to clarify the confusion above.
>
Non legacy driver wording is fine.
> > > when a legacy member device driver accesses a member device of
> > > a group using the
> > > legacy interface, an owner device driver can intercept
> > > the access and forward it to the owner device.
> > >
> > Above is not correct.
> >
> > We have 3 drivers in picture.
> > 1. guest driver
>
> this is legacy driver, so easy
>
> > 2. hypervisor level member device driver
>
> this is just for notifications, optionally, yes?
>
For notifications (optionally) and for configuration region access.
>
> > 3. group owner device driver
> >
> > Trying to write without introducing guest and hypervisor term,
> >
> > A group owner device driver provides the service to access member device's
> configuration region.
> > A member device driver avail this service when it wants to access the member
> device's configuration region.
>
>
> I agree, it's getting complicated.
>
> >
> > > > I will rewrite it as,
> > > >
> > > > The group owner device should not expose PCI BAR0 in the PCI
> > > > SR-IOV extended capability for the group member PCI VF devices
> > > > when it prefers to
> > > support legacy interface for legacy configuration access using this group
> owner.
> > >
> > >
> > > This seems to ignore all my comments.
> > >
> > I replied after that, probably missed in exchanges.
> >
> > How about:
> > The group owner device MUST hardwire PCI BAR0 in the PCI SR-IOV extended
> capability for the group member PCI VF devices when it supports legacy
> configuration access commands.
> >
>
> better but it's not a PCI BAR0. let's add link to sriov spec, and name it VF BAR0
> same as in that spec.
>
Yes, VF BAR0, will correct it.
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
^ permalink raw reply [flat|nested] 100+ messages in thread
end of thread, other threads:[~2023-06-21 20:43 UTC | newest]
Thread overview: 100+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-06-13 17:30 [virtio-dev] [PATCH v6 0/4] admin: Introduce legacy registers access using AQ Parav Pandit
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
2023-06-13 17:30 ` [virtio-dev] [PATCH v6 1/4] admin: Split opcode table rows with a line Parav Pandit
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
2023-06-13 17:30 ` [virtio-dev] [PATCH v6 2/4] admin: Fix section numbering Parav Pandit
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
2023-06-13 17:30 ` [virtio-dev] [PATCH v6 3/4] admin: Add group member legacy register access commands Parav Pandit
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
2023-06-19 16:20 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 16:20 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 16:29 ` [virtio-dev] " Parav Pandit
2023-06-19 16:29 ` [virtio-comment] " Parav Pandit
2023-06-19 16:40 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 16:40 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 16:45 ` [virtio-dev] " Parav Pandit
2023-06-19 16:45 ` [virtio-comment] " Parav Pandit
2023-06-19 17:10 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 17:10 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 17:21 ` [virtio-comment] RE: [virtio-dev] " Parav Pandit
2023-06-19 17:21 ` Parav Pandit
2023-06-19 17:33 ` Michael S. Tsirkin
2023-06-19 17:33 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 17:38 ` Parav Pandit
2023-06-19 17:38 ` [virtio-comment] " Parav Pandit
2023-06-13 17:30 ` [virtio-dev] [PATCH v6 4/4] transport-pci: Introduce group legacy group member config region access Parav Pandit
2023-06-13 17:30 ` [virtio-comment] " Parav Pandit
2023-06-19 16:16 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 16:16 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 21:07 ` [virtio-dev] " Parav Pandit
2023-06-19 21:07 ` [virtio-comment] " Parav Pandit
2023-06-21 20:05 ` [virtio-dev] " Michael S. Tsirkin
2023-06-21 20:05 ` Michael S. Tsirkin
2023-06-21 20:22 ` [virtio-dev] " Parav Pandit
2023-06-21 20:22 ` Parav Pandit
2023-06-21 20:31 ` [virtio-dev] " Michael S. Tsirkin
2023-06-21 20:31 ` Michael S. Tsirkin
2023-06-21 20:43 ` [virtio-dev] " Parav Pandit
2023-06-21 20:43 ` Parav Pandit
2023-06-19 16:37 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 16:37 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 16:39 ` [virtio-dev] " Parav Pandit
2023-06-19 16:39 ` [virtio-comment] " Parav Pandit
2023-06-19 17:19 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 17:19 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 17:26 ` [virtio-dev] " Parav Pandit
2023-06-19 17:26 ` [virtio-comment] " Parav Pandit
2023-06-19 17:37 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 17:37 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 17:45 ` [virtio-dev] " Parav Pandit
2023-06-19 17:45 ` [virtio-comment] " Parav Pandit
2023-06-19 17:57 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 17:57 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 18:07 ` [virtio-dev] " Parav Pandit
2023-06-19 18:07 ` [virtio-comment] " Parav Pandit
2023-06-20 14:12 ` [virtio-dev] " Parav Pandit
2023-06-20 14:12 ` [virtio-comment] " Parav Pandit
2023-06-21 15:50 ` [virtio-dev] " Parav Pandit
2023-06-21 15:50 ` [virtio-comment] " Parav Pandit
2023-06-21 15:56 ` [virtio-dev] " Michael S. Tsirkin
2023-06-21 15:56 ` [virtio-comment] " Michael S. Tsirkin
2023-06-21 16:01 ` [virtio-dev] " Parav Pandit
2023-06-21 16:01 ` [virtio-comment] " Parav Pandit
2023-06-21 19:43 ` [virtio-dev] " Michael S. Tsirkin
2023-06-21 19:43 ` Michael S. Tsirkin
2023-06-21 20:04 ` [virtio-dev] " Parav Pandit
2023-06-21 20:04 ` Parav Pandit
2023-06-21 20:08 ` [virtio-dev] " Michael S. Tsirkin
2023-06-21 20:08 ` Michael S. Tsirkin
2023-06-19 18:00 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 18:00 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 18:12 ` [virtio-dev] " Parav Pandit
2023-06-19 18:12 ` [virtio-comment] " Parav Pandit
2023-06-21 19:47 ` [virtio-dev] " Michael S. Tsirkin
2023-06-21 19:47 ` Michael S. Tsirkin
2023-06-19 17:04 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 17:04 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 17:11 ` [virtio-dev] " Parav Pandit
2023-06-19 17:11 ` [virtio-comment] " Parav Pandit
2023-06-19 17:26 ` Michael S. Tsirkin
2023-06-19 17:26 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 17:35 ` Parav Pandit
2023-06-19 17:35 ` [virtio-comment] " Parav Pandit
2023-06-19 17:46 ` Michael S. Tsirkin
2023-06-19 17:46 ` [virtio-comment] " Michael S. Tsirkin
2023-06-20 0:14 ` Parav Pandit
2023-06-20 0:14 ` [virtio-comment] " Parav Pandit
2023-06-20 10:21 ` Michael S. Tsirkin
2023-06-20 10:21 ` [virtio-comment] " Michael S. Tsirkin
2023-06-21 1:09 ` Parav Pandit
2023-06-21 1:09 ` [virtio-comment] " Parav Pandit
2023-06-21 5:05 ` Michael S. Tsirkin
2023-06-21 5:05 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 12:38 ` [virtio-dev] RE: [PATCH v6 0/4] admin: Introduce legacy registers access using AQ Parav Pandit
2023-06-19 12:38 ` [virtio-comment] " Parav Pandit
2023-06-19 15:18 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 15:18 ` [virtio-comment] " Michael S. Tsirkin
2023-06-19 15:58 ` [virtio-dev] " Parav Pandit
2023-06-19 15:58 ` [virtio-comment] " Parav Pandit
2023-06-19 16:28 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 16:28 ` [virtio-comment] " Michael S. Tsirkin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.