* RFC: Implementation of QMP documentation retrieval command @ 2021-06-16 15:48 Niteesh G. S. 2021-06-21 14:58 ` Philippe Mathieu-Daudé 0 siblings, 1 reply; 8+ messages in thread From: Niteesh G. S. @ 2021-06-16 15:48 UTC (permalink / raw) To: John Snow, armbru, Stefan Hajnoczi, kwolf, ehabkost, wainersm, qemu-devel, vsementsov [-- Attachment #1: Type: text/plain, Size: 1087 bytes --] Hello, We now have a reasonably working prototype that is capable of sending/receiving commands/responses, syntax highlighting, and a simple notification system. The prototype can be found here https://gitlab.com/niteesh.gs/qemu/-/tree/aqmp-tui-prototype Working on this prototype gives us a lot of hints on things to worry about and other ideas that were worth implementing. Our next goal is to start working on the real TUI based on this prototype. One requested feature of the TUI was to show documentation for the commands typed. To achieve this, a QMP command that lets us query the documentation from QEMU has to be implemented and some discussion has already been done on it in a previous thread. I request all continue that discussion here and suggest ideas regarding implementation. The goal is to come up with something small and simple which can also be improved upon after the summer. I have a simple implementation of the command that returns dummy documentation here https://gitlab.com/niteesh.gs/qemu/-/commit/796a41fb2840b9f3484c6fd5672e6fceb73acaef Thanks, Niteesh. [-- Attachment #2: Type: text/html, Size: 2477 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-16 15:48 RFC: Implementation of QMP documentation retrieval command Niteesh G. S. @ 2021-06-21 14:58 ` Philippe Mathieu-Daudé 2021-06-21 18:26 ` Niteesh G. S. 0 siblings, 1 reply; 8+ messages in thread From: Philippe Mathieu-Daudé @ 2021-06-21 14:58 UTC (permalink / raw) To: Niteesh G. S., John Snow Cc: kwolf, vsementsov, ehabkost, qemu-devel, wainersm, armbru, Stefan Hajnoczi Hi Niteesh, On 6/16/21 5:48 PM, Niteesh G. S. wrote: > Hello, > > We now have a reasonably working prototype that is capable of > sending/receiving > commands/responses, syntax highlighting, and a simple notification > system. The > prototype can be found here > https://gitlab.com/niteesh.gs/qemu/-/tree/aqmp-tui-prototype > <https://gitlab.com/niteesh.gs/qemu/-/tree/aqmp-tui-prototype> > Working on this prototype gives us a lot of hints on things to worry > about and other > ideas that were worth implementing. Our next goal is to start working on > the real TUI > based on this prototype. > > One requested feature of the TUI was to show documentation for the > commands typed. > To achieve this, a QMP command that lets us query the documentation from > QEMU has > to be implemented and some discussion has already been done on it in a > previous thread. > I request all continue that discussion here and suggest ideas regarding > implementation. > The goal is to come up with something small and simple which can also be > improved upon > after the summer. > > I have a simple implementation of the command that returns dummy > documentation here > https://gitlab.com/niteesh.gs/qemu/-/commit/796a41fb2840b9f3484c6fd5672e6fceb73acaef > <https://gitlab.com/niteesh.gs/qemu/-/commit/796a41fb2840b9f3484c6fd5672e6fceb73acaef> I noticed your mail got no comment, and realized you asked us to review your patches apart from the mailing list. I am not sure if this is an experiment or a misunderstanding, but so far it is unlikely you get review external to the mailing list, because it is not very practical to reviewers. Maybe some reviewers are willing to look at your tree, but if you are looking for a wider audience, I recommend you to follow this process (which you already used!): https://wiki.qemu.org/Contribute/SubmitAPatch#Submitting_your_Patches Regards, Phil. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-21 14:58 ` Philippe Mathieu-Daudé @ 2021-06-21 18:26 ` Niteesh G. S. 2021-06-22 9:35 ` Stefan Hajnoczi 0 siblings, 1 reply; 8+ messages in thread From: Niteesh G. S. @ 2021-06-21 18:26 UTC (permalink / raw) To: Philippe Mathieu-Daudé Cc: kwolf, vsementsov, ehabkost, armbru, wainersm, qemu-devel, Stefan Hajnoczi, John Snow [-- Attachment #1: Type: text/plain, Size: 3208 bytes --] Hello Philippe, On Mon, Jun 21, 2021 at 8:28 PM Philippe Mathieu-Daudé <philmd@redhat.com> wrote: > Hi Niteesh, > > On 6/16/21 5:48 PM, Niteesh G. S. wrote: > > Hello, > > > > We now have a reasonably working prototype that is capable of > > sending/receiving > > commands/responses, syntax highlighting, and a simple notification > > system. The > > prototype can be found here > > https://gitlab.com/niteesh.gs/qemu/-/tree/aqmp-tui-prototype > > <https://gitlab.com/niteesh.gs/qemu/-/tree/aqmp-tui-prototype> > > Working on this prototype gives us a lot of hints on things to worry > > about and other > > ideas that were worth implementing. Our next goal is to start working on > > the real TUI > > based on this prototype. > > > > One requested feature of the TUI was to show documentation for the > > commands typed. > > To achieve this, a QMP command that lets us query the documentation from > > QEMU has > > to be implemented and some discussion has already been done on it in a > > previous thread. > > I request all continue that discussion here and suggest ideas regarding > > implementation. > > The goal is to come up with something small and simple which can also be > > improved upon > > after the summer. > > > > I have a simple implementation of the command that returns dummy > > documentation here > > > https://gitlab.com/niteesh.gs/qemu/-/commit/796a41fb2840b9f3484c6fd5672e6fceb73acaef > > < > https://gitlab.com/niteesh.gs/qemu/-/commit/796a41fb2840b9f3484c6fd5672e6fceb73acaef > > > > I noticed your mail got no comment, and realized you asked us to > review your patches apart from the mailing list. I am not sure > if this is an experiment or a misunderstanding, but so far it is > unlikely you get review external to the mailing list, because it > is not very practical to reviewers. > Sorry for not conveying the main intent of this mail properly. The main goal of this mail was to start a discussion regarding the implementation of the QMP documentation retrieval command for AQMP TUI. There has already been a small discussion regarding it in my introduction mail but it wasn't anything in detail and it stagnated. So I created this mail to have a separate thread to collect all points regarding implementation and again start the discussion. We now have a prototype with the most basic features implemented and working. But there is a lot of clean-up that has to be done before I can send the prototype to the mailing list for review. And since the prototype is almost done John asked me to start a discussion regarding the QMP command so that I can work on it as a side project along side the TUI. TLDR: The goal of this mail wasn't to review the dummy command I had posted but rather start a discussion regarding the implementation of the QMP documentation retrieval command for the TUI. Thanks, Niteesh. > Maybe some reviewers are willing to look at your tree, but if you > are looking for a wider audience, I recommend you to follow this > process (which you already used!): > https://wiki.qemu.org/Contribute/SubmitAPatch#Submitting_your_Patches > > Regards, > > Phil. > > [-- Attachment #2: Type: text/html, Size: 5592 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-21 18:26 ` Niteesh G. S. @ 2021-06-22 9:35 ` Stefan Hajnoczi 2021-06-22 10:26 ` Daniel P. Berrangé 2021-06-22 18:57 ` Niteesh G. S. 0 siblings, 2 replies; 8+ messages in thread From: Stefan Hajnoczi @ 2021-06-22 9:35 UTC (permalink / raw) To: Niteesh G. S. Cc: kwolf, vsementsov, ehabkost, Philippe Mathieu-Daudé, armbru, wainersm, qemu-devel, John Snow [-- Attachment #1: Type: text/plain, Size: 1940 bytes --] On Mon, Jun 21, 2021 at 11:56:30PM +0530, Niteesh G. S. wrote: > TLDR: The goal of this mail wasn't to review the dummy command I had posted > but > rather start a discussion regarding the implementation of the QMP > documentation > retrieval command for the TUI. It's not clear to me what exactly you wanted to discuss. Here is the QMP schema from the commit you linked. It's something concrete that we can discuss: ## # @CommandDocumentation: # # A object representing documentation for a command. # # @name: Command name # # @doc: A string containing the documentation. Is @doc in some kind of markup or plain text? # ## { 'struct': 'CommandDocumentation', 'data': {'name': 'str', 'doc': 'str'} } ## # @query-cmd-doc: # # (A simple *prototype* implementation) # Command query-cmd-doc will return the documentation for the command # specified. This will help QMP clients currently the AQMP TUI to show # documentation related to a specific command. # # @command-name: The command name to query documentation for # # Returns: A @CommandDocumentation object containing the documentation. # # Since: TODO: Add a number ## { 'command': 'query-cmd-doc', 'data': { 'command-name': 'str' }, 'returns': 'CommandDocumentation' } Is there a way to retrieve struct/enum/etc documentation? Do you see a need to query multiple items of documentation in a single command? A single item query command can become a performance bottleneck if the clients wants to query the documentation for all commands, for example. This can be solved by making the the return value an array and allowing multiple commands to be queried at once. Do you see a need for wildcard queries where the client does not have the full command name? I guess the client has enough auto-completion information to search all commands on the client side, so maybe this functionality isn't necessary here? Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-22 9:35 ` Stefan Hajnoczi @ 2021-06-22 10:26 ` Daniel P. Berrangé 2021-06-22 11:04 ` Vladimir Sementsov-Ogievskiy 2021-06-22 18:57 ` Niteesh G. S. 1 sibling, 1 reply; 8+ messages in thread From: Daniel P. Berrangé @ 2021-06-22 10:26 UTC (permalink / raw) To: Stefan Hajnoczi Cc: kwolf, vsementsov, ehabkost, John Snow, armbru, wainersm, qemu-devel, Niteesh G. S., Philippe Mathieu-Daudé On Tue, Jun 22, 2021 at 10:35:47AM +0100, Stefan Hajnoczi wrote: > On Mon, Jun 21, 2021 at 11:56:30PM +0530, Niteesh G. S. wrote: > > TLDR: The goal of this mail wasn't to review the dummy command I had posted > > but > > rather start a discussion regarding the implementation of the QMP > > documentation > > retrieval command for the TUI. > > It's not clear to me what exactly you wanted to discuss. Here is the QMP > schema from the commit you linked. It's something concrete that we can > discuss: > > ## > # @CommandDocumentation: > # > # A object representing documentation for a command. > # > # @name: Command name > # > # @doc: A string containing the documentation. > > Is @doc in some kind of markup or plain text? There's a more fundamental question about structure here too. The documentation is not a arbitrary block of text. It contains specific data items for individual parameters, and return type, along with version number annotation too. If we're returning a single string, then the caller needs to know how to parser the current docs structure we're using for commands. Right now that's an internal only format, and I'm not sure we want to expose that as an ABI to consumers of QEMU. So quite possibly we need to return structured data instead eg potentially we need { 'struct': 'CommandDocumentation', 'data': {'name': 'str', 'summary': 'str' 'description: 'str' 'parameters': ['str'], 'return': str, 'since': str, 'notes': str, 'example':str, } } > > # > ## > { 'struct': 'CommandDocumentation', > 'data': {'name': 'str', 'doc': 'str'} } > > ## > # @query-cmd-doc: > # > # (A simple *prototype* implementation) > # Command query-cmd-doc will return the documentation for the command > # specified. This will help QMP clients currently the AQMP TUI to show > # documentation related to a specific command. > # > # @command-name: The command name to query documentation for > # > # Returns: A @CommandDocumentation object containing the documentation. > # > # Since: TODO: Add a number > ## > { 'command': 'query-cmd-doc', > 'data': { 'command-name': 'str' }, > 'returns': 'CommandDocumentation' } > > Is there a way to retrieve struct/enum/etc documentation? > > Do you see a need to query multiple items of documentation in a single > command? A single item query command can become a performance bottleneck > if the clients wants to query the documentation for all commands, for > example. This can be solved by making the the return value an array and > allowing multiple commands to be queried at once. If you want batch querying, then it needs to recursively return all the struct/enum docs related to the commands parameters/return values too. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-22 10:26 ` Daniel P. Berrangé @ 2021-06-22 11:04 ` Vladimir Sementsov-Ogievskiy 0 siblings, 0 replies; 8+ messages in thread From: Vladimir Sementsov-Ogievskiy @ 2021-06-22 11:04 UTC (permalink / raw) To: Daniel P. Berrangé, Stefan Hajnoczi Cc: Niteesh G. S., kwolf, ehabkost, Philippe Mathieu-Daudé, armbru, wainersm, qemu-devel, John Snow 22.06.2021 13:26, Daniel P. Berrangé wrote: > There's a more fundamental question about structure here too. > The documentation is not a arbitrary block of text. It contains > specific data items for individual parameters, and return > type, along with version number annotation too. > > If we're returning a single string, then the caller needs to > know how to parser the current docs structure we're using > for commands. Right now that's an internal only format, and > I'm not sure we want to expose that as an ABI to consumers > of QEMU. So quite possibly we need to return structured > data instead > > eg potentially we need > > { 'struct': 'CommandDocumentation', > 'data': {'name': 'str', > 'summary': 'str' > 'description: 'str' > 'parameters': ['str'], > 'return': str, > 'since': str, > 'notes': str, > 'example':str, > } } If make 'since' a separate field, it make sense to report appropriate 'since' for each parameter, so parameters would not be a simple list of strings.. Not saying of types of parameters. -- Best regards, Vladimir ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-22 9:35 ` Stefan Hajnoczi 2021-06-22 10:26 ` Daniel P. Berrangé @ 2021-06-22 18:57 ` Niteesh G. S. 2021-06-23 8:44 ` Stefan Hajnoczi 1 sibling, 1 reply; 8+ messages in thread From: Niteesh G. S. @ 2021-06-22 18:57 UTC (permalink / raw) To: Stefan Hajnoczi Cc: kwolf, vsementsov, ehabkost, Philippe Mathieu-Daudé, armbru, wainersm, qemu-devel, John Snow [-- Attachment #1: Type: text/plain, Size: 4787 bytes --] Hi Stefan, On Tue, Jun 22, 2021 at 3:05 PM Stefan Hajnoczi <stefanha@redhat.com> wrote: > On Mon, Jun 21, 2021 at 11:56:30PM +0530, Niteesh G. S. wrote: > > TLDR: The goal of this mail wasn't to review the dummy command I had > posted > > but > > rather start a discussion regarding the implementation of the QMP > > documentation > > retrieval command for the TUI. > > It's not clear to me what exactly you wanted to discuss. Here is the QMP > schema from the commit you linked. It's something concrete that we can > discuss: > I wanted to discuss the implementation of the documentation retrieval command. Things like 1) The JSON schema we will be using to represent the documentation. 2) How will we be parsing documentation from the JSON files under qemu/qapi? 3) How will/where we'll be storing this parsed information? And other questions which will have to be answered before proceeding to implement this command. 4) Where to get data for autocomplete for the TUI? - One easy way is to hardcode all available commands in the TUI autocomplete. But then we have to make sure to update the autocomplete list for TUI every time one new command gets added to QMP. ## > # @CommandDocumentation: > # > # A object representing documentation for a command. > # > # @name: Command name > # > # @doc: A string containing the documentation. > > Is @doc in some kind of markup or plain text? > Since this is just a prototype I have used plain text. But for the real command I expect something more structured since the comments I have seen in the QAPI schema has some structure associated with them. eg: ## # @query-status: # # Query the run status of all VCPUs # # Returns: @StatusInfo reflecting all VCPUs # # Since: 0.14 # # Example: # # -> { "execute": "query-status" } # <- { "return": { "running": true, # "singlestep": false, # "status": "running" } } # ## We have the following structure 1) Command name 2) Documentation 3) Arguments (if any) 4) Return type with reference to non-primitive data types like structs/enums etc 5) Since 6) Example In the case of commands referring structures/enums and other non-primitive data types if possible we should also add their documentation along with the documentation for the command. Yes, we could find out all the data types referenced by the current command and add them to the documentation if possible. This will make it easy for the user. If it isn't possible then we must allow to also query documentation related to struct/enums etc. # > ## > { 'struct': 'CommandDocumentation', > 'data': {'name': 'str', 'doc': 'str'} } > > ## > # @query-cmd-doc: > # > # (A simple *prototype* implementation) > # Command query-cmd-doc will return the documentation for the command > # specified. This will help QMP clients currently the AQMP TUI to show > # documentation related to a specific command. > # > # @command-name: The command name to query documentation for > # > # Returns: A @CommandDocumentation object containing the documentation. > # > # Since: TODO: Add a number > ## > { 'command': 'query-cmd-doc', > 'data': { 'command-name': 'str' }, > 'returns': 'CommandDocumentation' } > > Is there a way to retrieve struct/enum/etc documentation? > Not sure. I have gone through the parser code in qemu/scripts/qapi and also have seen the parser being used for documentation generation but I still don't understand the capabilities of the parser. > Do you see a need to query multiple items of documentation in a single > command? A single item query command can become a performance bottleneck > if the clients wants to query the documentation for all commands, for > example. This can be solved by making the the return value an array and > allowing multiple commands to be queried at once. > Why will clients want to query the documentation for all commands? Even if they do won't that be an infrequent operation? From the TUI perspective, I think it will be enough if we just have the capability to service one command at a time. We can also have the TUI cache the results and validate the cache during the greeting process by sending some kind of hash to notify if any documentation has changed or not. > > Do you see a need for wildcard queries where the client does not have > the full command name? I guess the client has enough auto-completion > information to search all commands on the client side, so maybe this > functionality isn't necessary here? > One of my major questions(also mentioned above) is how will the client-side get information regarding all the commands available in QMP? If we implement a proper autocomplete feature then I don't think we will have to worry about wildcard queries. > Stefan > [-- Attachment #2: Type: text/html, Size: 9447 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: RFC: Implementation of QMP documentation retrieval command 2021-06-22 18:57 ` Niteesh G. S. @ 2021-06-23 8:44 ` Stefan Hajnoczi 0 siblings, 0 replies; 8+ messages in thread From: Stefan Hajnoczi @ 2021-06-23 8:44 UTC (permalink / raw) To: Niteesh G. S. Cc: kwolf, vsementsov, ehabkost, Philippe Mathieu-Daudé, armbru, wainersm, qemu-devel, John Snow [-- Attachment #1: Type: text/plain, Size: 5496 bytes --] On Wed, Jun 23, 2021 at 12:27:55AM +0530, Niteesh G. S. wrote: > Hi Stefan, > On Tue, Jun 22, 2021 at 3:05 PM Stefan Hajnoczi <stefanha@redhat.com> wrote: > > > On Mon, Jun 21, 2021 at 11:56:30PM +0530, Niteesh G. S. wrote: > > > TLDR: The goal of this mail wasn't to review the dummy command I had > > posted > > > but > > > rather start a discussion regarding the implementation of the QMP > > > documentation > > > retrieval command for the TUI. > > > > It's not clear to me what exactly you wanted to discuss. Here is the QMP > > schema from the commit you linked. It's something concrete that we can > > discuss: > > > > I wanted to discuss the implementation of the documentation retrieval > command. Things like > 1) The JSON schema we will be using to represent the documentation. This is similar to the question I asked about markup below. > 2) How will we be parsing documentation from the JSON files under qemu/qapi? > 3) How will/where we'll be storing this parsed information? qapi/*.json files are processed when QEMU is built. The QAPI code generator (scripts/qapi/) can be extended if necessary to generate convenient output (e.g. C code containing an array of structs with the QMP command documentation). > And other questions which will have to be answered before proceeding to > implement this command. > 4) Where to get data for autocomplete for the TUI? > > - One easy way is to hardcode all available commands in the TUI > autocomplete. But then we have to make sure to update the autocomplete > list for TUI every time one new command gets added to QMP. The query-qmp-schema QMP command provides the information needed for autocompletion. > > > ## > > # @CommandDocumentation: > > # > > # A object representing documentation for a command. > > # > > # @name: Command name > > # > > # @doc: A string containing the documentation. > > > > Is @doc in some kind of markup or plain text? > > > > Since this is just a prototype I have used plain text. But for the real > command > I expect something more structured since the comments I have seen in the > QAPI schema has some structure associated with them. > eg: > ## > # @query-status: > # > # Query the run status of all VCPUs > # > # Returns: @StatusInfo reflecting all VCPUs > # > # Since: 0.14 > # > # Example: > # > # -> { "execute": "query-status" } > # <- { "return": { "running": true, > # "singlestep": false, > # "status": "running" } } > # > ## > We have the following structure > 1) Command name > 2) Documentation > 3) Arguments (if any) > 4) Return type with reference to non-primitive data types like > structs/enums etc > 5) Since > 6) Example > > In the case of commands referring structures/enums and other non-primitive > data types > if possible we should also add their documentation along with the > documentation > for the command. > Yes, we could find out all the data types referenced by the current command > and > add them to the documentation if possible. This will make it easy for the > user. > If it isn't possible then we must allow to also query documentation related > to struct/enums etc. > > # > > ## > > { 'struct': 'CommandDocumentation', > > 'data': {'name': 'str', 'doc': 'str'} } > > > > ## > > # @query-cmd-doc: > > # > > # (A simple *prototype* implementation) > > # Command query-cmd-doc will return the documentation for the command > > # specified. This will help QMP clients currently the AQMP TUI to show > > # documentation related to a specific command. > > # > > # @command-name: The command name to query documentation for > > # > > # Returns: A @CommandDocumentation object containing the documentation. > > # > > # Since: TODO: Add a number > > ## > > { 'command': 'query-cmd-doc', > > 'data': { 'command-name': 'str' }, > > 'returns': 'CommandDocumentation' } > > > > Is there a way to retrieve struct/enum/etc documentation? > > > Not sure. I have gone through the parser code in qemu/scripts/qapi and also > have > seen the parser being used for documentation generation but I still don't > understand > the capabilities of the parser. > > > > Do you see a need to query multiple items of documentation in a single > > command? A single item query command can become a performance bottleneck > > if the clients wants to query the documentation for all commands, for > > example. This can be solved by making the the return value an array and > > allowing multiple commands to be queried at once. > > > Why will clients want to query the documentation for all commands? Even if > they do > won't that be an infrequent operation? > From the TUI perspective, I think it will be enough if we just have the > capability to > service one command at a time. We can also have the TUI cache the results > and > validate the cache during the greeting process by sending some kind of hash > to > notify if any documentation has changed or not. Querying multiple commands is probably not necessary in the interactive TUI use case. But if we're adding a new command it's nice to make it general so it can be used for other things in the future. That said, I don't have a strong opinion either way. I was just curious if anyone can think of a reason to support multiple items in a single query. Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-06-23 8:46 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-06-16 15:48 RFC: Implementation of QMP documentation retrieval command Niteesh G. S. 2021-06-21 14:58 ` Philippe Mathieu-Daudé 2021-06-21 18:26 ` Niteesh G. S. 2021-06-22 9:35 ` Stefan Hajnoczi 2021-06-22 10:26 ` Daniel P. Berrangé 2021-06-22 11:04 ` Vladimir Sementsov-Ogievskiy 2021-06-22 18:57 ` Niteesh G. S. 2021-06-23 8:44 ` Stefan Hajnoczi
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.