Re: [SPDK] About dump/load (or export/edit/import) JSON file by SPDK

* Re: [SPDK] About dump/load (or export/edit/import) JSON file by SPDK
@ 2018-02-09 19:48 Wodkowski, PawelX
  0 siblings, 0 replies; 14+ messages in thread
From: Wodkowski, PawelX @ 2018-02-09 19:48 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 5657 bytes --]

Hi Shuhei,

I saw trello board you created. This is a great start to describe what we can do in SPDK RPC topic for next release.

About the “Doing” list and “SPDK Monitor like QEMU QMP” card (https://trello.com/c/CPq9jcyG/14-spdk-monitor-like-qemu-qmp):
I noticed your comment there about many things to do. I feel a bit lost there. Which part of that card you are currently working on? Can we split this card to smaller tasks that each of us can handle? Or, if you are handling this as a whole, I will take one of the TO DO item.

As an offtopic: can you attend to our next community meeting? It would be great to discuss this.

Pawel

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of ???? / MATSUMOTO,SHUUHEI
Sent: Monday, February 5, 2018 10:05 AM
To: spdk(a)lists.01.org
Subject: Re: [SPDK] About dump/load (or export/edit/import) JSON file by SPDK

Hi Jim, Pawel, Daniel, Ben, and All

Thank you so much for your really really helpful feedback.

My motivation was first and I will develop my ability to go forward together with you.

First I created the board "JSON Configuration File" and tried to include your feedback.

https://trello.com/b/lMJIrv91/json-configuration-file
Trello<https://trello.com/b/lMJIrv91/json-configuration-file>
trello.com
Organize anything, together. Trello is a collaboration tool that organizes your projects into boards. In one glance, know what's being worked on, who's working on what, and where something is in a process.

If you find anything not covered yet or not described correctly, please update it.

In particular I'm happy for your any feedback about how to preserve ordering of requests.
https://trello.com/c/Vy02q59e/3-preserve-ordering-of-the-list-of-json-rpc-requests-in-json-config-file
Trello<https://trello.com/c/Vy02q59e/3-preserve-ordering-of-the-list-of-json-rpc-requests-in-json-config-file>
trello.com
Organize anything, together. Trello is a collaboration tool that organizes your projects into boards. In one glance, know what's being worked on, who's working on what, and where something is in a process.

Besides, I'll have a presentation in the upcoming SPDK china summit. If you are OK, I would like to include JSON config file even just a little bit because we may be able to get any feedback helpful to go forward.

Thank you,
Shuhei

________________________________
差出人: SPDK <spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>> が Walker, Benjamin <benjamin.walker(a)intel.com<mailto:benjamin.walker(a)intel.com>> の代理で送信
送信日時: 2018年2月3日 2:48
宛先: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
件名: [!]Re: [SPDK] About dump/load (or export/edit/import) JSON file by SPDK

On Fri, 2018-02-02 at 10:24 -0700, Daniel Verkamp wrote:
> I agree that we definitely want to be able to have comments in the
> configuration file.
>
> We already have support for (non-standard) JavaScript-style // and /* */
> comments in our JSON parser; it just needs to be enabled with the
> SPDK_JSON_PARSE_FLAG_ALLOW_COMMENTS flag.
>
> However, if we make the config file parser an external Python tool, we would
> probably need something like JSON-minify to remove the comments first (or a
> JSON parser that has non-standard comment support).

I think the top level object should be a JSON list which contains the already-
defined RPC calls. The 'id' and 'jsonrpc' fields can be omitted and
automatically inserted by the config file parser. JavaScript-style comments
should be allowed, which can be removed using JSON-minify. I'm glad this is the
consensus that's emerging - it should make this really easy to use.

> I like the idea of having an external tool (possibly written in Python) that
> loads the JSON config file and sends RPC requests one at a time through the
> existing JSON-RPC interface; that keeps the SPDK library code modifications to
> a minimum.

I agree with this one. Let's just write a simple Python script that loads the
config file (json.loads), then iterates through the list of RPC commands and
sends them one by one using the regular interface.

On Fri, 2018-02-02 at 01:22 +0000, 松本周平 / MATSUMOTO，SHUUHEI wrote:
> We should keep the current config file implementation for compatibility?
> Or is it OK to replace the whole to the new one once everything is prepared?

We need to have some level of backward compatibility, at least for a full
release cycle, in my opinion. I see two options:

1) Leave the current config file parsing in our code. Add the ability to dump
JSON-RPC config files in response to SIGUSR (which is planned anyway). If the
user passes a config file on the command line, print out a big deprecation
warning, but otherwise continue. This gives users the ability to load up their
old configuration file, then dump the new JSON-RPC format. After a release
cycle, we can delete support for parsing the old config file. At that point, if
someone needs to use an old config file they can always go back to the release
that supported both formats and use it as a converter.

2) Write a Python script that converts the old config format into the new JSON
format. This allows us to delete the current config parsing code immediately and
provide the conversion utility with every release going forward. My only concern
with this approach is it may not be trivial to do the conversion, but I could be
wrong.
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 19990 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread