From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5179FC2D0DB for ; Thu, 23 Jan 2020 21:08:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F2E9B21835 for ; Thu, 23 Jan 2020 21:08:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="J5FJe49h" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2E9B21835 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iujhw-00051k-1L for qemu-devel@archiver.kernel.org; Thu, 23 Jan 2020 16:08:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53183) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iujhD-0004TL-GW for qemu-devel@nongnu.org; Thu, 23 Jan 2020 16:07:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iujhB-0004xD-5S for qemu-devel@nongnu.org; Thu, 23 Jan 2020 16:07:22 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:51799 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iujhA-0004wz-Vo for qemu-devel@nongnu.org; Thu, 23 Jan 2020 16:07:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579813640; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=jqUF97ZpnbkM6SDaT1xEsvUxuYOH3lEr7nh87onLwKM=; b=J5FJe49hHp5I92esPJRjDQBOuKyrtBt5j4eRQUknth5QUS2gWj0lnZhD3iEHldijxe9kH1 DJ1SIEg5DxTm6gU5Ugy3bMV3kw+4pXs/peowgRN4OS4P8SzNlPpfr/obPjzQAbhfjJ+IXK +2m0ezIW6MHvq03xvqzz8p6agGGk0bw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-429-ZrrhuJPRN2GWUyDEYz5XDg-1; Thu, 23 Jan 2020 16:07:17 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C8823DBA3; Thu, 23 Jan 2020 21:07:15 +0000 (UTC) Received: from [10.18.17.116] (dhcp-17-116.bos.redhat.com [10.18.17.116]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A12A81207; Thu, 23 Jan 2020 21:07:09 +0000 (UTC) Subject: Re: Making QEMU easier for management tools and applications To: =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?= References: <87h81unja8.fsf@dusky.pond.sub.org> <20191224134139.GD2710539@redhat.com> <30664f6e-81da-a6e6-9b20-037fc91290fb@redhat.com> <878slyej29.fsf@dusky.pond.sub.org> <20200123190145.GI657556@redhat.com> From: John Snow Autocrypt: addr=jsnow@redhat.com; prefer-encrypt=mutual; keydata= mQINBFTKefwBEAChvwqYC6saTzawbih87LqBYq0d5A8jXYXaiFMV/EvMSDqqY4EY6whXliNO IYzhgrPEe7ZmPxbCSe4iMykjhwMh5byIHDoPGDU+FsQty2KXuoxto+ZdrP9gymAgmyqdk3aV vzzmCa3cOppcqKvA0Kqr10UeX/z4OMVV390V+DVWUvzXpda45/Sxup57pk+hyY52wxxjIqef rj8u5BN93s5uCVTus0oiVA6W+iXYzTvVDStMFVqnTxSxlpZoH5RGKvmoWV3uutByQyBPHW2U 1Y6n6iEZ9MlP3hcDqlo0S8jeP03HaD4gOqCuqLceWF5+2WyHzNfylpNMFVi+Hp0H/nSDtCvQ ua7j+6Pt7q5rvqgHvRipkDDVsjqwasuNc3wyoHexrBeLU/iJBuDld5iLy+dHXoYMB3HmjMxj 3K5/8XhGrDx6BDFeO3HIpi3u2z1jniB7RtyVEtdupED6lqsDj0oSz9NxaOFZrS3Jf6z/kHIf h42mM9Sx7+s4c07N2LieUxcfqhFTaa/voRibF4cmkBVUhOD1AKXNfhEsTvmcz9NbUchCkcvA T9119CrsxfVsE7bXiGvdXnzyGLXdsoosjzwacKdOrVaDmN3Uy+SHiQXo6TlkSdV0XH2PUxTM LsBFIO9qXO43Ai6J6iPAP/01l8fuZfpJE0/L/c25yyaND7xA3wARAQABtCpKb2huIFNub3cg KEpvaG4gSHVzdG9uKSA8anNub3dAcmVkaGF0LmNvbT6JAlQEEwECAD4CGwMCHgECF4AFCwkI BwMFFQoJCAsFFgIDAQAWIQT665cRoSz0dYEvGPKIqQZNGDVh6wUCXF392gUJC1Xq3gAKCRCI qQZNGDVh6558D/9pM4pu4njX5aT6uUW3vAmbWLF1jfPxiTQgSHAnm9EBMZED/fsvkzj97clo LN7JKmbYZNgJmR01A7flG45V4iOR/249qAfaVuD+ZzZi1R4jFzr13WS+IEdn0hYp9ITndb7R ezW+HGu6/rP2PnfmDnNowgJu6Dp6IUEabq8SXXwGHXZPuMIrsXJxUdKJdGnh1o2u7271yNO7 J9PEMuMDsgjsdnaGtv7aQ9CECtXvBleAc06pLW2HU10r5wQyBMZGITemJdBhhdzGmbHAL0M6 vKi/bafHRWqfMqOAdDkv3Jg4arl2NCG/uNateR1z5e529+UlB4XVAQT+f5T/YyI65DFTY940 il3aZhA8u788jZEPMXmt94u7uPZbEYp7V0jt68SrTaOgO7NaXsboXFjwEa42Ug5lB5d5/Qdp 1AITUv0NJ51kKwhHL1dEagGeloIsGVQILmpS0MLdtitBHqZLsnJkRvtMaxo47giyBlv2ewmq tIGTlVLxHx9xkc9aVepOuiGlZaZB72c9AvZs9rKaAjgU2UfJHlB/Hr4uSk/1EY0IgMv4vnsG 1sA5gvS7A4T4euu0PqHtn2sZEWDrk5RDbw0yIb53JYdXboLFmFXKzVASfKh2ZVeXRBlQQSJi 3PBR1GzzqORlfryby7mkY857xzCI2NkIkD2eq+HhzFTfFOTdGrkCDQRUynn8ARAAwbhP45BE d/zAMBPV2dk2WwIwKRSKULElP3kXpcuiDWYQob3UODUUqClO+3aXVRndaNmZX9WbzGYexVo3 5j+CVBCGr3DlU8AL9pp3KQ3SJihWcDed1LSmUf8tS+10d6mdGxDqgnd/OWU214isvhgWZtZG MM/Xj7cx5pERIiP+jqu7PT1cibcfcEKhPjYdyV1QnLtKNGrTg/UMKaL+qkWBUI/8uBoa0HLs NH63bXsRtNAG8w6qG7iiueYZUIXKc4IHINUguqYQJVdSe+u8b2N5XNhDSEUhdlqFYraJvX6d TjxMTW5lzVG2KjztfErRNSUmu2gezbw1/CV0ztniOKDA7mkQi6UIUDRh4LxRm5mflfKiCyDQ L6P/jxHBxFv+sIgjuLrfNhIC1p3z9rvCh+idAVJgtHtYl8p6GAVrF+4xQV2zZH45tgmHo2+S JsLPjXZtWVsWANpepXnesyabWtNAV4qQB7/SfC77zZwsVX0OOY2Qc+iohmXo8U7DgXVDgl/R /5Qgfnlv0/3rOdMt6ZPy5LJr8D9LJmcP0RvX98jyoBOf06Q9QtEwJsNLCOCo2LKNL71DNjZr nXEwjUH66CXiRXDbDKprt71BiSTitkFhGGU88XCtrp8R9yArXPf4MN+wNYBjfT7K29gWTzxt 9DYQIvEf69oZD5Z5qHYGp031E90AEQEAAYkCPAQYAQIAJgIbDBYhBPrrlxGhLPR1gS8Y8oip Bk0YNWHrBQJcXf3JBQkLVerNAAoJEIipBk0YNWHrU1AP/1FOK2SBGbyhHa5vDHuf47fgLipC e0/h1E0vdSonzlhPxuZoQ47FjzG9uOhqqQG6/PqtWs/FJIyz8aGG4aV+pSA/9Ko3/2ND8MSY ZflWs7Y8Peg08Ro01GTHFITjEUgHpTpHiT6TNcZB5aZNJ8jqCtW5UlqvXXbVeSTmO70ZiVtc vUJbpvSxYmzhFfZWaXIPcNcKWL1rnmnzs67lDhMLdkYVf91aml/XtyMUlfB8Iaejzud9Ht3r C0pA9MG57pLblX7okEshxAC0+tUdY2vANWFeX0mgqRt1GSuG9XM9H/cKP1czfUV/FgaWo/Ya fM4eMhUAlL/y+/AJxxumPhBXftM4yuiktp2JMezoIMJI9fmhjfWDw7+2jVrx9ze1joLakFD1 rVAoHxVJ7ORfQ4Ni/qWbQm3T6qQkSMt4N/scNsMczibdTPxU7qtwQwIeFOOc3wEwmJ9Qe3ox TODQ0agXiWVj0OXYCHJ6MxTDswtyTGQW+nUHpKBgHGwUaR6d1kr/LK9+5LpOfRlK9VRfEu7D PGNiRkr8Abp8jHsrBqQWfUS1bAf62bq6XUel0kUCtb7qCq024aOczXYWPFpJFX+nhp4d7NeH Edq+wlC13sBSiSHC7T5yssJ+7JPa2ATLlSKhEvBsLe2TsSTTtFlA0nBclqhfJXzimiuge9qU E40lvMWBuQINBFTKimUBEADDbJ+pQ5M4QBMWkaWImRj7c598xIZ37oKM6rGaSnuB1SVb7YCr Ci2MTwQcrQscA2jm80O8VFqWk+/XsEp62dty47GVwSfdGje/3zv3VTH2KhOCKOq3oPP5ZXWY rz2d2WnTvx++o6lU7HLHDEC3NGLYNLkL1lyVxLhnhvcMxkf1EGA1DboEcMgnJrNB1pGP27ww cSfvdyPGseV+qZZa8kuViDga1oxmnYDxFKMGLxrClqHrRt8geQL1Wj5KFM5hFtGTK4da5lPn wGNd6/CINMeCT2AWZY5ySz7/tSZe5F22vPvVZGoPgQicYWdNc3ap7+7IKP86JNjmec/9RJcz jvrYjJdiqBVldXou72CtDydKVLVSKv8c2wBDJghYZitfYIaL8cTvQfUHRYTfo0n5KKSec8Vo vjDuxmdbOUBA+SkRxqmneP5OxGoZ92VusrwWCjry8HRsNdR+2T+ClDCO6Wpihu4V3CPkQwTy eCuMHPAT0ka5paTwLrnZIxsdfnjUa96T10vzmQgAxpbbiaLvgKJ8+76OPdDnhddyxd2ldYfw RkF5PEGg3mqZnYKNNBtwjvX49SAvgETQvLzQ8IKVgZS0m4z9qHHvtc1BsQnFfe+LJOFjzZr7 CrDNJMqk1JTHYsSi2JcN3vY32WMezXSQ0TzeMK4kdnclSQyp/h23GWod5QARAQABiQRbBBgB AgAmAhsCFiEE+uuXEaEs9HWBLxjyiKkGTRg1YesFAlxd/coFCQtV2mQCKcFdIAQZAQIABgUC VMqKZQAKCRB974EGqvw5DiJoEACLmuiRq9ifvOh5DyBFwRS7gvA14DsGQngmC57EzV0EFcfM XVi1jX5OtwUyUe0Az5r6lHyyHDsDsIpLKBlWrYCeLpUhRR3oy181T7UNxvujGFeTkzvLAOo6 Hs3b8Wv9ARg+7acRYkQRNY7k0GIJ6YZz149tRyRKAy/vSjsaB9Lt0NOd1wf2EQMKwRVELwJD y0AazGn+0PRP7Bua2YbtxaBmhBBDb2tPpwn8U9xdckB4Vlft9lcWNsC/18Gi9bpjd9FSbdH/ sOUI+3ToWYENeoT4IP09wn6EkgWaJS3nAUN/MOycNej2i4Yhy2wDDSKyTAnVkSSSoXk+tK91 HfqtokbDanB8daP+K5LgoiWHzjfWzsxA2jKisI4YCGjrYQzTyGOT6P6u6SEeoEx10865B/zc 8/vN50kncdjYz2naacIDEKQNZlnGLsGkpCbfmfdi3Zg4vuWKNdWr0wGUzDUcpqW0y/lUXna+ 6uyQShX5e4JD2UPuf9WAQ9HtgSAkaDd4O1I2J41sleePzZOVB3DmYgy+ECRJJ5nw3ihdxpgc y/v3lfcJaqiyCv0PF+K/gSOvwhH7CbVqARmptT7yhhxqFdaYWo2Z2ksuKyoKSRMFCXQY5oac uTmyPIT4STFyUQFeqSCWDum/NFNoSKhmItw2Td+4VSJHShRVbg39KNFPZ7mXYAkQiKkGTRg1 YesWJA/+PV3qDUtPNEGwjVvjQqHSbrBy94tu6gJvPHgGPtRDYvxnCaJsmgiC0pGB2KFRsnfl 2zBNBEWF/XwsI081jQE5UO60GKmHTputChLXpVobyuc+lroG2YhknXRBAV969SLnZR4BS/1s Gi046gOXfaKYatve8BiZr5it5Foq3FMPDNgZMit1H9Dk8rkKFfDMRf8EGS/Z+TmyEsIf99H7 TH3n7lco8qO81fSFwkh4pvo2kWRFYTC5vsIVQ+GqVUp+W1DZJHxX8LwWuF1AzUt4MUTtNAvy TXl5EgsmoY9mpNNL7ZnW65oG63nEP5KNiybvuQJzXVxR8eqzOh2Mod4nHg3PE7UCd3DvLNsn GXFRo44WyT/G2lArBtjpkut7bDm0i1nENABy2UgS+1QvdmgNu6aEZxdNthwRjUhuuvCCDMA4 rCDQYyakH2tJNQgkXkeLodBKF4bHiBbuwj0E39S9wmGgg+q4OTnAO/yhQGknle7a7G5xHBwE i0HjnLoJP5jDcoMTabZTIazXmJz3pKM11HYJ5/ZsTIf3ZRJJKIvXJpbmcAPVwTZII6XxiJdh RSSX4Mvd5pL/+5WI6NTdW6DMfigTtdd85fe6PwBNVJL2ZvBfsBJZ5rxg1TOH3KLsYBqBTgW2 glQofxhkJhDEcvjLhe3Y2BlbCWKOmvM8XS9TRt0OwUs= Message-ID: <2561a069-ce5f-3c30-b04e-db7cd2fcdc85@redhat.com> Date: Thu, 23 Jan 2020 16:07:09 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.3.0 MIME-Version: 1.0 In-Reply-To: <20200123190145.GI657556@redhat.com> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: ZrrhuJPRN2GWUyDEYz5XDg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , "Denis V. Lunev" , Cleber Rosa , Stefan Hajnoczi , Markus Armbruster , Eduardo Habkost , qemu-devel , Paolo Bonzini , =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= , Dominik Csapak Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On 1/23/20 2:01 PM, Daniel P. Berrang=C3=A9 wrote: > On Thu, Jan 23, 2020 at 12:58:45PM -0500, John Snow wrote: >> Yes, I agree: Scrap and start over. >> >> What SHOULD the syntax look like, though? Clearly the idea of qmp-shell >> is that it offers a convenient way to enter the top-level keys of the >> arguments dict. This works absolutely fine right up until you need to >> start providing nested definitions. >> >> For the nesting, we say: "Go ahead and use JSON, but you have to take >> all the spaces out." >> >> This... works, charitably, but is hardly what I would call usable. >> >> For the CLI, we offer a dot syntax notation that resembles nothing in >> particular. It often seems the case that it isn't expressive enough to >> map losslessly to JSON. I suspect it doesn't handle siblings very well. >> >> A proper HMP-esque TUI would likely have need of coming up with its own >> pet syntax for commands that avoid complicated nested JSON definitions, >> but for effort:value ratio, having a QMP shorthand shell that works >> arbitrarily with any command might be a better win. >> >> Do we still have a general-case problem of how to represent QAPI >> structures in plaintext? Will this need to be solved for the CLI, too? >=20 > I don't know if you've ever looked at how Kubernetes/OpenShift exposes > its functionality on the command line ? I think it is interesting to > note that they largely don't try to solve this problem of flattening > JSON for humans on the CLI using their client. >=20 > Everything in their world is an object described in JSON/YAML, and > there are a small set of generic commands that can act on any type > of object. These commands primarily input and output JSON or YAML > documents directly. As a user you can pick either format since it > can do a lossless conversion in both directions server side. >=20 > So when configuring objects you'll always provide a JSON/YAML doc. > They've got some clever stuff for updating objects where you can > provide a JSON patch for only the bits which need changing. >=20 > When querying/listing objects by default it displays only a small > subset of their config information in a human friendly-ish format. > If you want to see everything then you ask for it in JSON/YAML > format. There's also an expression language that lets you extract > particular pieces of information based on requested properties, > and you can filter the list of objects based on attributes and so > on. >=20 > I think it is fair to say the structure of kubernetes object config > is on a par with hierarchical complexity of QEMU. The lack of a simple > human targetted data input format does not appear to have negatively > impacted the adoption of Kubernetes. It is worth questioning why this > is the case, while we feel the human CLI syntax for QEMU is so > critically important to QEMU's future ? >=20 Well, if the "human CLI syntax" is "Feed it YAML documents", that's perfectly peachy for me, too! We need a good, consistent interface. Exactly what that interface is isn't really a blocking concern. Configuring a VM is a complicated process and has a lot of moving widgets. Feeding it a YAML file is a reasonable thought. Having JSON and requiring people to type bastardized and differing versions of it in 8 places _is_ a concern. We can't document reasonably all of the different flavors and why they differ from one place to the next. We can unify it. If unifying it means using JSON/YAML/TOML/MAML[1]* everywhere and abandoning a CLI altogether, that's just as well. >=20 >=20 > Part of it is that the machine oriented data input format via QMP > suffers from the fact that it came second in QEMU after HMP. As a > result, 90% of the documentation that illustrates QEMU will use the > human CLI syntax, varying vintages of that. Since all the docs are > focused on the HMP/CLI syntax, whenever there's a new feature we > feel pressured to expose it & document it in the human syntax too. >=20 Decent observation; but there's often no reasonable way to NOT use the CLI, so I think it's not unreasonable that we try to expose features via the CLI still. However, the docs being badly out of date are a problem. We actively lead people towards harmful / difficult to support paradigms. > All this results in a situation where JSON is functionally the best > way to configure QEMU, but practically the worse, since very few > people understand how to actually use it. This is a vicious circle > holding back QMP/JSON and making the human syntax an ever greater > burden for users & maintainers >=20 I do basically agree. >=20 > IOW, the difficulty with configuring QEMU via JSON is not the fault > of JSON itself, it is the lack of knowledge amongst users and docs, > compounded by our never ending "improvements" to the human syntax. > There are other factors too, such as our only partial coverage of > config using JSON - some is only possible via the CLI still. >=20 I'm fine with getting rid of HMP entirely, I think. It's a weird interface with bizarre behavior that's hard to support. There's a few commands in there we just don't support at all, but maybe it's time to start deprecating one-by-one any of the individual commands that are better served by QMP these days, to send the message that HMP's days are numbered. Bye-bye! As for the CLI, well, that's part of the discussion at hand... >=20 > I guess my point is that with a scrap & startover view point, we > should arguably completely ignore the design question of how to > flatten JSON for humans/command line, as it is the wrong problem. > Instead focus on the problem of making use of JSON the best way > to deal with QEMU both functionally and practically, for humans > and machines alike. >=20 Well, sure. The context of this email was qmp-shell though, which is meant to help facilitate the entry of JSON commands so that you *can* indeed just forego the CLI/HMP entirely. If you are of the opinion that every user of QEMU should be copy/pasting JSON straight into a socket and we should delete qmp-shell, that's certainly a fine opinion. I'm coming from the side that I love qmp-shell; I find it useful, but it has some syntax problems. How do I solve them? Is there a way to solve them? QAPI is here to stay, and QAPI involves hierarchical data. That data is usually best represented by something like json or yaml, but those are hard to type in a shell. What do we do about that? >=20 > Regards, > Daniel >=20 --=20 =E2=80=94js [1]*: Markus Armbruster's Markup Language; I have written this joke with the explicit goal of tormenting Markus.