From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([209.51.188.92]:33208) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEHRm-0007Zs-IY for qemu-devel@nongnu.org; Wed, 10 Apr 2019 13:55:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEHRk-0003nL-RI for qemu-devel@nongnu.org; Wed, 10 Apr 2019 13:55:42 -0400 References: <20190410012413.31569-1-jsnow@redhat.com> <20190410012413.31569-3-jsnow@redhat.com> <87sguq2yn1.fsf@dusky.pond.sub.org> From: John Snow Message-ID: <004289a6-9df9-ae99-53e6-56e555709434@redhat.com> Date: Wed, 10 Apr 2019 13:55:33 -0400 MIME-Version: 1.0 In-Reply-To: <87sguq2yn1.fsf@dusky.pond.sub.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Markus Armbruster Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Kevin Wolf , Eduardo Habkost , Max Reitz , Cleber Rosa On 4/10/19 1:54 AM, Markus Armbruster wrote: > John Snow writes: > >> Presently we use hxtool and a .hx format to generate a few things like >> the qemu_img subcommand dispatch table, the qemu_img help() display output, >> and a help output in qemu-img.texi. >> >> Unfortunately, this means that this information is duplicated in at least >> three places: >> >> (1) in qemu-img-cmds.hx as a human readable string >> (2) in qemu-img-cmds.hx as a texi string >> (3) in qemu-img.texi again, as a texi string >> >> We can do a little better, though: all of these sources can be generated >> from a single json file. Add a new small tool ("pxtool") that can do this. >> >> This tool can at least handle generating (1) and (2) from the same source >> without needing to reduplicate that information. Deduplicating (3) happens >> in the next patch. >> >> Notes: >> - The json format was chosen to be very "stupid", and the command line >> documentation is being kept one-per-line to make future diffs easier >> to read. >> - It's easy enough to generate the human-readable output from the texi >> output by removing '@var{foo}' with 'foo'. >> - The qemu-img command dispatch always calls img_cmdname, so we don't >> need to store this information separately, either. >> - The need for DEF() macros could be removed as well, but I left it in >> to create a minimally disruptive patch. >> Signed-off-by: John Snow > > We got just five .hx: > > qemu-img.cmds.hx killed off by this patch > qemu-options.hx CLI QAPIfication should kill this off > hw/audio/pl041.hx misnamed, not actually food for hxtool > hmp-commands.hx no exit strategy > hmp-commands-info.hx for these two, yet > > CLI QAPIfication got stuck in the back-burner, and as long as that's the > case, it's not an alternative to your patches. > Good to know. I figure that at least while we wait on something more comprehensive there's no real short term harm in tidying up. Something I'd really like to do is define a python/json-esque configuration file that allows you to specify some "common options" that are shared between all of the sub-commands, and then define each command in terms of both which common options it possesses, and then any local command-specific options it has. (Why the common options design? So that argument names and command options can be encouraged to be identical and identically documented between all subcommands that use them.) Then, from the configuration file, generate all of the necessary C parsers (I have a protoype for this, it's not difficult to do in at least the basic case) that can return boxed command arguments, and then also generate the help strings from that metadata. I suspect that work *IS* something that might brush up against / should use (or extend) QAPI code. Then, I'd like to find a way to split qemu-img.texi into sub-command files and find a way to source the same "informative text" for both: (1) The texi output, as per usual, and (2) qemu-img subcommand --help such that --help, when passed as an argument to the subcommand, prints out help only relevant to the subcommand instead, e.g. `qemu-img check --help` might print the "common options" section only as it relates to commands actually used by the check command, then prints the check section of the texi as formatted for terminals. This way, we can have manpages, html pages, and interactive help text all derived from the same semi-automated sources, always up to date and much easier to read/discover/parse by human eyeballs. That's what I'd like to accomplish, ultimately. For now, I think this RFC set is not harmful and won't bother anybody. It's definitely not worse than what we have now, fragility of removing @var{} tokens and all. --js From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A72DC10F11 for ; Wed, 10 Apr 2019 17:56:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E866F2063F for ; Wed, 10 Apr 2019 17:56:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E866F2063F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([127.0.0.1]:35328 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEHSx-000080-0f for qemu-devel@archiver.kernel.org; Wed, 10 Apr 2019 13:56:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33208) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hEHRm-0007Zs-IY for qemu-devel@nongnu.org; Wed, 10 Apr 2019 13:55:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hEHRk-0003nL-RI for qemu-devel@nongnu.org; Wed, 10 Apr 2019 13:55:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52619) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hEHRg-0003gs-OV; Wed, 10 Apr 2019 13:55:37 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6F411307CB29; Wed, 10 Apr 2019 17:55:35 +0000 (UTC) Received: from [10.18.17.206] (dhcp-17-206.bos.redhat.com [10.18.17.206]) by smtp.corp.redhat.com (Postfix) with ESMTP id 42070272D2; Wed, 10 Apr 2019 17:55:34 +0000 (UTC) To: Markus Armbruster References: <20190410012413.31569-1-jsnow@redhat.com> <20190410012413.31569-3-jsnow@redhat.com> <87sguq2yn1.fsf@dusky.pond.sub.org> From: John Snow Openpgp: preference=signencrypt Autocrypt: addr=jsnow@redhat.com; prefer-encrypt=mutual; keydata= mQINBFTKefwBEAChvwqYC6saTzawbih87LqBYq0d5A8jXYXaiFMV/EvMSDqqY4EY6whXliNO IYzhgrPEe7ZmPxbCSe4iMykjhwMh5byIHDoPGDU+FsQty2KXuoxto+ZdrP9gymAgmyqdk3aV vzzmCa3cOppcqKvA0Kqr10UeX/z4OMVV390V+DVWUvzXpda45/Sxup57pk+hyY52wxxjIqef rj8u5BN93s5uCVTus0oiVA6W+iXYzTvVDStMFVqnTxSxlpZoH5RGKvmoWV3uutByQyBPHW2U 1Y6n6iEZ9MlP3hcDqlo0S8jeP03HaD4gOqCuqLceWF5+2WyHzNfylpNMFVi+Hp0H/nSDtCvQ ua7j+6Pt7q5rvqgHvRipkDDVsjqwasuNc3wyoHexrBeLU/iJBuDld5iLy+dHXoYMB3HmjMxj 3K5/8XhGrDx6BDFeO3HIpi3u2z1jniB7RtyVEtdupED6lqsDj0oSz9NxaOFZrS3Jf6z/kHIf h42mM9Sx7+s4c07N2LieUxcfqhFTaa/voRibF4cmkBVUhOD1AKXNfhEsTvmcz9NbUchCkcvA T9119CrsxfVsE7bXiGvdXnzyGLXdsoosjzwacKdOrVaDmN3Uy+SHiQXo6TlkSdV0XH2PUxTM LsBFIO9qXO43Ai6J6iPAP/01l8fuZfpJE0/L/c25yyaND7xA3wARAQABtCpKb2huIFNub3cg KEpvaG4gSHVzdG9uKSA8anNub3dAcmVkaGF0LmNvbT6JAlQEEwECAD4CGwMCHgECF4AFCwkI BwMFFQoJCAsFFgIDAQAWIQT665cRoSz0dYEvGPKIqQZNGDVh6wUCXF392gUJC1Xq3gAKCRCI qQZNGDVh6558D/9pM4pu4njX5aT6uUW3vAmbWLF1jfPxiTQgSHAnm9EBMZED/fsvkzj97clo LN7JKmbYZNgJmR01A7flG45V4iOR/249qAfaVuD+ZzZi1R4jFzr13WS+IEdn0hYp9ITndb7R ezW+HGu6/rP2PnfmDnNowgJu6Dp6IUEabq8SXXwGHXZPuMIrsXJxUdKJdGnh1o2u7271yNO7 J9PEMuMDsgjsdnaGtv7aQ9CECtXvBleAc06pLW2HU10r5wQyBMZGITemJdBhhdzGmbHAL0M6 vKi/bafHRWqfMqOAdDkv3Jg4arl2NCG/uNateR1z5e529+UlB4XVAQT+f5T/YyI65DFTY940 il3aZhA8u788jZEPMXmt94u7uPZbEYp7V0jt68SrTaOgO7NaXsboXFjwEa42Ug5lB5d5/Qdp 1AITUv0NJ51kKwhHL1dEagGeloIsGVQILmpS0MLdtitBHqZLsnJkRvtMaxo47giyBlv2ewmq tIGTlVLxHx9xkc9aVepOuiGlZaZB72c9AvZs9rKaAjgU2UfJHlB/Hr4uSk/1EY0IgMv4vnsG 1sA5gvS7A4T4euu0PqHtn2sZEWDrk5RDbw0yIb53JYdXboLFmFXKzVASfKh2ZVeXRBlQQSJi 3PBR1GzzqORlfryby7mkY857xzCI2NkIkD2eq+HhzFTfFOTdGrkCDQRUynn8ARAAwbhP45BE d/zAMBPV2dk2WwIwKRSKULElP3kXpcuiDWYQob3UODUUqClO+3aXVRndaNmZX9WbzGYexVo3 5j+CVBCGr3DlU8AL9pp3KQ3SJihWcDed1LSmUf8tS+10d6mdGxDqgnd/OWU214isvhgWZtZG MM/Xj7cx5pERIiP+jqu7PT1cibcfcEKhPjYdyV1QnLtKNGrTg/UMKaL+qkWBUI/8uBoa0HLs NH63bXsRtNAG8w6qG7iiueYZUIXKc4IHINUguqYQJVdSe+u8b2N5XNhDSEUhdlqFYraJvX6d TjxMTW5lzVG2KjztfErRNSUmu2gezbw1/CV0ztniOKDA7mkQi6UIUDRh4LxRm5mflfKiCyDQ L6P/jxHBxFv+sIgjuLrfNhIC1p3z9rvCh+idAVJgtHtYl8p6GAVrF+4xQV2zZH45tgmHo2+S JsLPjXZtWVsWANpepXnesyabWtNAV4qQB7/SfC77zZwsVX0OOY2Qc+iohmXo8U7DgXVDgl/R /5Qgfnlv0/3rOdMt6ZPy5LJr8D9LJmcP0RvX98jyoBOf06Q9QtEwJsNLCOCo2LKNL71DNjZr nXEwjUH66CXiRXDbDKprt71BiSTitkFhGGU88XCtrp8R9yArXPf4MN+wNYBjfT7K29gWTzxt 9DYQIvEf69oZD5Z5qHYGp031E90AEQEAAYkCPAQYAQIAJgIbDBYhBPrrlxGhLPR1gS8Y8oip Bk0YNWHrBQJcXf3JBQkLVerNAAoJEIipBk0YNWHrU1AP/1FOK2SBGbyhHa5vDHuf47fgLipC e0/h1E0vdSonzlhPxuZoQ47FjzG9uOhqqQG6/PqtWs/FJIyz8aGG4aV+pSA/9Ko3/2ND8MSY ZflWs7Y8Peg08Ro01GTHFITjEUgHpTpHiT6TNcZB5aZNJ8jqCtW5UlqvXXbVeSTmO70ZiVtc vUJbpvSxYmzhFfZWaXIPcNcKWL1rnmnzs67lDhMLdkYVf91aml/XtyMUlfB8Iaejzud9Ht3r C0pA9MG57pLblX7okEshxAC0+tUdY2vANWFeX0mgqRt1GSuG9XM9H/cKP1czfUV/FgaWo/Ya fM4eMhUAlL/y+/AJxxumPhBXftM4yuiktp2JMezoIMJI9fmhjfWDw7+2jVrx9ze1joLakFD1 rVAoHxVJ7ORfQ4Ni/qWbQm3T6qQkSMt4N/scNsMczibdTPxU7qtwQwIeFOOc3wEwmJ9Qe3ox TODQ0agXiWVj0OXYCHJ6MxTDswtyTGQW+nUHpKBgHGwUaR6d1kr/LK9+5LpOfRlK9VRfEu7D PGNiRkr8Abp8jHsrBqQWfUS1bAf62bq6XUel0kUCtb7qCq024aOczXYWPFpJFX+nhp4d7NeH Edq+wlC13sBSiSHC7T5yssJ+7JPa2ATLlSKhEvBsLe2TsSTTtFlA0nBclqhfJXzimiuge9qU E40lvMWBuQINBFTKimUBEADDbJ+pQ5M4QBMWkaWImRj7c598xIZ37oKM6rGaSnuB1SVb7YCr Ci2MTwQcrQscA2jm80O8VFqWk+/XsEp62dty47GVwSfdGje/3zv3VTH2KhOCKOq3oPP5ZXWY rz2d2WnTvx++o6lU7HLHDEC3NGLYNLkL1lyVxLhnhvcMxkf1EGA1DboEcMgnJrNB1pGP27ww cSfvdyPGseV+qZZa8kuViDga1oxmnYDxFKMGLxrClqHrRt8geQL1Wj5KFM5hFtGTK4da5lPn wGNd6/CINMeCT2AWZY5ySz7/tSZe5F22vPvVZGoPgQicYWdNc3ap7+7IKP86JNjmec/9RJcz jvrYjJdiqBVldXou72CtDydKVLVSKv8c2wBDJghYZitfYIaL8cTvQfUHRYTfo0n5KKSec8Vo vjDuxmdbOUBA+SkRxqmneP5OxGoZ92VusrwWCjry8HRsNdR+2T+ClDCO6Wpihu4V3CPkQwTy eCuMHPAT0ka5paTwLrnZIxsdfnjUa96T10vzmQgAxpbbiaLvgKJ8+76OPdDnhddyxd2ldYfw RkF5PEGg3mqZnYKNNBtwjvX49SAvgETQvLzQ8IKVgZS0m4z9qHHvtc1BsQnFfe+LJOFjzZr7 CrDNJMqk1JTHYsSi2JcN3vY32WMezXSQ0TzeMK4kdnclSQyp/h23GWod5QARAQABiQRbBBgB AgAmAhsCFiEE+uuXEaEs9HWBLxjyiKkGTRg1YesFAlxd/coFCQtV2mQCKcFdIAQZAQIABgUC VMqKZQAKCRB974EGqvw5DiJoEACLmuiRq9ifvOh5DyBFwRS7gvA14DsGQngmC57EzV0EFcfM XVi1jX5OtwUyUe0Az5r6lHyyHDsDsIpLKBlWrYCeLpUhRR3oy181T7UNxvujGFeTkzvLAOo6 Hs3b8Wv9ARg+7acRYkQRNY7k0GIJ6YZz149tRyRKAy/vSjsaB9Lt0NOd1wf2EQMKwRVELwJD y0AazGn+0PRP7Bua2YbtxaBmhBBDb2tPpwn8U9xdckB4Vlft9lcWNsC/18Gi9bpjd9FSbdH/ sOUI+3ToWYENeoT4IP09wn6EkgWaJS3nAUN/MOycNej2i4Yhy2wDDSKyTAnVkSSSoXk+tK91 HfqtokbDanB8daP+K5LgoiWHzjfWzsxA2jKisI4YCGjrYQzTyGOT6P6u6SEeoEx10865B/zc 8/vN50kncdjYz2naacIDEKQNZlnGLsGkpCbfmfdi3Zg4vuWKNdWr0wGUzDUcpqW0y/lUXna+ 6uyQShX5e4JD2UPuf9WAQ9HtgSAkaDd4O1I2J41sleePzZOVB3DmYgy+ECRJJ5nw3ihdxpgc y/v3lfcJaqiyCv0PF+K/gSOvwhH7CbVqARmptT7yhhxqFdaYWo2Z2ksuKyoKSRMFCXQY5oac uTmyPIT4STFyUQFeqSCWDum/NFNoSKhmItw2Td+4VSJHShRVbg39KNFPZ7mXYAkQiKkGTRg1 YesWJA/+PV3qDUtPNEGwjVvjQqHSbrBy94tu6gJvPHgGPtRDYvxnCaJsmgiC0pGB2KFRsnfl 2zBNBEWF/XwsI081jQE5UO60GKmHTputChLXpVobyuc+lroG2YhknXRBAV969SLnZR4BS/1s Gi046gOXfaKYatve8BiZr5it5Foq3FMPDNgZMit1H9Dk8rkKFfDMRf8EGS/Z+TmyEsIf99H7 TH3n7lco8qO81fSFwkh4pvo2kWRFYTC5vsIVQ+GqVUp+W1DZJHxX8LwWuF1AzUt4MUTtNAvy TXl5EgsmoY9mpNNL7ZnW65oG63nEP5KNiybvuQJzXVxR8eqzOh2Mod4nHg3PE7UCd3DvLNsn GXFRo44WyT/G2lArBtjpkut7bDm0i1nENABy2UgS+1QvdmgNu6aEZxdNthwRjUhuuvCCDMA4 rCDQYyakH2tJNQgkXkeLodBKF4bHiBbuwj0E39S9wmGgg+q4OTnAO/yhQGknle7a7G5xHBwE i0HjnLoJP5jDcoMTabZTIazXmJz3pKM11HYJ5/ZsTIf3ZRJJKIvXJpbmcAPVwTZII6XxiJdh RSSX4Mvd5pL/+5WI6NTdW6DMfigTtdd85fe6PwBNVJL2ZvBfsBJZ5rxg1TOH3KLsYBqBTgW2 glQofxhkJhDEcvjLhe3Y2BlbCWKOmvM8XS9TRt0OwUs= Message-ID: <004289a6-9df9-ae99-53e6-56e555709434@redhat.com> Date: Wed, 10 Apr 2019 13:55:33 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <87sguq2yn1.fsf@dusky.pond.sub.org> Content-Type: text/plain; charset="UTF-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Wed, 10 Apr 2019 17:55:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, qemu-devel@nongnu.org, Max Reitz , Cleber Rosa Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Message-ID: <20190410175533.6hBf77wjN4ob_hoBc8M8j0hAkrl8CJvLYUljhKOEvBw@z> On 4/10/19 1:54 AM, Markus Armbruster wrote: > John Snow writes: > >> Presently we use hxtool and a .hx format to generate a few things like >> the qemu_img subcommand dispatch table, the qemu_img help() display output, >> and a help output in qemu-img.texi. >> >> Unfortunately, this means that this information is duplicated in at least >> three places: >> >> (1) in qemu-img-cmds.hx as a human readable string >> (2) in qemu-img-cmds.hx as a texi string >> (3) in qemu-img.texi again, as a texi string >> >> We can do a little better, though: all of these sources can be generated >> from a single json file. Add a new small tool ("pxtool") that can do this. >> >> This tool can at least handle generating (1) and (2) from the same source >> without needing to reduplicate that information. Deduplicating (3) happens >> in the next patch. >> >> Notes: >> - The json format was chosen to be very "stupid", and the command line >> documentation is being kept one-per-line to make future diffs easier >> to read. >> - It's easy enough to generate the human-readable output from the texi >> output by removing '@var{foo}' with 'foo'. >> - The qemu-img command dispatch always calls img_cmdname, so we don't >> need to store this information separately, either. >> - The need for DEF() macros could be removed as well, but I left it in >> to create a minimally disruptive patch. >> Signed-off-by: John Snow > > We got just five .hx: > > qemu-img.cmds.hx killed off by this patch > qemu-options.hx CLI QAPIfication should kill this off > hw/audio/pl041.hx misnamed, not actually food for hxtool > hmp-commands.hx no exit strategy > hmp-commands-info.hx for these two, yet > > CLI QAPIfication got stuck in the back-burner, and as long as that's the > case, it's not an alternative to your patches. > Good to know. I figure that at least while we wait on something more comprehensive there's no real short term harm in tidying up. Something I'd really like to do is define a python/json-esque configuration file that allows you to specify some "common options" that are shared between all of the sub-commands, and then define each command in terms of both which common options it possesses, and then any local command-specific options it has. (Why the common options design? So that argument names and command options can be encouraged to be identical and identically documented between all subcommands that use them.) Then, from the configuration file, generate all of the necessary C parsers (I have a protoype for this, it's not difficult to do in at least the basic case) that can return boxed command arguments, and then also generate the help strings from that metadata. I suspect that work *IS* something that might brush up against / should use (or extend) QAPI code. Then, I'd like to find a way to split qemu-img.texi into sub-command files and find a way to source the same "informative text" for both: (1) The texi output, as per usual, and (2) qemu-img subcommand --help such that --help, when passed as an argument to the subcommand, prints out help only relevant to the subcommand instead, e.g. `qemu-img check --help` might print the "common options" section only as it relates to commands actually used by the check command, then prints the check section of the texi as formatted for terminals. This way, we can have manpages, html pages, and interactive help text all derived from the same semi-automated sources, always up to date and much easier to read/discover/parse by human eyeballs. That's what I'd like to accomplish, ultimately. For now, I think this RFC set is not harmful and won't bother anybody. It's definitely not worse than what we have now, fragility of removing @var{} tokens and all. --js