Re: [Qemu-devel] [RFC PATCH 01/16] doc: Add QBM format specification

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

On 26.01.2016 13:38, Fam Zheng wrote:
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>   docs/specs/qbm.md | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 118 insertions(+)
>   create mode 100644 docs/specs/qbm.md
>
> diff --git a/docs/specs/qbm.md b/docs/specs/qbm.md
> new file mode 100644
> index 0000000..b91910b
> --- /dev/null
> +++ b/docs/specs/qbm.md
> @@ -0,0 +1,118 @@
> +QEMU Block Bitmap (QBM)
> +=======================
> +
> +QBM is a multi-file disk format to allow storing persistent block bitmaps along
> +with the tracked data image.  A QBM image includes one json descriptor file,
> +one data image, one or more bitmap files that describe the block dirty status
> +of the data image.
> +
> +The json file describes the structure of the image. The structure of the json
> +descriptor file is:
> +
> +    QBM-JSON-FILE := { "QBM": DESC-JSON }
> +
> +    DESC-JSON := { "version": 1,
> +                   "image": IMAGE,
> +                   "BITMAPS": BITMAPS
> +                 }
> +
> +Fields in the top level json dictionary are:
> +
> +@version: An integer which must be 1.
> +@image: A dictionary in IMAGE schema, as described later. It provides the
> +        information of the data image where user data is stored. Its format is
> +        documented in the "IMAGE schema" section.

I think,  ', as described later' or entire third sentence may be dropped 
as duplication.

> +@bitmaps: A dictionary that describes one ore more bitmap files. The keys into
> +          the dictionary are the names of bitmap, which must be strings, and
> +          each value is a dictionary describing the information of the bitmap,
> +          as documented below in the "BITMAP schema" section.

Agree with others, that it should be an array. It seems like it is more 
common practice.

> +
> +=== IMAGE schema ===
> +
> +An IMAGE records the information of an image (such as a data image or a backing
> +file). It has following fields:
> +
> +@file: The file name string of the referenced image. If it's a relative path,
> +       the file should be found relative to the descriptor file's
> +       location.
> +@format: The format string of the file.
> +
> +=== BITMAP schema ===
> +
> +A BITMAP dictionary records the information of a bitmap (such as a dirty bitmap
> +or a block allocation status bitmap). It has following mandatory fields:
> +
> +@file: The name of the bitmap file. The bitmap file is in little endian, both
> +       byte-order-wise and bit-order-wise, which means the LSB in the byte 0
> +       corresponds to the first sectors.

What about reuse my bitmap-table approach (one indexing table, like L1), 
to save space? Or, add @format field, to allow adding this later as 
native option, without ext-hard-.

> +@granularity-bytes: How many bytes of data does one bit in the bitmap track.
> +                    This value must be a power of 2 and no less than 512.
> +@type: The type of the bitmap.  Currently only "dirty" and "allocation" are
> +       supported.
> +       "dirty" indicates a block dirty bitmap; "allocation" indicates a
> +       allocation status bitmap. There must be at most one "allocation" bitmap.
> +
> +If the type of the bitmap is "allocation", an extra field "backing" is also
> +accepted:
> +
> +@backing: a dictionary as specified in the IMAGE schema. It can be used to
> +          adding a backing file to raw image.
> +
> +
> +=== Extended fields ===
> +
> +Implementations are allowed to extend the format schema by inserting additinoal
> +members into above dictionaries, with key names that starts with either
> +an "ext-hard-" or an "ext-soft-" prefix.
> +
> +Extended fields prefixed with "ext-soft-" are optional and can be ignored by
> +parsers if they do not support it; fields starting with "ext-hard-" are
> +mandatory and cannot be ignored, a parser should not proceed parsing the image
> +if it does not support it.
> +
> +It is strongly recommended that the application names are also included in the
> +extention name string, such as "ext-hard-qemu-", if the effect or
> +interpretation of the field is local to a specific application.
> +
> +For example, QEMU can implement a "checksum" feature to make sure no files
> +referred to by the json descriptor are modified inconsistently, by adding
> +"ext-soft-qemu-checksum" fields in "image" and "bitmaps" descriptions, like in
> +the json text found below.
> +
> +=== QBM descriptor file example ===
> +
> +This is the content of a QBM image's json descriptor file, which contains a
> +data image (data.img), and three bitmaps, out of which the "allocation" bitmap
> +associates a backing file to this image (base.img).
> +
> +{ "QBM": {
> +    "version": 1,
> +    "image": {
> +        "file": "data.img",
> +        "format": "raw"
> +        "ext-soft-qemu-checksum": "9eff24b72bd693cc8aa3e887141b96f8",
> +    },
> +    "bitmaps": {
> +        "0": {
> +            "file": "bitmap0.bin",
> +            "granularity-bytes": 512,
> +            "type": "dirty"
> +        },
> +        "1": {
> +            "file": "bitmap1.bin",
> +            "granularity-bytes": 4096,
> +            "type": "dirty"
> +        },
> +        "2": {
> +            "file": "bitmap3.bin",
> +            "granularity-bytes": 4096,
> +            "type": "allocation"
> +            "backing": {
> +                "file": "base.img",
> +                "format": "raw"
> +                "ext-soft-qemu-checksum": "fcad1f672b2fb19948405e7a1a18c2a7",
> +            },
> +        }
> +    }
> +} }
> +


-- 
Best regards,
Vladimir