[diamon-discuss] CTF2-PMETA-1.0: CTF 2 packetized metadata stream format

[Philippe Proulx <pproulx@efficios.com>]

HTML version:

    http://diamon.org/ctf/files/CTF2-PMETA-1.0.html

Philippe Proulx
EfficiOS Inc.
http://www.efficios.com/

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

= CTF2-PMETA-1.0: CTF{nbsp}2 packetized metadata stream format
Philippe Proulx <pproulx@efficios.com>
v1.0, 21 October 2016
:toc:
:toclevels: 5
:docid: did:CTF2-PMETA-1.0

This document defines a standard format for wrapping a CTF{nbsp}2
metadata stream (see did:CTF2-SPEC-2.0) into one or more packets.


[[compliance]]
== Compliance

A CTF{nbsp}2 producer is said to _apply {docid}_ if it can encode a
CTF{nbsp}2 metadata stream as a packetized metadata stream as
defined by this document.

A CTF{nbsp}2 consumer is said to _support {docid}_ if it can decode a
packetized metadata stream, as defined by this document, as a CTF{nbsp}2
metadata stream.


== Packetized metadata stream layout

A packetized metadata stream is a sequence of metadata packets:

    [metadata packet]
    [metadata packet]
    [metadata packet]
    [metadata packet]
    ...

The layout of a metadata packet is:

    [metadata packet header]
    [metadata packet content]


== Metadata packet header format

The fields of a metadata packet header are, in this order, and without
any padding between the fields:

[options="header"]
.Metadata packet header fields
|===
|Name |Offset in metadata packet header (bits) |Type |Notes

|Magic number
|0
|32-bit, unsigned integer, any byte order.
|The field's value _must_ be 1976638807 (0x75d11d57), in any byte
order (little-endian or big-endian). The byte order of this field
indicates the byte order of the following fields, when needed.

|Metadata stream's UUID.
|32
|16 bytes.
|This is a UUID assigned to _all_ the metadata packets wrapping a
given metadata stream.

|Metadata packet's checksum.
|160
|32-bit, unsigned integer, deduced byte order.
|Producers and consumers _must_ ignore this field as of {docid}.

|Metadata packet's content size (bits)
|192
|32-bit, unsigned integer, deduced byte order.
|This is the content size, in bits, of the metadata packet, including
this header.

The value of this field _must_ be a multiple of 8.

|Metadata packet's total size (bits)
|224
|32-bit, unsigned integer, deduced byte order.
|This is the total size, in bits, of the whole metadata packet,
including this header.

The difference between this total size and the packet's content size
is the padding size.

The value of this field _must_ be a multiple of 8.

|Content's compression scheme
|256
|8-bit, unsigned integer
|As of {docid}, this field _must_ be set to 0 (means no compression).

|Content's encryption scheme
|264
|8-bit, unsigned integer
|As of {docid}, this field _must_ be set to 0 (means no encryption).

|Content's checksum scheme
|272
|8-bit, unsigned integer
|As of {docid}, this field _must_ be set to 0 (means no checksum).

|Metadata packet's major version number
|280
|8-bit, unsigned integer
|As of {docid}, this field _must_ be set to 2.

|Metadata packet's minor version number
|288
|8-bit, unsigned integer
|As of {docid}, this field _must_ be set to 0.

|Reserved
|296
|24-bit, unsigned integer
|Producers and consumers _must_ ignore this field as of {docid}.

|Metadata packet header's size (bits)
|320
|32-bit, unsigned integer, deduced byte order.
|Size of the whole metadata packet header, including this field and the
following fields, if any.

Any unknown field of which the offset relative to the beginning of the
metadata packet, in bits, is greater than the value of this field,
_must_ be ignored by consumers.

As of {docid}, this field _must_ be set to 352 (44 bytes).
|===

Note that all the fields from the metadata packet's magic number to the
metadata packet's minor version number (inclusive) are compatible with
the metadata packet header format of _Common Trace Format v1.8.2_,
section 7.1, with the following exceptions:

* The metadata packet's checksum field _must_ be ignored.
* The content's compression scheme field _must_ be set to 0.
* The content's encryption scheme field _must_ be set to 0.
* The content's checksum scheme field _must_ be set to 0.
* The content's checksum scheme field _must_ be set to 0.
* The metadata packet's major version number field _must_ be set to 2.
* The metadata packet's minor version number field _must_ be set to 0.


== Metadata packet content

The concatenated content bytes of all the metadata packets of a
packetized metadata stream form the raw, JSON metadata stream of the
CTF{nbsp}2 trace as described by did:CTF2-SPEC-2.0.