Re: [RFC] yamldt v0.5, now a DTS compiler too

[Grant Likely <grant.likely@secretlab.ca>]

On Sun, Oct 22, 2017 at 7:23 PM, Pantelis Antoniou
<pantelis.antoniou@konsulko.com> wrote:
> Hi Grant,
>
>> On Oct 22, 2017, at 19:54 , Grant Likely <grant.likely@secretlab.ca> wrote:
>>
>> On Fri, Oct 20, 2017 at 8:16 PM, Pantelis Antoniou
>> <pantelis.antoniou@konsulko.com> wrote:
>>> Hi Grant,
>>>
>>>> On Oct 20, 2017, at 20:46 , Grant Likely <grant.likely@secretlab.ca> wrote:
>>>>
>>>> On Thu, Sep 28, 2017 at 8:58 PM, Pantelis Antoniou
>>>> <pantelis.antoniou@konsulko.com> wrote:
[...]
>>> Yes. This is expected. I don’t think pure YAML form is a good match for all
>>> the crazy things that are possible (and actually used in practice) with DTS.
>>
>> I don’t think it is as dire as that. The DTS structure is not complex
>> and I think can easily be mapped into pure YAML. But, it does require
>> treating the DTS structure separately from its semantic meaning. For
>> example, the grammar of nodes, properties and labels easily maps to
>> pure YAML, but phandles and overlay trees have semantic meaning that
>> must be resolved by DT specific code. I’ll respond to you’re specific
>> examples below…
>>
>
> We are in complete agreement here. Single nodes and properties map to YAML perfectly.
> It’s the complex way that we build the complete DTB that stress the YAML structures.

Okay, good.

>>> If we were to force YAML/DT file to be regular YAML files parseable by everything
>>> we’d have to rethink a lot of conventions that DT files currently use and I’m afraid
>>> a machine translation from DTS to YAML might not be feasible then.
>>>
>>> OTOH moving to that model might make it possible to use YAML constructs that are not
>>> mapped at all to DTS. For instance
>>>
>>> - foo: true
>>>  bar: “string”
>>>
>>> - foo: false
>>>  bar: “another-string”
>>>
>>> is not possible to be mapped to a DT node/property structure right now.
>>
>> I’m not following. Are you saying DT has no way to encode a list of
>> nodes? What use case are you imagining?
>>
>
> This threw me off too at first.
>
> The problem is when you try to convert this into a live tree structure.
>
> The YAML encoding is
>
> +SEQ
>  +MAP
>   =VAL :foo
>   =VAL :true
>   =VAL :bar
>   =VAL "string
>  -MAP
>  +MAP
>   =VAL :foo
>   =VAL :false
>   =VAL :bar
>   =VAL "another-string
>  -MAP
> -SEQ
>
> So it’s a sequence of MAPs.
>
> In a live tree DTS form would be
>
> noname-property = { foo=true; bar = “string”; }, { foo = false; bar=“another-string”; };
>
> We don’t have the capability as right now to express something like this.
>
> Namely properties containing nodes, and the root of the live tree not being a node.

Ah, so you're proposing property data that encodes key/value pairs,
correct? Effectively adding another layer of namespace under a
property, but that namespace is invisible to and generic DT parsing
code in an OS (for how DT is defined today).

I would straight out make that structure illegal in the YAML DT
metaschema until such time as the semantics are defined. For now
properties should only be allowed to contain scalars, or lists of
scalars/lists which can be nested.

[...]

>>>> The encoding differentiates between nodes and properties implicitly
>>>> base on whether the contents are a map, or a scalar/list. This does
>>>> mean any parser needs to do a bit more work and it may impact what can
>>>> be done with validation (I'm not going to talk about validation in
>>>> this email though. We'll talk next week.)
>>>
>>> Hmm, I’m not sure I follow this. Is that related to the metadata problem.
>>
>> It has to do with iterating over properties or nodes. Nodes and props
>> are mixed together, and any code processing them has to explicitly
>> check the type to differentiate between the two. The alternative would
>> be to collect all properties under a single key:
>>
>> parent:
>>  $props:
>>    foo: hello
>>    bar: world
>>  child1: {}
>>  child2: {}
>>
>
> That’s no problem. It does look weird though. I am using the same parser
> for parsing bindings and this might make the files a little bit weird.

That it does. I don’t feel strongly about either approach, but I want
to see how the different encodings affect doing the schema bits.

>
> Keep in mind that we have two YAML formats to parse; the hardware description
> and the bindings describing them.

Three! Hw description, schema & metaschema. :-) But metaschema should
be a normal jsonschema file.

>
>>>
>>> BTW, a blog post explaining the rationale behind yamldt is going to come
>>> up soon at our website, I’ll post the link when it does.
>>
>> Cool. I see it got posted, but didn’t get a chance to read it before I
>> got on this airplane. Will try to read this evening.
>>
>> (BTW, I’m arriving at about 11:00 tonight)
>>
>
> Have a safe flight, you’ll get to beat me up tomorrow :)

See you tomorrow. It will be good to catch up.

g.