Origin Annotations

A configuration value has a history. The value comes from a file, and that file can itself come from a different file. The chain can go back through a code generator, a register list, an interface description or an export from a spreadsheet. Those files are often not JSON, and Confµgami never reads most of them.

Origin annotations record that chain. A tool can then show the file, the line and the column for a value, follow the chain back to the first source that a person wrote, and find out that a location in the chain is no longer correct.

A merge is one link in that chain and not the whole chain. More than one file can give a value, and the strongest file wins. An annotation keeps each of those files, and the sources behind each of them.

This section is the specification of that format.

Format Version 1 holds the whole specification: the $originSpec header, the file table, the grammar and the sequence of more than one origin. It also gives the line anchor scheme a1, which defines the fingerprint inside an annotation. A consumer uses that fingerprint to check a location, and to find the line again after the line moved.

The format and the anchor scheme have one version number together, and version 1 of the format makes a1 the anchor scheme.

Requirement Words

Interpret the keywords MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY and OPTIONAL as RFC 2119 describes them.

Design Goals

The format is a set of compromises between four goals. The goals are in order of importance. Most parts of the specification that look strange are the result of one of these goals.

Machine independent: The format has no values that are specific to one machine, so a consumer can resolve a file that a different machine made. A path is relative to the file that holds it, so a consumer that has that file needs nothing else. There is one exception, and it is a last resort: a file on a different Windows drive has no relative path, so its entry holds an absolute path and does not travel to a different machine.
Stable in a diff: A small change to a source file MUST NOT make a large change to the export file. If an entity moves 300 lines down its source file, only one number changes. The annotations below it do not change.
Small: The annotations are an overhead. The format keeps one entry short, because an export can hold a large number of them.
Verifiable: A consumer can find out that a location is no longer correct. The consumer does this without a new run of the generator.

When Origins Are Useful

A consumer that can run the producer again on the same source files gets little value from this format. A check of a location is then a new run and a comparison of the two results.

Origins are for the other situation. The file goes to a tool that cannot run the producer, the first source in the chain is often not available to that tool, and persons continue to edit the source files. That is also why an anchor is REQUIRED on each traced annotation.

Scope

The format is independent of the producer and of the file type of a source. A source in the chain can be a .cfg.json5 file, or a file of any other type. Any tool that writes a .cfg.json5 file can also write a $originSpec header and an _origin annotation into it. Confµgami reads such an annotation, keeps it, and puts it into the chain. The export of Confµgami is therefore one producer among others.

The --with-origin option of cfu cfg export-merged controls the traced origins in the export of Confµgami. Without the option, the export writes no traced origin. It keeps an _origin annotation that an author or a different tool wrote into an input file, because such an annotation is data of that file and not a result of the trace. The export therefore writes the $originSpec header as soon as one _origin annotation reaches the output, with the option and without it.

These items are outside the scope:

The generated Rust and Python code has no origins of its own.
The generated JSON Schemas permit _origin and $originSpec in an input file. The schemas have no origins of their own.

The Elements That Have an Origin

The _origin annotation holds an origin. It is a public annotation type in the built-in Confugami.def.json5 file, and its target is All, so it can occur at each position where an annotation can occur: entities, structs, unions, parameter objects and the root object of a file. The root object is the position for the origin of a complete file, which is what a generated file records.

_origin is therefore an annotation and not a special mechanism, and two results come from that.

An author can write _origin in an input file, to record that a different source made the file. Such an entry is a transitive origin, and the export keeps both the transitive origin and the traced origins.
A scalar value cannot have an origin, so the export changes a scalar parameter into its object form { _origin: …, value: … }. An element inside a scalar list cannot have an origin.

Limits

A scalar value cannot have an origin: There is no position for the annotation, so the element must first become an object.
A change to a source file also changes the data area: The annotations are inside the data, so a change to a source file does not keep the data area equal, byte for byte. This is a result of the inline format.
The export file repeats the same location: Two hundred properties can come from one line of a template, and the export file then has two hundred annotations for that line.
A change to a parent changes the deltas below it: A parent can get an origin for a file, or lose one, and the export must then calculate the deltas of all its children for that file again. The effect stops at the limits of one subtree, and it is the largest cause of changes in the format.