AQL sub-queries (somewhat like SQL sub-select)

Frequent developer feedback around data queries is that they want to enrich data with secondary information, for example for display on a dashboard or export to an external system. An example is that for each produced piece there is an event written when it is finished, and the record to export needs to contain information from the youngest preceding article definition event.

AQL sub-queries

The idea behind AQL is that it can serve as selector and transformation of historical as well as live event streams. To avoid an explosion of needed stream state, sub-queries are added such that they do not turn into live subscriptions by themselves. Instead, a sub-query is evaluated as a static query limited by the currently known present. We will allow this only on the /query endpoint for now, the extension to /subscribe will then need to record which event ranges may later cause time travel (possibly with bounded space dedicated to optimising live dissemination time travel over the past few seconds).

The syntax for sub-queries is very simple: a complete FROM ... END query is allowed wherever a value can be computed.

If the last step of the sub-query is an AGGREGATE, then a single value is produced. Otherwise, all produced values are wrapped in an array (which may be empty).

Example usage

The example mentioned in the introduction could be modelled like this:

FROM 'workpiece' -- assuming this tag on all events pertaining to the workpieces
FILTER _.type = 'finished'
    id: _.serial
    article: _.article
        FROM 'article-definition' & `article:{_.article}` & to(_) -- here _ is still the workpiece event
        AGGREGATE LAST(_.description)