aqmen
← All docs

Transformations

SQL that derives a new dataset, the method stated once, re-run when its inputs change.

Transformations

A transformation is SQL that derives a new dataset from others: a driver adapted to a model's grain, a join that feeds a model, a mapping applied. The SQL is the method, stated once; the docs say what was done and why.

Writing one

Transformations → New: name it, write a SELECT over your datasets (the dataset names autocomplete), run it, save. The transformation and its output are one artifact on one page: the result rows, the schema and the SQL that produced them.

Staleness

When an input dataset gets a new version, the transformation is marked out of date: its output no longer reflects the current data. Re-run it to refresh. The Lineage page can run the whole pipeline in dependency order, upstream first, so one pass leaves everything fresh.

What to use it for

Deriving, not cleaning. Aggregate a driver weighted by its base, allocate a total by a sourced key, interpolate between survey years, join the drivers a model needs into one feed. Each of these is a method worth naming, and a transformation names it.

Faster with an agent — describe the table you want: "adapt the adoption survey to yearly values, interpolating between survey years, and say so in the docs." The agent writes the SQL, runs it, documents the method, and you review the result.