Aggregation
Download .mdAggregation
Roll many rows into numbers — counts, sums, percentiles, breakdowns — in a single call. Everything is pushed down to Postgres; you get plain objects back, never SQL.
import { count, percentile, timeBucket } from "@mauroandre/weave-sdk";
const series = await weave.appRequest.aggregate({
where: { host: "api", ts: { gte: since } },
groupBy: { ts: timeBucket("ts", "5min") }, // one row per 5-minute bucket
select: { requests: count(), p95: percentile("durationMs", 0.95) },
orderBy: { ts: "asc" },
});Accumulators
Import the helpers from the SDK and put them in select:
count() // number of rows
sum(field) · avg(field) // over a numeric field
min(field) · max(field)
distinct(field) // count of distinct values
first(field) // one representative per group (earliest by created_at)
percentile(field, p) // exact percentile — p strictly between 0 and 1 (p95 → 0.95)
histogram(field, [bounds]) // bucket counts (see below)Every accumulator takes an optional { where } — a filtered aggregate, computed
in the same pass:
select: {
total: count(),
errors: count({ where: { status: { gte: 500 } } }),
}This inner where is a reduced grammar: scalar operators, and/or/not, and
dot-paths — no some/every/none. It's also the one filter slot that isn't typed
against the entity, so a typo'd field or an unknown operator surfaces at runtime (loudly)
rather than at compile time.
histogram turns N boundaries into N+1 buckets — one per interval plus an overflow
bucket above the last boundary — and returns the counts as one array value:
select: { bars: histogram("durationMs", [100, 200, 500]) }
// → [ <100, 100–200, 200–500, ≥500 ]groupBy — fields or a time bucket
groupBy: ["host", "route"] // by raw fields (aliases match the names)
groupBy: { bucket: timeBucket("ts", "1h") } // by a time bucket — "30s" · "5min" · "1h" · "1d"timeBucket aligns on epoch/UTC, so "1d" buckets don't drift with the session
timezone. Omit groupBy entirely to aggregate the whole set into a single row.
Group by a reference too — it buckets by the target's foreign key. groupBy: ["department", "company"] groups by department_id + company_id; you can name the
reference (department) or its id (departmentId). latestPer takes the reference name.
having and the aggregate's orderBy are different: they address your select
aliases, not fields. having: { revenue: { gte: 100 } } refers to the revenue you
selected — naming a field that isn't an alias is an error.
Grouping through relationships — paths
Any field slot in an aggregate — groupBy, an accumulator's field, a FILTER — takes a
dot-path through what you own and what you reference. Weave joins the tables for you;
you never write the join.
await weave.order.aggregate({
groupBy: ["customer.region"], // reference → a scalar on the target
select: { n: count(), revenue: sum("total") },
});A path steps through a reference (N:1) or an owned object, to any depth —
avg("fulfilment.cost") (an owned object), ["customer.company.tier"] (two hops). A
reference at the leaf buckets by its foreign key. Paths that share a prefix share one
join. In the array form the output alias is the path string itself, so read it back as
row["customer.region"].
Unnesting an owned list — `unnest`
Everything above rolls up parent rows. To roll up the elements of an owned list
instead — an average or a distribution per element, not per parent — name the list in
unnest. The aggregate then runs one row per element (Postgres' answer to Mongo's
$unwind), and groupBy / an accumulator's field / a FILTER address the element's fields:
const perSku = await weave.order.aggregate({
where: { status: "paid" }, // filters the PARENT orders
unnest: "items", // one row per line item
groupBy: ["items.sku"],
select: {
lines: count(),
qty: sum("items.qty"),
backorders: count({ where: { "items.status": { eq: "backordered" } } }), // a band
label: first("items.name"),
},
orderBy: { "items.sku": "asc" },
});The two filters play different roles: where filters the parents (which rows' elements
count at all), while an accumulator's { where } filters the elements (the band —
count(… FILTER …)). first gives one representative per group — the earliest element by
created_at — for metadata that's constant within the group (the label tied to a sku).
With unnest the unit of aggregation is the element, so a parent contributes as many rows
as it has elements. Counting parents under unnest therefore needs distinct("id")
(the parent id repeats per element). One unnest per call — for several lists, run one
aggregate each.
having, orderBy & top-N
having filters groups by their aggregates; orderBy sorts by an output alias;
perPage / page take the top-N:
{
groupBy: ["route"],
select: { n: count() },
having: { n: { gte: 100 } }, // only busy routes
orderBy: { n: "desc" },
perPage: 10, // top 10
}Expressions over aggregates
div · mul · add · sub combine aliases arithmetically, and the result is usable
in orderBy / having — so you filter and sort by a derived rate server-side,
before pagination:
import { count, div } from "@mauroandre/weave-sdk";
await weave.appRequest.aggregate({
groupBy: ["route"],
select: {
errors: count({ where: { status: { gte: 500 } } }),
total: count(),
errorRate: div("errors", "total"), // references the aliases by name
},
orderBy: { errorRate: "desc" }, // the routes that fail most, proportionally
});div guards against divide-by-zero (0 denominators come back as null). An operand
can be an alias, a number, or an inline accumulator (div(count(...), count())).
Facets — many breakdowns in one pass
The dashboard case: headline numbers plus several independent breakdowns of the same filtered set, in one request.
const { rows, facets } = await weave.order.aggregate({
where: { status: "paid" },
select: { revenue: sum("total") },
facets: {
byCategory: { groupBy: ["category"], select: { r: sum("total") }, orderBy: { r: "desc" }, limit: 10 },
byState: { groupBy: ["shipState"], select: { n: count() }, orderBy: { n: "desc" } },
},
});
rows[0].revenue as number; // the headline
facets.byCategory; // [{ category, r }, ...]The values are numbers, the type isn't yet. A numeric accumulator (count, sum,
avg, percentile, an expression, min/max over a numeric column) comes back as a
real number — same as a findMany on the same column, no Number(...) needed. But an
aggregate row is still typed Record<string, unknown>, so annotate at the boundary
(as number) until the select's types are inferred. Group keys are raw JSON — a
timeBucket key is an ISO string, not a Date.
Each facet is its own aggregate under the parent where. The return type follows
your input: with facets, you get { rows, facets }; without, a plain array.
Rolling up over time — `accumulate`
Everything above reads raw rows and aggregates on read — exact, but it keeps every
row. When you can't keep every row forever (telemetry, metrics, counters), roll them up
as they arrive with accumulate. One call folds a data point into a running rollup
keyed by a composite unique — atomically, in Postgres:
import { inc, max, min, setOnInsert } from "@mauroandre/weave-sdk";
await weave.metricRollup.accumulate(
{ workerId, name, ts: bucket }, // the key — a declared unique
{
sampleCount: inc(1), // running counter
cpuSum: inc(cpu), cpuMax: max(cpu), // sum + peak
memSum: inc(mem), memMin: min(mem), // sum + valley
firstSeen: setOnInsert(bucket), // written once, kept forever
},
); // → the resulting rowThe golden rule: store what merges, derive the rest on read. Keep sum and
count; compute the average when you read (cpuSum / sampleCount). Never store an
average — two averages can't be merged, a sum and a count always can.
The ops
inc(n) // col = col + n — counters, sums (monotonic)
max(v) · min(v) // col = greatest/least — peaks and valleys
setOnInsert(v) // written on insert only — preserved on every later mergeinc/max/min merge in the database (+, greatest, least); setOnInsert
writes on the first insert and is left untouched afterwards. There is no
read-modify-write and no race — the whole thing is a single upsert.
The key must be a declared unique
accumulate upserts on the key, so the key has to be a unique the entity declares —
a composite group (the rollup key) or a single .unique() column:
export default defineEntity(
"metricRollup",
{
workerId: text().notNull(),
name: text().notNull(),
ts: timestamptz().notNull(),
sampleCount: int4().notNull().default(0),
cpuSum: float8().notNull().default(0),
cpuMax: float8().notNull().default(0),
},
{ unique: [["workerId", "name", "ts"]] }, // ← the ON-CONFLICT key
);A key that doesn't match a declared unique is a clear error — nothing is written. Read
the rollups back with the ordinary aggregators (sum, avg over the buckets), or plain
findMany / latestPer for the raw rollup rows.
Batch ingest — createMany
Insert many rows in one transaction — the shape a batched producer wants:
await weave.appRequest.createMany([
{ host: "api", route: "/x", durationMs: 12, status: 200 },
{ host: "api", route: "/y", durationMs: 40, status: 500 },
]); // → the created rows, in input orderRetention — partition by time
A high-volume event table (requests, logs, audit trails) can't grow forever. Declare
partitionBy + retention and Weave keeps a rolling window for you — natively, with
zero maintenance on your side:
export default defineEntity(
"appRequest",
{ host: text().notNull(), route: text().notNull(), ts: timestamptz().notNull(), status: int4().notNull() },
{ partitionBy: timeBucket("ts", "1d"), retention: "30d" }, // daily partitions, keep 30 days
);Under the hood the table is RANGE-partitioned by ts. On each write Weave lazily
creates the partition the incoming row falls into (so a late/backfilled batch still lands
correctly), and once a new day opens it drops whole partitions past the retention
window — a DROP TABLE, not a row-by-row DELETE that would bloat under load. You never
run a cron or a cleanup job; it's internal to Weave. Reads (findMany, aggregate) span
all partitions transparently — Postgres prunes by the ts predicate.
Things to know:
- Append-only. The partition key rides in the primary key (
(id, ts)), so a partitioned entity is insert-only — ingest withcreateMany, neverupdateOne. That's exactly what a raw event tier wants. - Past the window is skipped. A row whose
tsis already older thanretentionis dropped on ingest (its partition is gone) —createManyskips it and logs the count; a singlecreateof such a row is a clear error. - Partitions materialize on first write. A brand-new partitioned table has no
partitions until the first row lands — the day's partition is created then, lazily. An
empty table showing zero partitions is expected, not a problem. Insert through Weave
(not a raw SQL
INSERT), so it can create the partition the row needs. - Buckets align to UTC. A
"1d"bucket is a UTC day, not a local-timezone day — the same boundarytimeBucket("ts", "1d")uses inaggregate, so the raw partitioned tier and your rollups agree on what "a day" is. In practice a late-local-evening event can land in the next UTC day's partition; that's intended.
The partition field must be a timestamptz().notNull(). This is a general time-series
capability — logs, metrics, events — not tied to any one domain.
Latest per group — `latestPer`
Greatest-n-per-group: one row per key, the latest wins. It feeds live-metrics widgets (“the most recent doc per worker”). You declare the group; Weave arranges the ordering:
await weave.worker.findMany(
{},
{ latestPer: ["workerId"], orderBy: { ts: "desc" } }, // newest row per worker
);Aggregation respects scopes just like reads — the scope's row filter is AND-ed into
the aggregate (and every facet) before anything is grouped.