Data Authority Limitations

Known constraints and edge cases of the Data Authority query system. Companion to Data Authority Module Pattern and Query DSL.

Last verified against common-module on 2026-08-17, at release 16.0.0.

Current Limitations

Query Schema Validation

Filter keys are validated at execution time, not at construction time. If a client sends a filter referencing a non-existent column, the error is raised when the FilterExposedVisitor attempts to resolve the locator against the table schema. Clients receive an ArgumentValidation error with the message "Column with name '<name>' not found in table '<table>'".

Impact: Malformed queries are not rejected early; debugging requires inspecting the full error response.

Field Projections

All REST endpoints return complete EntityRecord<EP, EM> objects. There is no mechanism for clients to request a subset of fields. This means:

• Payloads are always full-size, regardless of what the client needs
• List views that display only a few columns still transfer entire entity records
• No fields parameter is available on query endpoints

General Aggregation API Exposure

The persistence layer (Universe) supports COUNT, SUM, AVERAGE, MIN and MAX with GROUP BY, and these are tested. Only one shape of aggregation reaches the REST surface: the distinct-values read (see below). There is no general aggregation endpoint, and Query carries no aggregation field.

Workaround: other aggregations are performed server-side within the service layer, not requested by API consumers.

Requested Page Size Is Bounded

POST /v1/<entity>/query and the history route refuse a paginate.size above 1000 rows, and refuse a negative paginate.index. The refusal names the offending field.

An oversize request is refused rather than served short, deliberately: a page that comes back not-full is how a client learns it has reached the end of the data, so silently reducing the page would announce itself as the last one and the client would stop early.

The bound applies where a request arrives from outside. In-process reads sized to a computed set — “fetch exactly these 600 items” — are bounded by their own callers and are not affected.

Sorting on Calculated Fields

Sorting supports direct table columns and nested component property paths (via ExposedLocatorTranslator). It does not support:

• Sorting by computed expressions (e.g., price * quantity)
• Sorting by derived fields not stored in the database
• Collection quantifiers in sort keys (e.g., lines[any].status)

Resolved Limitations

These items were limitations in earlier versions but have since been addressed:

Page Tokens (Resolved)

Page tokens are now full Query state objects: GZIP-compressed, Base64-encoded JSON serializations of the complete Query (filter + sort + pagination). They are properly decoded with error handling for malformed or tampered tokens. No longer placeholder implementations.

Full-Text Search (Resolved)

Free-text search is served by POST /v1/<entity>/lookup against the generated search_text column and its GIN trigram index, delivered by PDEV-1251. It provides fuzzy matching and relevance ordering across the indexed document, which the structured filter terms deliberately do not attempt.

Note the consequence for ordering: a lookup answers in relevance order, so a caller combining search with a column sort gets relevance, and a UI should say so.

Distinct Values Over a Filtered Set (Resolved)

POST {resource}/distinct and GET {resource}/distinct/{pageId} answer “what values does this field actually hold?” over the latest, non-retired, tenant-scoped entities — the question a filter option list asks. It is a classifier-only aggregation, which is why it needed no new SQL: a grouped read with no aggregators already produces exactly this.

Four constraints on the read are deliberate and not negotiable, and all four protect one guarantee — that every value returned round-trips into an equality filter matching the rows it came from:

• absent values are excluded in the filter rather than sifted out of the result;
• the caller’s sort is replaced by the value ascending;
• a field resolving to a non-text column is refused rather than rendered through toString();
• values are never folded or trimmed, so Acme, acme and Acme are three entries.

Each value can optionally carry the entities holding it. That is off by default, capped per value rather than across the response, and an entry whose members were capped says so — a partial list that cannot be told from a complete one is the failure this exists to avoid.

OR and NOT Filters (Resolved)

The Filter sealed interface supports Filter.Or (composite clauses) and Filter.Not (transform) operators. Both are translated to SQL via FilterExposedVisitor.visitComposite() and visitTransform(). Clients can express arbitrary boolean logic in filter trees.

Potential Enhancements

These are documented directions for future work, not committed roadmap items:

Enhancement	Benefit	Complexity
Upfront query schema validation	Fail-fast on malformed queries before execution	Low
Expose general aggregation via REST	Client-side analytics without full data transfer	Medium
Field projections	Reduced payload size for list views	Medium
Calculated sort expressions	Sort by derived values	Medium
Async query execution for exports	Handle very large result sets without timeouts	High