Low-level extension points. For adding new base-fns / fn-defs to an
existing package, see PACKAGES.md — that’s the common
case and the package system (resources/packages/) already handles
registration, sync, and ordering.
This document covers:
A :fn-typed slot in an fns.edn declaration tells the compiler
“this slot receives a callable”. The slot’s type-fn-id resolves to
the :fn primitive (overlayable per-fn via
binding.type-override-fn-id); at compile-time, refs bound to such
slots go through hof-wrap instead of the normal thunk path. The
legacy :is-fn boolean was retired — type IS the marker.
The compiler first determines which free arguments the HOF supplies on
each call and which are captured from the surrounding graph. Authored
:lambda-params takes precedence; otherwise the slot’s structural
[:fn {ARGS} RET] signature determines the call-site shape. Structural
names participate in that matching. Ambiguous positional cases require an
explicit declaration rather than guessing from all leftover free args.
Bare :fn slots use the fallback capture analysis. See
CLOSURE_CAPTURE.md and
executor.compile.renames/hof-lambda-params.
After that selection, hof-wrap uses the call-site parameter count:
| Call-site parameters | Callable shape | Impl invokes as |
|---|---|---|
| 0 | variadic, ignores input; free args are captured | (f) or (f anything) |
| 1 | single-arg | (f value) — value fills that parameter |
| ≥ 2 | map-callable | (f {:name1 v1 :name2 v2 …}) — keys match the call-site parameters |
A callback can therefore have several free args while receiving only one argument per invocation: the remaining args are captured at wrap time.
map (one call-site parameter);; In fns.edn
{:name :map
:args {:func {:type [:fn {:item a} b]}
:coll {:type [:list a]}}
:return-type [:list b]}
;; In impls.clj
(defbase map-fn [func coll]
(doall (map func coll))) ; func is called (func item) per element; SEQ, not mapv→vector
A user binds a callback as :func; its call-site parameter receives each
item, while other free arguments are captured. A differently named unary
parameter is accepted positionally when capture analysis is unambiguous.
reduce (one parameter via vector packing)(defbase reduce-fn [func init coll]
(reduce (fn [acc item] (func [acc item])) init coll))
Impl packs [acc item] as a single value. The user’s fn-graph is
unary (leftover = :pair or similar) and destructures via
:get :key 0 / :key 1 inside.
middleware (two call-site parameters)(defbase middleware [name body]
{:name name
:wrap (fn [handler]
(fn [request]
;; body is map-callable: two free args :request + :next-handler
(body {:request request :next-handler handler})))})
The body fn-graph declares {:as :request} and {:as :next-handler}
renames somewhere; those match the slot’s two structural call-site
parameters. The compiler produces a map-callable that the impl populates;
additional free arguments are captured.
make-single-arg-callable — raw fn-id entry pointWhen a :fn-typed arg arrives as a raw fn-id UUID (e.g. in test code
that doesn’t go through hof-wrap), use
exec/make-single-arg-callable to get the same behaviour as the
compile-time wrap.
Implement these protocols from graphden.storage.protocol.core to
create a new backend:
| Protocol | Purpose |
|---|---|
Storage |
initialize, close — lifecycle |
StorageIntrospection |
current-entities, current-enums |
StorageCRUD |
create-entity, read-entity, update-entity, delete-entity, query-entities |
GraphConstraints |
validate-no-dependency-cycle! |
ExecutionGraph |
resolve-execution-graph |
(ns my.custom-storage
(:require [graphden.storage.protocol.core :as sp]))
(defrecord CustomStorage [connection metadata]
sp/Storage
(initialize [this schema] this)
(close [this] nil)
sp/StorageCRUD
(create-entity [this entity-name data] …)
(read-entity [this entity-name id] …)
;; …
sp/ExecutionGraph
(resolve-execution-graph [this fn-id] …))
(defn create-storage [opts]
(->CustomStorage (:connection opts) (atom {})))
(require '[graphden.storage.protocol.contract-tests :as contract]
'[graphden.storage.protocol.core :as sp])
(contract/run-graph-constraints-tests create-my-storage sp/close)
A backend that only needs to SERVE a graph — compile fn-defs and execute them, without authoring — implements far less than a full read/write backend. The executor’s compile + execute path calls only:
StorageCRUD/query-entities — with {}, {:name v}, {:id [ids]},
{:fn-id [ids]} shapes (equality on scalars, membership on vectors);StorageCRUD/read-entity and StorageBatchCRUD/read-entities;ExecutionGraph to pass
executor.context/create-context’s satisfies? gate — the method is
never called (the compiled registry reads the tables and does the BFS
in-process), so it can throw.Everything else (writes, GraphConstraints, StorageIntrospection,
StorageValueCodec) is only reached by the authoring / schema paths.
graphden.storage.remote.core/RemoteStorage is exactly this: a read-only
leaf that bootstraps the whole graph over HTTP
(GET /api/export/graph-rows, org-scoped) into an in-memory index and
answers the reads from there. It backs the external / BYO executor — see
SCALING.md § external executor. Its writes throw
:remote-storage/read-only; the FaaS app path (app_router → cr/execute)
is read-only and works, while the /api/execute persistence path is a
hosted-editor concern.
Five entity types — see PHILOSOPHY.md for rationale and CLAUDE.md for the up-to-date field list.
| Entity | Notes |
|---|---|
fn |
function or type-row; parent-ids ref-many for inheritance |
slot |
atomic (name, type-fn-id); immutable post-create |
fn-slot |
junction (fn-id, slot-id, position) |
binding |
per-(fn, slot) overlay (value, ref-fn-id, type-override-fn-id, terminal, list-{append,closed}, description, resolver-fn-id) |
binding-list-item |
sequence content under a list-typed binding |
A composed fn carries parent-ids and a binding shadows the
inherited slot:
;; Base fn `add` has slot `a` (type :int)
(sp/create-entity storage :fn
{:name "add" :parent-ids [] :return-type-fn-id int-fn-id})
(let [slot-a (sp/create-entity storage :slot
{:name "a" :type-fn-id int-fn-id})]
(sp/create-entity storage :fn-slot
{:fn-id add-id :slot-id (:id slot-a) :position 0})
;; Composed fn that binds a=5
(let [add-5 (sp/create-entity storage :fn
{:name "add-5" :parent-ids [add-id]})]
(sp/create-entity storage :binding
{:fn-id (:id add-5) :slot-id (:id slot-a) :value 5})))
(ns my.extended-schema
(:require [graphden.schema.graph.schema :as graph]
[graphden.schema.protocol.protocol :as ds]))
(defn build-extended-schema [builder]
(-> (graph/extend-builder builder)
(ds/add-entity :my-entity
{:id {:type :uuid :primary-key true}
:name {:type :text :nullable false}
:data {:type :jsonb :nullable true}})))
:ref column — :indexed? true:ref fields get a btree index automatically (via create-ref-indexes!
on entity creation). To index a NON-:ref column — e.g. a plain
:uuid you filter reverse-references on — declare :indexed? true on
the field spec:
:ref-fn-id {:type :uuid :indexed? true}
Two callbacks act on the flag:
create-field-indexes! (postgres ddl) — issues
CREATE INDEX IF NOT EXISTS for every :indexed? field when the
table is created.ensure-field-indexes! (postgres migration) — runs on every
migration pass, so a flag newly added to a field on an already-created
table lands on existing dev/prod DBs (the entity-create callback never
re-fires for them). It is idempotent (IF NOT EXISTS).The schema validators accept :indexed? on any non-variant field spec.
Use it instead of a hand-written CREATE INDEX migration so the index
is declarative and travels with the schema.