graphden’s existing HOF wrapping (hof-wrap in
graphden.executor.compile-eager) treats every free-arg of a
wrapped fn-graph as a call-site arg — the executor passes the
arg’s value when invoking the callable. That works perfectly for
data-flow HOFs like :map, :filter, :reduce: the callable
receives one arg per element / accumulator and pure-functionally
produces a result.
It does NOT work for supervised long-running patterns like cron. Concretely:
;; Hypothetical pure-composition for cron schedule:
{:name :schedule
:parent :future ; ← :body is HOF-wrapped
:args {:body :_cron-loop}}
{:name :_cron-loop
:parent :loop-until-interrupted ; ← :body is HOF-wrapped
:args {:body :_cron-step}}
{:name :_cron-step
:parent :do
:args {:steps [:_sleep-to-next :_fire-target]}}
{:name :_fire-target
:parent :call-noargs
:args {:func {:as :fn}}} ; ← `:fn` becomes a free-arg
When admin derives :my-cron :parent :schedule :args {:cron … :fn …}
and the reconciler starts it:
:schedule is invoked with :cron and :fn bound.:future HOF-wraps :_cron-loop and calls it as (body).(body args-map) — but we pass zero
args, so args-map is {}.:_cron-loop internally references :_cron-step which references
:_fire-target which expects free-arg :fn. With args-map = {},
:fn is unbound → runtime error.The :fn binding exists at admin’s :my-cron level — but the
HOF wrapper doesn’t carry it through. Inner free-args are treated as
call-site args even when no caller in the chain supplies them.
Each free-arg of a wrapped fn-graph belongs to ONE of two categories:
| Category | Source | Resolved when |
|---|---|---|
| call-site | Listed in the parent slot’s structural [:fn {args-shape} ret] type |
At each invocation, the executor passes the arg |
| captured | Free-arg NOT in args-shape |
At wrap time, executor captures the binding-chain |
For :map :func [:fn {:item a} b]:
:item is in args-shape → call-site (passed per element)For :future :body [:fn {} :any]:
args-shape = {} → no call-site argsFor :schedule composition:
:_cron-loop wraps :_cron-step (via :loop-until-interrupted):_cron-step references :_fire-target which uses :fn:fn is NOT in :loop-until-interrupted’s :body args-shape ({}):fn is a captured arg of the wrapped callable:my-cron :args {:fn …} binds it before
:future evaluates its :body arghof-wrap)When a fn-graph is HOF-wrapped, the executor:
[:fn ARGS _] shape.call-site-args, evaluates
the fn-graph with (merge captured call-site-args).Captured args resolve once, at wrap time — invariant per wrap. Call-site args resolve per invocation.
free-arg-slot-map)A composed fn-def’s free-args include:
For :schedule (fn-def):
:future: {:body} — bound to :_cron-loop:_cron-loop: {:cron, :fn} (transitive through
:_cron-step → :_fire-target):schedule level → both become free-args of :scheduleWhen admin derives :my-cron :parent :schedule :args {:cron … :fn …}:
:cron and :fn are now bound at :my-cron’s level:schedule are now resolved:my-cron has zero free-args → service-eligibleStructural fn-type [:fn {args-shape} ret effects] is unchanged. The
distinction between call-site and captured is computed at use-site
(when matching a binding’s value against the slot type’s structural
shape), not encoded in the type itself.
Two cooperating type-system changes ship with this extension:
Free-arg propagation (types/check.clj). ref-free-args
lifts a HOF-bound ref’s free args MINUS its authored :lambda-params
when present, otherwise the slot’s structural call-site arg names.
An explicit [] captures every free arg, including names matching
the slot. Callable type checking uses the same authored partition,
retaining the parameter types even when their names differ from the
slot’s names. Locally bound captures do not become public args again;
a local rename exposes only its destination name. The remainder
surface on the calling fn-def’s :args. Without this, :schedule’s
computed :args would be {} and the editor / parser would
have no way for admins to bind :cron / :fn.
fn-args subtyping (types/core.clj). fn-args-subtype?
accepts ANY sub arity when the sup slot is 0-arg. This is the
structural counterpart to wrap-time capture: a sub callable
with captured args still satisfies a 0-arg HOF slot — the wrap
passes nothing per call; captured args resolve from the binding
chain. Without this, :_cron-step (computed type
[:fn {:cron :text, :fn [:fn {} a]} :any]) would fail the
[:fn {} :any] subtype check at :loop-until-interrupted :body.
hof-wrap (executor/compile_eager.clj) is the captured-vs-call-site merge
point — outer-free-args snapshots the binding chain at wrap time,
lambda-params overlay per call. ALREADY structurally correct pre-
extension (verified in commit 3).
hof-lambda-params (executor/compile/renames.clj) IS the wrap-arity
dispatcher. It reads the slot’s structural shape (via
slot-structural-call-site-args) and chooses:
[:fn {} ret]) → 0 lambda-params → variadic-ignore
wrap. All sub free-args captured. The cron / :future :body /
:loop-until-interrupted :body path.[:fn {:x T} ret]) → structural name when R’s free
args include it (the fast path — sub-fn was written to match the
slot, e.g. :try’s :on-throw typed [:fn {:exception :any} a]
with _rollback declaring :exception); otherwise the
alpha-equivalence resolver picks the lambda-param name from R’s
non-captured frees — covers conventional positional callsites like
:filter :pred :some? (slot’s positional :item vs :some?’s
domain-named :value). Resolution is explicit-first: an authored
:lambda-params on the callable fn-def (registry-carried, like
:lazy-seq-args — the ordered call-site parameter list, []
meaning “everything captured”) overrides ALL inference and is
validated against R’s frees (:compile/invalid-lambda-params on a
typo). Without a declaration: an unambiguous single candidate that
is R’s OWN declared arg is accepted; anything else throws
:compile/ambiguous-lambda-params, naming the candidates. The
legacy one-shot guess (candidates all env-binding names somewhere →
variadic-ignore) is RETIRED: every callable the
packaged graph landed on it with — 72, almost all route/partial
handlers — now declares :lambda-params explicitly (values baked
from the guess’s own output, so compiled behaviour is unchanged;
tighten per-fn at leisure with compile feedback). Boundary
reminder unchanged: :http-server :handler is a genuine per-call
site (the request exists only at invocation — capture cannot feed
it; a uniform 0-arg flip broke /health), so handler declarations
keep :request in their params. In the editor the declaration is
the λ chip at the end of a composed fn’s return-type strip
(λ derived / λ [] / λ request; editor-edit-modes-flags.js),
written through PUT /api/entities/fn/:id lambda-params= — blank
clears back to derived, [] captures everything, a CSV is the
ordered list. Tutorial lesson 09 sets one.[:fn {:a A :b B} ret]) → sub free args matching
slot’s structural names. Map-callable; covers
:middleware :body ({:request _ :next-handler _}).:fn keyword constraint → REJECTED at compile time. Every
HOF slot must declare its callable shape structurally as
[:fn {ARGS} RET] — gives the type-checker something to constrain
on and makes the wrap-time dispatch deterministic.Without this dispatcher rewrite, the cron’s wrap would be built with
sub’s full free-arg count (e.g. 2-arg map-callable for :_cron-step),
and :future’s impl (body) would throw ArityException at runtime
— a bug the type system couldn’t catch.
{:name :my-mapper
:parent :map
:args {:item :_some-default}} ; ← binds `:item` (the call-site arg!)
:map’s :func slot declares :item as call-site. Admin bound :item
to :_some-default. What wins at invocation?
Rule: call-site arg WINS over captured / bound. :map’s impl
passes :item element per iteration; this overrides admin’s binding.
This matches Clojure semantics: (fn [item] item) always sees its
caller’s arg, regardless of a let that might happen to define a
local item later.
{:name :_floating
:parent :call-noargs
:args {:func {:as :missing-binding}}}
{:name :wrap-floating
:parent :future
:args {:body :_floating}}
:_floating requires captured arg :missing-binding. :wrap-floating
doesn’t bind it. At wrap-time, executor looks for :missing-binding
in the binding-chain and doesn’t find it.
Rule: captured arg unresolved at wrap time = the fn-def has
:missing-binding as a free-arg. Surfaces at validate-execute /
service-create / wherever free-args matter.
{:name :outer
:args {:my-thing "outer-value"} ; ← ancestor binding
:parent :wrap-floating} ; from previous example
Now :wrap-floating’s :_floating wants :my-thing (captured). The
binding-chain at wrap time includes :outer’s :my-thing "outer-value".
Captured.
Rule: captured arg resolution walks the standard binding-chain, same as any other resolved binding. Closer-fn-wins applies.
{:name :_html-error-body
:parent :str
:args {:parts [{:value "<p class=\"error\">"} {:as :reason} {:value "</p>"}]}}
{:name :html-error-response
:parent :assoc
:args {:map {…}
:key {:value :body}
:value :_html-error-body}} ; ← ref-target to :_html-error-body
{:name :_delete-err-fn-in-use
:parent :html-error-response
:args {:status {:value 409}
:reason {:as :fn-in-use-reason}}} ; ← renames R's free arg
:_html-error-body is reached via the :value ref-target of
:html-error-response. Inside its :str :parts it positionally
captures {:as :reason}, exposing :reason as a deep free arg
of :html-error-response. The descendant :_delete-err-fn-in-use
binds :reason {:as :fn-in-use-reason}, creating a renamed-view
slot on the descendant whose :source-slot-id FK points at
:_html-error-body.reason.
Rule: at compile time, build-ref-renames (in
executor/compile/renames.clj) resolves cross-tree renames in two
passes:
Same-name binding rename — F’s collected bindings contain a
:free row with a renamed :ext-name matching one of R’s deep
frees BY NAME. Covers e.g. :_pocb-rows-consumer.func {:as
:storage-query}, where F directly binds a slot R exposes under
the same name. Locked by
executor.compile.renames-test/build-ref-renames-translates-
renamed-slot.
Cross-tree slot-id rename — for any R-free not covered by
pass 1, walk R’s tree (inheritance + non-HOF ref-targets + seq
items) via find-slot-id-in-tree to find R’s slot-id, then ask
F’s inheritance chain via l/rename-for-slot whether it owns a
renamed-view slot whose :source-slot-id FK points at THAT
source. Picks up the :html-error-response-style case above —
where the rename’s source slot lives on a fn reached via
ref-target, not via inheritance. Locked by
executor.compile.renames-test/build-ref-renames-cross-tree-via-
slot-id.
The two passes are merged; the same-name pass wins on conflict.
The cross-tree pass relies on the parser ALSO emitting the empty
pure-rename binding row (signal that triggers
effective-binding’s direct-inheritance rename path in
executor/compile/bindings.clj); both mechanisms work side by
side.
:map / :filter / :reduce / middleware / router / Ring handlers —
all use HOF wrap. After this change:
Compatibility check: existing fn-defs that reference HOF args without binding the inner free-args were either:
No regressions expected for the existing HOF use cases. Specifically:
:request free-arg matches :handler’s
[:fn {:request _} _] call-site arg. Unchanged.:request /
:next-handler matching the structural type. Unchanged.:map / :filter: the wrapped fn’s :item matches the structural
type’s call-site arg. Unchanged.| # | Commit | What landed | Touches |
|---|---|---|---|
| 1 | 6d4c0f43 |
Spec + failing tests + regression baseline (3 :hof-regression invariants pinned) |
docs/CLOSURE_CAPTURE.md, test/graphden/types/closure_capture_test.clj |
| 2 | 62b276f9 |
free-arg-slot-map walks captured chain — service-eligibility recognizes :schedule’s captured args. Internal recursion via free-args-via threads a pre-queried bulk-entity db map to avoid re-fetching |
src/graphden/crud/fn_execution/lookup.clj |
| 3 | 7d309ce2 |
Verified hof-wrap already implements wrap-time capture per spec; added regression test + docstring pointer (no implementation change in compile.clj’s hof-wrap) |
src/graphden/executor/compile.clj (doc), test/graphden/types/closure_capture_test.clj |
| 4 | ecb99fc4 |
Type-checker propagates transitive captured free-args at HOF boundaries. New hof-call-site-arg-names; ref-free-args lifts ref-args - call-site |
src/graphden/types/check.clj, test/graphden/types/check_test.clj |
| 5 | a8e95f41 |
Re-composed :schedule as fn-def over :future / :loop-until-interrupted / :do / :sleep-until-ms / :cron-next-after / :call-noargs. Added :call-noargs + :sleep-until-ms base-fns. fn-args-subtype? accepts any sub arity for 0-arg sup slots. Type-level acceptance tests green |
src/graphden/types/core.clj, resources/packages/core/system/{fns.edn,impls.clj}, resources/packages/core/concurrency/{fns.edn,impls.clj}, test/graphden/types/closure_capture_test.clj |
| 6 | a44ca326 |
HOF regression audit — 279 tests across executor / type-system / reitit / crud / hof / collections / concurrency / fn-execution all clean. Retired final :closure-capture-pending placeholder (semantic structurally guaranteed by hof-wrap’s merge order) |
test/graphden/types/closure_capture_test.clj |
| 7 | eb5687d5 |
Runtime gap fix. Type-level acceptance was green but the end-to-end cron wouldn’t actually fire — hof-lambda-params computed wrap arity from sub’s free-arg count instead of slot’s structural shape, so the cron loop’s (body) got an N-arg map-callable wrap and threw ArityException. New slot-structural-call-site-args + 4-way dispatch (0-arg / 1-arg / 2+-arg / bare-:fn). :slot-id added to classify-slot output so the wrap site can read the slot’s structural type. New cron-schedule-runtime-test brings up the full Integrant system + creates a derived :_cron-runtime-target, watches the counter increment from inside the background thread, asserts the stopper unwinds cleanly |
src/graphden/executor/compile/{renames.clj,bindings.clj}, src/graphden/executor/compile.clj, test/graphden/integration/cron_schedule_runtime_test.clj (NEW) |
hof-wrap wraps a fn-graph
into a Clojure callable. Typically inside a parent fn’s binding
resolution (e.g. :map’s :func slot).(clojure.core/map fn coll)).[:fn ARGS _] shape.
Supplied per-invocation by the parent fn’s impl.