Graphden documentation

Lesson 09 — Higher-order functions and :fn-typed slots

Goal: by the end of this lesson you can write a fn-def that takes another fn as input, understand why :fn-typed slots behave differently from other slots, and distinguish per-invocation input (:request in a Ring handler) from closure capture (a :threshold retained by a :map callback).

Concepts introduced: :fn-typed slot, HOF, closure-capture, lambda-params, iterating vs one-shot HOF.

The two kinds of slot

So far every slot we’ve seen wanted a VALUE: a literal, a ref-fn-id (to invoke and use the result), a sequence of items. The executor resolves these when the implementation reads them. Reads are lazy: an if implementation can leave its unselected branch unevaluated, just as Clojure if does. A reference used as an ordinary value supplies the result of that computation, rather than a callable.

There’s a SECOND kind of slot: one that wants a CALLABLE, not a value. The slot’s type carries :fn as its outermost constructor:

{:type [:fn {:item a} b]}  ; a callable from `:item` of type `a` to `b`

When a slot is :fn-typed, the implementation receives a callable prepared by the executor, rather than a function ID. A reference to an ordinary function becomes a callable without running its body. A reference to a function whose return type is itself [:fn …] is evaluated to obtain that returned callable. The implementation then invokes the callable — once per element for :map, once with no input for :future, or as its own contract specifies.

Where a callable runs

The full callable type is [:fn args ret effects execution]. An execution profile identifies a backend: :jvm-server for the Clojure executor or :browser-main for the browser interpreter. The execution field records proven support, an unknown contract, or an execution variable. The shorter type forms above leave support unknown. A pure Clojure implementation is not therefore browser-compatible: the browser needs an implemented primitive ABI and a supported composition. Invoking a callback also requires support for that invocation; passing a callable as data is a separate operation. See TYPES.md — Callable execution contracts.

Authority means permission to perform the operation in the current account, organization and branch. It is separate from execution support and effect permissions: being able to read a definition does not grant permission to execute it, and declaring an effect does not authorize it.

core.system/server-call is an explicit browser-to-server call boundary. The current browser implementation supports a separate asynchronous entry for a checked, read-only server callable with statically closed configuration. The exported browser plan contains an opaque descriptor, not the server implementation or configured captures. The server checks the current source and execute permissions, context and typed input on every call. This is not a way to run arbitrary server functions in a browser expression. See the server-call contract.

:map — the canonical example

{:name :map
 :args {:func {:type [:fn {:item a} b]
               :description "Per-element transform."}
        :coll {:type [:list a]
               :description "Input collection."}}
 :return-type [:list b]}

:func is :fn-typed. Using it:

{:name :double-each
 :parent :map
 :args  {:func :double
         :coll [1 2 3]}}

{:name :double
 :parent :mul
 :args  {:nums [{:as :item} 2]}}

:double declares :item as its free arg via the {:as :item} rename. When :map runs, it iterates :coll and invokes :double once per element, passing each element AS :item. The result is [2 4 6].

If :func weren’t :fn-typed, :map would just receive :double’s already-executed value (which doesn’t make sense without an input) and crash.

:future — non-iterating HOF

:future runs a callable EXACTLY ONCE, in a background thread. Its :body slot is :fn-typed but the callable takes no input:

{:name :future
 :args {:body {:type [:fn {} :any]
               :description "Thunk to run in background."}}
 :return-type [:fn {} :null]   ; a stopper-thunk: call it to interrupt
 :effects #{:process}}

A user fn-def that wraps :future:

{:name :startup-task
 :parent :future
 :args  {:body :greet}}

{:name :greet
 :parent :const
 :args  {:value "starting up"}}

:future invokes :greet once, in a thread; the body’s result ("starting up") is discarded, and :startup-task returns the stopper-thunk.

The bind type-checks because fn-typed slots are subtype-checked with a covariant return — :greet statically returns :text, and [:fn {} :text] fits a slot wanting [:fn {} :any]; that is what lets a plain const-thunk drive :future in Lesson 35.

Invocation input: how :request propagates

This is the tricky part. Consider a Ring handler:

{:name :get-user
 :parent :json-handler
 :args  {:data :build-user-doc}}

{:name :build-user-doc
 :parent :assoc
 :args  {:map {} :key "id" :value :user-id-from-request}}

{:name :user-id-from-request
 :parent :get
 :args  {:coll {:as :request}
         :key  {:value :user-id}
         :default nil}}

:request is a free arg of :user-id-from-request. It needs to come from somewhere — but :build-user-doc doesn’t bind it, and neither does :get-user.

Where does :request come from? From the HOF call site at the top — :http-server invokes :get-user with `{:request

}`. This is **call-site input**: a new request supplied on each invocation, not a value captured when the handler was prepared. But how does it get FROM the top call site DOWN INTO `:user-id-from-request`, several refs deep? Free-argument analysis follows the references and exposes `:request` on `:get-user`. The handler's callable contract identifies it as a call-site parameter, so the request supplied on each call reaches the nested reader. **Closure capture** supplies a different category of input: a callback's free argument that comes from the enclosing caller rather than the HOF's per-call arguments. For example, a `:map` callback can receive a new `:item` for every element while retaining a `:threshold` supplied by its enclosing caller. The executor retains the enclosing inputs when preparing the callable and supplies its call-site parameters on each invocation. See [CLOSURE_CAPTURE.md](/CLOSURE_CAPTURE.html#specification) for the distinction and the parameter-selection rules below. ## A function that returns a function You can specialize a graph by inheritance and pass it directly as a callback. You can also construct a callable at execution time, just as in Clojure. Its declared return type tells the executor to evaluate the constructor when another function needs the returned callable. For example, an application can accept a database-query callable rather than connection parameters. A constructor captures the connection parameters; its returned function accepts one request record containing SQL and query parameters: ```edn {:name :query-sql :parent :get :args {:coll {:as :request :type {:sql :text :params [:list :jsonb]}} :key {:value :sql} :default nil} :return-type :text} {:name :query-params :parent :get :args {:coll {:as :request :type {:sql :text :params [:list :jsonb]}} :key {:value :params} :default nil} :return-type [:list :jsonb]} {:name :query-body :parent :sql-query :args {:sql :query-sql :params :query-params} :lambda-params [:request]} {:name :query-constructor :parent :const :args {:value {:type [:fn {:request {:sql :text :params [:list :jsonb]}} [:secret [:list :jsonb]] #{:db :network}]}} :return-type [:fn {:request {:sql :text :params [:list :jsonb]}} [:secret [:list :jsonb]] #{:db :network}]} {:name :make-query :parent :query-constructor :args {:value :query-body}} ``` First declare the open function-typed `:value` slot on `query-constructor`, then bind `query-body` on its child. Combining `:ref` and `:type` in one binding instead asserts a type for the referenced function's evaluated result; that is a different contract and fails this check. The typed slot prepares `query-body` as a callable; `const` returns that callable. `request` is its per-call input. The other free arguments — `url`, `user`, and `password` — belong to the constructor and are captured when it runs. Constructing the callable does not execute a SQL query; invoking it does. The callable's result keeps `:secret`: the SQL password is secret-typed, and the existing taint rules also mark the query result. Consumers must retain that contract; this constructor does not make SQL results public or bypass trace and result redaction. Separate `prod-query` and `test-query` functions can extend `make-query` with different connection values. The shared application takes their result in a function-typed argument. These are ordinary functions and references, with normal type and effect checks; they introduce no new environment entity. For merge behavior, see [configuration and branches](/tutorial/23-branches.html#what-doesnt-merge-branch-local-fn-defs). The existing `sql-query` implementation borrows an external connection from an executor-owned bounded pool when that facility is configured, with scope and credential isolation and session reset before reuse. The graph constructor does not create or own that pool. See [external SQL pools](/TUTORIAL_API_POLL.html). ## Iterating vs one-shot Two HOF flavours interact with closure-capture differently: | Flavour | Examples | Per-call input shape | |---|---|---| | **Iterating** | `:map`, `:filter`, `:update-vals` | The structural slot is named `:item` for sequences (`:map`, `:filter`), `:value` for the map key/value HOFs (`:update-vals`, `:update-keys`), and `:pair` for `:reduce` (the `[acc item]` vector); the callable's own same-named arg (or its single unambiguous free) receives the element | | **One-shot** | `:future`, `:assoc-fn`, `:invoke` | The slot contract determines input: `:future` supplies none (`{}`); `:assoc-fn` and `:invoke` use a generic `:arg`. The callable can declare its call-site parameters via `:lambda-params` (`[]` = everything captured). | When the callable has several candidate free args and no authored `:lambda-params`, the compile refuses with `:compile/ambiguous-lambda-params` naming the candidates — the old guessing heuristic is retired (it silently mis-wired captured callables). Declare the contract on the callable: ```clojure {:name :my-handler :lambda-params [:request] ; ← per-call inputs, in order ...} ``` In the editor the declaration is the **Call-site params** row of the Inspector's Overview (and, with *Details* on, the **λ chip** at the end of the card's signature strip): `λ derived` until you say otherwise, `λ []` for "everything captured", `λ request` for a named list. Click it on a fn you own: **Derived** / **None — []** / **These, in order** with a tick-box per free arg (the order you tick is the order they are filled). Clearing back to *Derived* is a Save away. Without that explicitness, a Ring handler whose ref chain happens to mention `:request` would have `:request` swallowed as the one-shot lambda input — breaking the wrap. When authoring a graph, declare which inputs the callback receives on each call if several free arguments could qualify. The remaining inputs are captured. When authoring a base implementation, follow the callable shape declared on its slot: the compiler uses that type together with `:lambda-params` to prepare the argument mapping. ## Try it > Prefer to be shown? This lesson exists as a guided in-editor tour: > [open the demo with the tour running](https://app.graphden.dev/?demo=1&tutorial=09) > (no sign-up), or pick “Interactive tutorial” in the editor's > account menu. 1. Type `map` in the Explorer filter and click the `map` row (`core.hof`). Read the two slots: the `:coll` chip reads `['a]` — a list of anything; the `:func` chip reads `(item:'a) → 'b` — a callable taking `:item` and returning something else. Only the second one is a HOF slot — and `'a` is the SAME unknown in both: whatever one slot learns about it, the other learns too. 2. Click `⋯` on the `map` row, choose **Extend**, name it `tutorial-map`, then **Save**. (`map` itself is package-owned, so its own slots are read-only — you customize by extending.) Both inherited slots show a `+` on your card. 3. **Bind what you know first — the data.** Click the `+` on `:coll`, **Append literal**, type `graph`, **Save**; the `:coll` edge now fans out after its chip — a branch to `graph` and a branch to a fresh `+`, the list's next free slot. Click it, **Append literal**, `den`, **Save**. The card reads `graph, den`. (An empty list's first `+` also offers **Bind fn-ref (whole list)**: the slot then takes one fn's *result* as the entire list — how a pipeline feeds `:map` with the output of `:str-split`; lesson 18 builds one.) 4. Now look at the `:func` chip: a moment ago it read `(item:'a) → 'b`; it reads `(item:text) → 'b`, and you never touched it. Two strings went into `:coll`, so `'a` is `text`, and the type variable carried that across to the slot you have not bound yet. Every value you set narrows the choices left. 5. Click the `+` on `:func`. Instead of the value form you saw in earlier lessons, the fn picker opens straight away: a literal is not a thing you can put in a callable slot. It states *Expected: (item:text) → b* — not `(item:a)`. Every fn the slot can take wears a ✓, grouped by namespace like the Explorer's tree, and the ones that take exactly one TEXT value come first in their group: `str-upper` from lesson 04 qualifies, a fn over numbers does not. Fns that would leave you more to wire carry an **Extra inputs** chip; constants that would ignore the item, an **Ignores the input** chip. Fns of other types are one toggle away at the bottom — and the moment you type a name, every match is listed, the incompatible ones dimmed with a ✗ (click one and the checker explains why). For a single-argument callable the argument's NAME does not matter — `map` hands each element to the callee's one free argument, whatever it is called. Type `str-upper` into the picker's filter: it is the **Exact match** on top; click its row. 6. An edge now runs from `str-upper` into `tutorial-map`'s `:func` — the callback is wired, and nothing has run. `⋯ → ▶ Run` → **Run** — nothing to fill in, every slot is bound: `["GRAPH" "DEN"]`. Two strings went in and each came out upper-cased: `str-upper` ran once per item, and you never called it — `map` did, handing each element in as its one argument. That is the whole HOF contract: the argument is passed unrun and the impl drives it. 7. `str-upper` is package-owned, so make a callable you can shape: filter for `str-upper`, `⋯` → **Extend** → `tutorial-upper` → **Save**. After a run the Inspector sits on its **Runs** tab — click **Overview**; it lists **Call-site params: λ derived**. 8. Click that row, choose **These, in order**, tick `string`, **Save**. It reads `λ string`: when a HOF hands `tutorial-upper` its argument, `string` receives it; anything else the fn had stays captured from the graph. With one free arg *derived* would have found the same answer — with two it would have refused, and this is where you say which. Notice the order you worked in: the data first, the callback last. Each value you bind is information about the slots still free, and the picker for the next one reads it — fewer, better candidates every step. Had you opened `:func` first, its picker would have said *(item:a) → b* and offered every one-argument fn in the graph (the reverse works too: binding `str-upper` first would have narrowed `:coll` to `[text]`). Bind what you know first, choose what you don't last; [docs/TYPES.md § Narrowing as a way of working](/TYPES.html#narrowing-as-a-way-of-working) has the rules and the limits. When a callable slot takes SEVERAL arguments, the picker matches by name, and a callee of your own has to expose args of those names; `{:as :item}` in `fns.edn` (or a rename on the edge label, lesson 04) is how you give it one. graphden's own test for `map`, `map-applies-the-callable-to-every-item` (`core.tests`), shows the named form: a small fn adding 1 to `:item`, over `[1 2 3]`. Graphden's `reduce` callback takes one `:pair` vector containing `[acc item]`; it does not receive two separate positional arguments. ### Going further (fns.edn / MCP only) An inline `{:parent …}` map in a `:fn`-typed slot anonymizes the callback (the named alternative is `:double` above): ```edn {:name :tutorial-double-each :parent :map :args {:func {:parent :mul :args {:nums [{:as :item} 2]}} :coll [1 2 3]}} ``` Run it: `[2 4 6]`. Replace the constant `2` with `{:as :factor}` and the card grows a free arg `:factor` — the Run pane asks for it before running. ## What we glossed over - **Effects propagating through HOF boundaries** — `:effects` declared on the callable lift onto the HOF's effective effect set. Lesson 16. - **The `:secret` type-marker** — how taint flows through HOF refs without spilling. Lesson 16. - **Pure-only callable slots** — a slot typed `[:fn … #{}]` (the trailing empty effect set: `:filter`'s `:pred`, `:swap`'s `:func`) accepts only a PURE callable; the picker ✗-marks an effectful one and the checker says why. Lesson 12 meets one. - **`hof-wrap` / `hof-lambda-params` source** — the actual Clojure code that implements the dispatch lives in `executor/compile_eager.clj` (`hof-wrap`) and `executor/compile/renames.clj` (`hof-lambda-params`) if you want to dig. ## Next [Lesson 10 — Composing pages from components](/tutorial/10-components-and-pages.html)